Docker-Compose with Mullvad Wireguard & arbitrary service

I was able to make this work really easily in native Kubernetes pods, but lots of folks had been asking questions about getting Wireguard connected to an arbitrary service properly and safely that may not have the means to use that infrastructure. Below are my notes on making that dream a reality with only compose and a few minutes of trial and error.

 

This compose shows wireguard + qbittorrent with some useful notes in-line. The crux of it though is as follows:

  1. Move the exposed ports off the qbittorrent service definition, and into the wireguard definition
  2. Add network_mode: "service:wireguard" to force the containers to use the same interfaces.
version: "3.7"
services:
  wireguard:
    image: linuxserver/wireguard
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /appdata/config/wireguard-test/wg:/config
      - /lib/modules:/lib/modules
    ports:
      - 6881:6881
      - 6881:6881/udp
      - 8088:8088
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped

qbittorrent: image: linuxserver/qbittorrent container_name: qbittorrent environment: - PUID=1000 - PGID=1000 - TZ=Europe/London - UMASK_SET=022 #Remember to make this the same port as the exposed port - WEBUI_PORT=8088 volumes: - /appdata/config/wireguard-test/qbt:/config - /appdata/downloads:/downloads #"ports" moved to wireguard config restart: unless-stopped #use the wireguard interfaces instead network_mode: "service:wireguard"

In the wireguard wg0.conf configuration, you must add a route back to your host network only if you want to access things like webUIs from your host. If everything's in the same network, you can just leave this headless, too.

PostUp = ip route add 192.168.0.0/16 via $(ip route | grep default | awk '{print $3}')

[Interface]
PrivateKey = <MULLVAD KEY>
Address = <MULLVAD ADDRESS>
DNS = <MULLVAD DNS>
PostUp = DROUTE=$(ip route | grep default | awk '{print $3}'); HOMENET=192.168.0.0/16; HOMENET2=10.0.0.0/8; HOMENET3=172.16.0.0/12; ip route add $HOMENET3 via $DROUTE;ip route add $HOMENET2 via $DROUTE; ip route add $HOMENET via $DROUTE;iptables -I OUTPUT -d $HOMENET -j ACCEPT;iptables -A OUTPUT -d $HOMENET2 -j ACCEPT; iptables -A OUTPUT -d $HOMENET3 -j ACCEPT;  iptables -A OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT
PreDown = HOMENET=192.168.0.0/16; HOMENET2=10.0.0.0/8; HOMENET3=172.16.0.0/12; ip route del $HOMENET3 via $DROUTE;ip route del $HOMENET2 via $DROUTE; ip route del $HOMENET via $DROUTE; iptables -D OUTPUT ! -o %i -m mark ! --mark $(wg show %i fwmark) -m addrtype ! --dst-type LOCAL -j REJECT; iptables -D OUTPUT -d $HOMENET -j ACCEPT; iptables -D OUTPUT -d $HOMENET2 -j ACCEPT; iptables -D OUTPUT -d $HOMENET3 -j ACCEPT

[Peer] PublicKey = jHxY2OKpxjqAwWH4r1Pb2K6xDUDt087ivxpM1KpE0Ec= AllowedIPs = 0.0.0.0/0 Endpoint = <MULLVAD SERVER>:51820

Pretty simple, right? Here's the results of what you came here to see.

 

[email protected]:/# curl https://am.i.mullvad.net/connected
You are connected to Mullvad (server us32-wireguard). Your IP address is 206.217.xxx.xxx

 

If you're curious about the nitty gritty, here's the output from each containers interfaces & routes to give an illustration on how this works as if it were on the same host instead of dedicated network stacks:

From Wireguard

[email protected]:/# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 16:52 ?        00:00:00 s6-svscan -t0 /var/run/s6/services
root        32     1  0 16:52 ?        00:00:00 s6-supervise s6-fdholderd
root       265     1  0 16:52 ?        00:00:00 s6-supervise coredns
root       266     1  0 16:52 ?        00:00:00 s6-supervise wireguard
root       268   266  0 16:52 ?        00:00:00 bash ./run
root       270   265  0 16:52 ?        00:00:00 /app/coredns -dns.port=53
root       357   268  0 16:52 ?        00:00:00 sleep infinity
root       358     0  0 16:59 pts/0    00:00:00 bash
root       378   358  0 17:00 pts/0    00:00:00 ps -ef

[email protected]:/# ip route default via 172.24.0.1 dev eth0 172.24.0.0/16 dev eth0 proto kernel scope link src 172.24.0.2 192.168.0.0/16 via 172.24.0.1 dev eth0

[email protected]:/# ip address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 link/none inet 10.67.xxx.xx/32 scope global wg0 valid_lft forever preferred_lft forever 151: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:18:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.24.0.2/16 brd 172.24.255.255 scope global eth0 valid_lft forever preferred_lft forever

[email protected]:/# iptables-save

Generated by iptables-save v1.6.1 on Sat Aug 8 14:48:00 2020

*filter :INPUT ACCEPT [16:2307] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [17:1615] -A OUTPUT -d 192.168.0.0/16 -j ACCEPT -A OUTPUT -d 10.0.0.0/8 -j ACCEPT -A OUTPUT -d 172.16.0.0/12 -j ACCEPT -A OUTPUT ! -o wg0 -m mark ! --mark 0xca6c -m addrtype ! --dst-type LOCAL -j REJECT --reject-with icmp-port-unreachable COMMIT

Completed on Sat Aug 8 14:48:00 2020

Generated by iptables-save v1.6.1 on Sat Aug 8 14:48:00 2020

*mangle :PREROUTING ACCEPT [16:2307] :INPUT ACCEPT [16:2307] :FORWARD ACCEPT [0:0] :OUTPUT ACCEPT [19:1729] :POSTROUTING ACCEPT [19:1729] -A PREROUTING -p udp -m comment --comment "wg-quick(8) rule for wg0" -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff -A POSTROUTING -p udp -m mark --mark 0xca6c -m comment --comment "wg-quick(8) rule for wg0" -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff COMMIT

Completed on Sat Aug 8 14:48:00 2020

Generated by iptables-save v1.6.1 on Sat Aug 8 14:48:00 2020

*raw :PREROUTING ACCEPT [16:2307] :OUTPUT ACCEPT [19:1729] -A PREROUTING -d 10.67.xxx.xxx/32 ! -i wg0 -m addrtype ! --src-type LOCAL -m comment --comment "wg-quick(8) rule for wg0" -j DROP COMMIT

Completed on Sat Aug 8 14:48:00 2020

From qbittorrent

[email protected]:/# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 16:52 ?        00:00:00 s6-svscan -t0 /var/run/s6/services
root        32     1  0 16:52 ?        00:00:00 s6-supervise s6-fdholderd
root       250     1  0 16:52 ?        00:00:00 s6-supervise qbittorrent
abc        252   250  0 16:52 ?        00:00:02 /usr/bin/qbittorrent-nox --webui-port=8088
root       276     0  0 17:00 pts/0    00:00:00 bash
root       669   276  0 17:02 pts/0    00:00:00 ps -ef

[email protected]:/# ip route default via 172.24.0.1 dev eth0 172.24.0.0/16 dev eth0 proto kernel scope link src 172.24.0.2 192.168.0.0/16 via 172.24.0.1 dev eth0

[email protected]:/# ip address 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 3: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000 link/none inet 10.67.xx.xx scope global wg0 valid_lft forever preferred_lft forever 151: [email protected]: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:18:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 172.24.0.2/16 brd 172.24.255.255 scope global eth0 valid_lft forever preferred_lft forever

Sources:

https://nbsoftsolutions.com/blog/routing-select-docker-containers-through-wireguard-vpn