WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

ectospasm · 2021-12-02 03:29:14

I have an Intel NUC with dual Gigabit Ethernet interfaces, that I use as my home router running Arch Linux (hostname is barbican, after the fortified gate of a castle). I'm trying to set up WireGuard, according to the Arch Wiki WireGuard article. The router is what I've set up as Peer A, and it uses systemd-networkd as its network manager. It also runs firewalld, BIND, and the ISC DHCP server. The WireGuard interface on this router is wg0, and that interface is in the firewalld home zone, which also has my LAN interfaces. Here is the home zone:

home (active)
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: enp4s0 wg0
  sources:
  services: dhcpv6-client dns mdns samba-client ssh
  ports:
  protocols:
  forward: no
  masquerade: yes
  forward-ports:
  source-ports:
  icmp-blocks:
  rich rules:

enp4s0 is my LAN interface. In general devices in the home zone get DHCP leases and use the BIND service to resolve DNS. This works pretty well.

Now back to WireGuard. I have a laptop running Arch Linux that uses NetworkManager as its network management system (its DNS name is radon when connected through WiFi on the LAN). It can connect to WireGuard when I'm remote (I'm trying to set it up as Peer B, in a point-to-site configuration), the DNS server gets set to the barbican WireGuard IP address (10.11.12.254) as seen in /etc/resolv.conf, but the routes to the WireGuard subnet do not appear to work (pinging barbican on its WireGuard address gets 100 percent packet loss).

Since I'm home now, I'm trying to simulate being remote by connecting radon's WiFi to my mobile phone hot spot, to try and get this working (I originally started setting this up while traveling over the holiday weekend, I have an external SSH port forwarded to a stationary host within my LAN, that I then used to log into barbican). Here's what I see when running

ip route

while connected to WireGuard in this fashion:

default via 192.168.177.146 dev wlp4s0 proto dhcp metric 600
10.11.12.0/24 dev wg0 proto static scope link metric 50
10.11.12.0/24 dev wg0 proto kernel scope link src 10.11.12.3 metric 50
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.177.0/24 dev wlp4s0 proto kernel scope link src 192.168.177.218 metric 600

It looks like 192.168.177.146 is my mobile phone, which is the only thing I can ping at this point. If I try pinging barbican's wg0 address (10.11.12.254), I get no response. Here's barbican's (Peer A's) WireGuard/systemd-network configuration:

/etc/systemd/network/30-wg0.netdev:
[NetDev]
Name=wg0
Kind=wireguard
Description=WireGuard tunnel wg0

[WireGuard]
ListenPort=31987
PrivateKey=redacted

[WireGuardPeer]
PublicKey=jHlgmOJuxLe/XdaX+X/x0INqJknylHC+TR37idn16kg=
PresharedKey=redacted
AllowedIPs=10.20.30.0/24
AllowedIPs=10.11.12.0/24


/etc/systemd/network/30-wg0.network:
[Match]
Name=wg0

[Network]
Address=10.11.12.254/24

[Route]
Destination=10.11.12.0/24
Scope=link

[Route]
Destination=10.20.30.0/24
Scope=link

[Route]
Destination=172.16.87.0/24
Scope=link

Here is my NetworkManager configuration on radon/Peer B:

/etc/NetworkManager/system-connections/wg0.nmconnection:
[connection]
id=wg0
uuid=32b49f2b-1b8f-4d6b-a782-2df3cc335cd8
type=wireguard
autoconnect=false
interface-name=wg0
permissions=
timestamp=1638411474

[wireguard]
listen-port=31987
private-key=redacted

[wireguard-peer.kBa6+51y56T5zaJ03T4e86ZmgIwBZRla+fpKxngsJHE=]
endpoint=redacted.dyndns.org:31987
preshared-key=redacted
preshared-key-flags=0
allowed-ips=10.11.12.254/24;0.0.0.0/0;::/0;

[ipv4]
address1=10.11.12.3/24
dns=10.11.12.254;
dns-priority=-50
dns-search=~.;ceti;
method=manual
route1=10.11.12.0/24,10.11.12.254
route2=10.20.30.0/24,10.11.12.254

[ipv6]
addr-gen-mode=stable-privacy
address1=fdc9:281f:4d7:9ee9::3/64
dns-priority=-50
dns-search=
method=manual

[proxy]

I'm not sure if the redacted hostname for the barbican endpoint will work (that normally points to the WAN interface). My WAN IP address changes every time I reboot barbican, I haven't figured out how to have the systemd-networkd client request the same DHCP lease when it boots up (that's a problem for another day). My ultimate goal is to be able to access all hosts on the 10.11.12.0/24 (WireGuard) and 10.20.30.0/24 (LAN) networks, as well as tunnel all traffic to 0.0.0.0/0 (WAN) through the barbican WAN interface.

It feels like I'm missing something simple, but I haven't been able to figure out what it is. I read through this systemd bug report (which is linked to on the Arch Wiki WireGuard article) and I got totally confused. Is my problem with my barbican router? Or is it with radon? I see stuff about firewall marks and routing policy rules, but I don't know where to begin with that. I am not familiar with ip-route2 routing table intricacies, and I thought NetworkManager handles a lot of this stuff automatically.

Last edited by ectospasm (2021-12-02 03:35:21)

Xyne · 2021-12-02 04:27:42

What's the output of

wg

on barbican and radon? That will immediately tell you if the wireguard connection has actually been established.

Try a simple setup with wg-quick to isolate the initial configuration from systemd-networkd.

Peer A (barbican)

[Interface]
Address = 10.0.0.1/24
ListenPort = 51871
PrivateKey = PEER_A_PRIVATE_KEY

[Peer]
PublicKey = PEER_B_PUBLIC_KEY
PresharedKey = PEER_A-PEER_B-PRESHARED_KEY
AllowedIPs = 10.0.0.2/32

Peer B (radon or the wireguard app on your phone)

[Interface]
Address = 10.0.0.2/24
ListenPort = 51871
PrivateKey = PEER_B_PRIVATE_KEY

[Peer]
PublicKey = PEER_A_PUBLIC_KEY
PresharedKey = PEER_A-PEER_B-PRESHARED_KEY
AllowedIPs = 10.0.0.1/32
Endpoint = PEER_A_IP_ADDRESS:51871

First try setting PEER_A_IP_ADDRESS in Peer B's configuration file to barbican's current address on your internal network (whichever one it shares with your phone and radon). I would recommend installing the official wireguard app on your phone so that you're not trying to debug two instances of wg-quick. You can convert Peer B's configuration file to a scannable QR code with qrencode:

qrencode -t ANSIUTF8 -r /path/to/Peer_B_config

.

Check the output of wg on barbican and the connection status in the app. If that works, you'll know that it's a systemd-networkd error, If it doesn't, then there's something wrong with your general network settings.

Sanity checks

Double-check that the wireguard-service is running without error (systemctl status wg-quick@your_wg_name.service)
Have you opened the wireguard port on barbican for UDP connections? If you can't create a wireguard connection to barbican on the internal network then try disabling to your firewall to double-check that it's not a firewall issue.
Double-check that the endpoint addresses correspond to the internal non-wireguard addresses for this test, especially if any of them are dynamic.
For external connections, if you have an intermediate router between barbacan and the external network, double-check that it is configured to forward UDP traffic to barbican's wireguard port at barbican's current internal address.

Setting up DNS and tunnelling should be simple once the wireguard connection is established.

ectospasm · 2021-12-05 20:21:21

OK, I've confirmed that my mobile phone can connect to barbican's wg0 using the official WireGuard app:

# wg                                                                                                                        
interface: wg0
  public key: kBa6+51y56T5zaJ03T4e86ZmgIwBZRla+fpKxngsJHE=
  private key: (hidden)
  listening port: 31987

peer: jHlgmOJuxLe/XdaX+X/x0INqJknylHC+TR37idn16kg=
  preshared key: (hidden)
  endpoint: 172.58.78.228:31582
  allowed ips: 10.11.12.3/32, fdc9:281f:4d7:9ee9::3/128
  latest handshake: 1 minute, 30 seconds ago
  transfer: 5.05 KiB received, 6.10 KiB sent

However, DNS doesn't appear to work for hosts on my LAN, and I don't have access to the 10:20.30.0/24 network (my main internal LAN). I'm guessing this is because I don't have 10.20.30.0/24 in the AllowedIPs section. If I add it, and try to restart wg-quick@wg0.service, it fails because this route already exists. Here's my barbican:/etc/wireguard/wg0.conf:

[Interface]
Address = 10.11.12.254/24
ListenPort = 31987
PrivateKey = SEghIAJyxSaHO+rb2pr27rchL/ml7ci92UcDYpCPc3g=

[Peer]
PublicKey = jHlgmOJuxLe/XdaX+X/x0INqJknylHC+TR37idn16kg=
PresharedKey = FY7T7JBecDhJxtV2Fupc0ZFjXWIXBvExZV31mR6TSSc=
AllowedIPs = 10.11.12.3/32, fdc9:281f:4d7:9ee9::3/128, 10.20.30.0/24

If I have that, wg-quick@wg0.service fails to start, with RNETLINK answers: file exists when trying to add a route for 10.20.30.0.24:

Dec 05 15:05:16 barbican wg-quick[2199064]: [#] wg setconf wg0 /dev/fd/63
Dec 05 15:05:16 barbican wg-quick[2199064]: [#] ip -4 address add 10.11.12.254/24 dev wg0
Dec 05 15:05:16 barbican wg-quick[2199064]: [#] ip link set mtu 1420 up dev wg0
Dec 05 15:05:16 barbican wg-quick[2199064]: [#] ip -6 route add fdc9:281f:4d7:9ee9::3/128 dev wg0
Dec 05 15:05:17 barbican wg-quick[2199064]: [#] ip -4 route add 10.20.30.0/24 dev wg0
Dec 05 15:05:17 barbican wg-quick[2199098]: RTNETLINK answers: File exists
Dec 05 15:05:17 barbican wg-quick[2199064]: [#] ip link delete dev wg0
Dec 05 15:05:17 barbican systemd[1]: wg-quick@wg0.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 05 15:05:17 barbican systemd[1]: wg-quick@wg0.service: Failed with result 'exit-code'.
Dec 05 15:05:17 barbican systemd[1]: Failed to start WireGuard via wg-quick(8) for wg0.

Here's my current routing table, per "ip route" when wg0 does not exist:

default via 69.131.144.1 dev enp3s0 proto dhcp src redacted metric 1024
10.20.30.0/24 dev enp4s0 proto kernel scope link src 10.20.30.254
69.131.144.0/21 dev enp3s0 proto kernel scope link src redacted metric 1024
69.131.144.1 dev enp3s0 proto dhcp scope link src redacted metric 1024
172.16.87.0/24 dev enp4s0.66 proto kernel scope link src 172.16.87.254
216.165.129.158 via 69.131.144.1 dev enp3s0 proto dhcp src redacted metric 1024
216.170.153.146 via 69.131.144.1 dev enp3s0 proto dhcp src redacted metric 1024

I redacted my public IP address, which is likely to change with the next reboot.

I also can't confirm that I'm tunneling all of my traffic through the WireGuard VPN, my method for returning my public IP address doesn't appear to do anything.

Xyne · 2021-12-05 21:35:06

There seems to be some confusion about what the AllowedIPs setting does. It tells the host which peer to use for which addresses. In your case, adding 10.20.30.0/24 to AllowedIPs on one of barbican's peers tells barbican to try to route all traffic to 10.20.30.0/24 through that peer. That won't work because barbican still needs route the underlying physical traffic through the local network, so it needs to go out through the physical interface. There's already a route for that, which is why you're getting the error.

If you want to reach your internal network from outside of your lan, create a wireguard configuration for your remote client (e.g. radon, when not on the local network) and add barbican as a peer, with the Endpoint set to your local networks external IP and AllowedIPs set to 10.20.30.0/24.

To reach remote clients from the internal network you have to use their wireguard addresses. Local clients need to be configured to route their non-Wireguard traffic to 10.11.12.0/24 or whatever through barbican. The easier alternative is to add all local clients to the same wireguard network and let barbican manage the forwarding between wireguard clients.

For DNS you need to open port 53 and for forwarding traffic from your WireGuard network you need to set up forwarding and postrouting on the hub (barbacan). I have lines similar to the following in my hub's wg-quick config:

# Open UDP port 53 on wg0.
PostUp = /usr/bin/iptables -A INPUT -i %i -m udp -p udp --dport 53 -j ACCEPT
PreDown = /usr/bin/iptables -D INPUT -i %i -m udp -p udp --dport 53 -j ACCEPT

# Open TCP port 53 on wg0.
PostUp = /usr/bin/iptables -A INPUT -i %i -m tcp -p tcp --dport 53 -j ACCEPT
PreDown = /usr/bin/iptables -D INPUT -i %i -m tcp -p tcp --dport 53 -j ACCEPT

# Forward traffic from wg0 to eth0.
PostUp = /usr/bin/iptables -A FORWARD -i %i -o eth0 -j ACCEPT
PostUp = /usr/bin/iptables -A FORWARD -o %i -i eth0 -j ACCEPT
PostUp = /usr/bin/iptables -t nat -A POSTROUTING -s 10.11.12.0/24 -o eth0 -j MASQUERADE
PreDown = /usr/bin/iptables -D FORWARD -i %i -o eth0 -j ACCEPT
PreDown = /usr/bin/iptables -D FORWARD -o %i -i eth0 -j ACCEPT
PreDown = /usr/bin/iptables -t nat -D POSTROUTING -s 10.11.12.0/24 -o eth0 -j MASQUERADE

If you want to forward from the local network to the WireGuard network (i.e. clients go through barbican to reach the remote clients instead of running wireguard themselves), you probably need to add

PostUp = /usr/bin/iptables -t nat -A POSTROUTING -s 10.20.30.0/24 -o %i -j MASQUERADE
PreDown = /usr/bin/iptables -t nat -A POSTROUTING -s 10.20.30.0/24 -o %i -j MASQUERADE

I haven't tested that though as I prefer to have all traffic go through full wireguard tunnels without insecure local hops. The forwarding from wg0 to the local network in my case is only for using the local network as a vpn to reach the internet, not the LAN.

You also need to enable forwarding via sysctl:

sysctl -w net.ipv4.ip_forward=1 # IPv4
sysctl -w net.ipv6.conf.all.forwarding=1 # IPv6

Create configuration files for persistent configuration: https://wiki.archlinux.org/title/Sysctl#Configuration

ectospasm · 2021-12-06 02:13:36

I got it working with wg-quick. On barbican:

interface: wg0
  public key: kBa6+51y56T5zaJ03T4e86ZmgIwBZRla+fpKxngsJHE=
  private key: (hidden)
  listening port: 31987

peer: WNKHtuRqPUilZwGE8XkNddG892ZGwADDjnNWUDsj2gg=
  preshared key: (hidden)
  endpoint: 172.58.155.144:59219
  allowed ips: 10.11.12.1/32
  latest handshake: 25 minutes, 54 seconds ago
  transfer: 98.02 KiB received, 144.06 KiB sent

The peer is my laptop while connected through the hotspot on my mobile phone. Here's the output of sudo wg while my laptop is connected:

interface: wg0
  public key: WNKHtuRqPUilZwGE8XkNddG892ZGwADDjnNWUDsj2gg=
  private key: (hidden)
  listening port: 31987
  fwmark: 0xca6c

peer: kBa6+51y56T5zaJ03T4e86ZmgIwBZRla+fpKxngsJHE=
  preshared key: (hidden)
  endpoint: 96.60.55.199:31987
  allowed ips: 10.11.12.0/24, 10.20.30.0/24, 0.0.0.0/0
  latest handshake: 1 minute, 33 seconds ago
  transfer: 1.35 MiB received, 358.07 KiB sent

I can resolve hostnames on my internal network, and can ping everything there. I can also ping the WAN, but I can't tell if my requests are going over wg0. This is my usual command I use to resolve my public IP address:

dig +short @resolver1.opendns.com myip.opendns.com

But it doesn't return anything. If this were to work as expected, I expect to see the WAN IP address of barbican output by that command. However, traceroute does confirm the first hop is the wg0 interface on barbican, so it's working as expected!

I'd mark this as resolved, but I want to get this working with systemd-networkd (on barbican), and NetworkManager (on radon). At least I know my WireGuard configuration works on a basic level, in regards to firewalld, DNS, and wg.

Arch Linux

#1 2021-12-02 03:29:14

WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

#2 2021-12-02 04:27:42

Re: WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

#3 2021-12-05 20:21:21

Re: WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

#4 2021-12-05 21:35:06

Re: WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

#5 2021-12-06 02:13:36

Re: WireGuard woes (Peer A: systemd-networkd, Peer B: NetworkManager)

Board footer