You are not logged in.

#1 2018-11-27 17:15:40

RainmakerRaw
Member
Registered: 2015-03-30
Posts: 11

[SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

Hi everyone. I have an issue I'm pretty sure is a routing/policy problem, but I only know enough about netwoking to be dangerous (still learning every day).

I have an x86_64 box I had been using for OPNsense as my home network's edge router and firewall for my 380/22 cable connection. It's a Pentium G4560 (2c4t @ 3.5GHz) with 4GB DDR4 and a 32GB mSATA SSD. After swapping it over to a base Arch install I've seen a nice little speed boost and web pages load in a much snappier manner. So far so good! I get my full ISP speed at very little (single digit) CPU usage. I also run WireGuard from the router to cover all devices on my LAN automatically. The Arch router runs dnscrypt-proxy as DNS server, dhcpd as dhcp server, Shorewall as firewall and NAT, and wireguard-tools/wireguard-dkms (providing wg-quick) for the VPN. My network topology is like this:

https://i.imgur.com/NS7AIMz.png

In Shorewall I defined four zones:

$FW ipv4
net ipv4
loc ipv4
wg ipv4

Interfaces:

net physical=enp0s31f6
loc physical=en0s3
wg

Policy is:

#SOURCE DEST            POLICY          LOGLEVEL        RATE    CONNLIMIT
$FW     net             ACCEPT
$FW     wg              ACCEPT
$FW     loc             ACCEPT
loc     $FW             ACCEPT
loc     wg              ACCEPT
loc     net             ACCEPT
wg      loc             ACCEPT
wg      $FW             ACCEPT
net     all             DROP            $LOG_LEVEL
# THE FOLOWING POLICY MUST BE LAST
all     all             REJECT          $LOG_LEVEL

This all works fine. Unfortunately my DNAT port forwards in /rules only works when the VPN is down and SNAT masquerades to the physical external interface. Once I bring up the WireGuard tunnel and set /etc/shorewall/snat to masquerade azirevpn's wg interface instead, external networks are no longer able to connect to the NAS/DiskStation for Plex, SABnzbd etc. This is pretty understandable to me, as its route is now the wg and not physical/ISP interface. However I could really do with a solution to this and I'm not sure how to go about it. Do I modify Shorewall to skip the NAS from the tunnel somehow? Add custom routing on the NAS itself via its Network > static routing GUI? Something else?

The whole Arch router works amazingly well. It's so lean and powerful and because I built it, I know just where everything goes and what to do to fix any issues. It's brought my knowledge on significantly (I've been using Linux and BSD for 15 or so years, but only properly started learning it when I moved to Arch!). Within my own LAN I can obviously still connect to Plex (for example) using 192.168.1.5 but this isn't any help when I'm away or overseas or something and want to access files or watch a movie. So, it'd be nice to solve the problem. Can anyone please offer some help or suggestions? TIA.

Last edited by RainmakerRaw (2018-12-10 03:03:28)

Offline

#2 2018-11-27 17:25:34

HalosGhost
Forum Moderator
From: Twin Cities, MN
Registered: 2012-06-22
Posts: 2,089
Website

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

I have removed the image tag from your post; please read the Code of Conduct that you agreed to when you signed up (please pay particular attention to the section regarding posting images on the BBS).

All the best,

-HG

Offline

#3 2018-11-27 17:31:15

RainmakerRaw
Member
Registered: 2015-03-30
Posts: 11

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

My apologies HG. I signed up for the forum three years ago and only started posting this week. I'd completely forgotten the image rule, as most forum software these days auto-resizes everything. Thanks for just removing the img tags and not the entire thing, I'll make sure I remember in future.
Best wishes.

Offline

#4 2018-11-27 17:35:04

HalosGhost
Forum Moderator
From: Twin Cities, MN
Registered: 2012-06-22
Posts: 2,089
Website

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

RainmakerRaw wrote:

I signed up for the forum three years ago and only started posting this week.

Ahh, I missed that (a good reminder that we mods are fallible). In that case, do read the Code of Conduct as it did not exist when you signed up, but does govern this community.

Best wishes.

And to you.

All the best,

-HG

Offline

#5 2018-11-28 03:34:18

RainmakerRaw
Member
Registered: 2015-03-30
Posts: 11

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

This is driving me a little crazy now. It's 3.20am and I'm still chewing this over trying to get it to work lol. I added snat and dnat hairpin rules for the LAN so at least I can hairpin to my domain from local machines (i.e. still access my NAS services using mydomain.com from local devices as before). Thinking about the WireGuard issue, and still thinking it's routing, I tried something new.

Rather than let wg-quick bring up the interface (with its associated route changes), I added the link manually:

ip link add dev azirevpn-uk1 type wireguard
ip address add dev azirevpn-uk1 10.0.0.1/19 #the address given in my conf file from Azire
wg setconf azirevpn-uk1 /etc/wireguard/azirevpn-uk1.conf #amended to comment out Address and DNS so this will work
ip link set up dev azirevpn-uk1

This now gave the Arch router/firewall access to both interfaces instead of just the WireGuard tunnel. If I did either

curl --interface enp0s31f6 ipinfo.io

or

curl --interface azirevpn-uk1 ipinfo.io

I would get a reply corresponding to that interface - either my ISP address or the AzireVPN endpoint address. Great! So now I just change /etc/shorewall/snat to masquerade to azirevpn-uk1 and double check the policy file to ensure wg and lan can communicate. Done... And there's no internet access on LAN clients again. Since the firewall now has access to both interfaces and both can access the net through their respective public IPs, I expected that using snat to azirevpn-uk1 for LAN clients as usual would still allow them to access the internet via the VPN interface, but still leave the firewall able to communicate on the ISP IP also - and thus handle incoming packets for the local servers on the NAS as per the dnat rules.

But I obviously messed something up, or it just won't work. Again, I invite any comments or help here. I've pored over shorewall.org's FAQ and I'm no further on. My hair is ever more grey haha.

Last edited by RainmakerRaw (2018-11-28 03:35:26)

Offline

#6 2018-12-10 02:59:28

RainmakerRaw
Member
Registered: 2015-03-30
Posts: 11

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

OK seeing as I got no replies here I spent the week learning about policy routing (aka source routing). This problem is now solved. For the benefit of anyone searching in future, I upgraded my router with a quad port Intel Pro 1000 VT server NIC to make my life easier (I could have used VLANs instead if I had a suitable switch). Now I have separated the DMZ and trusted LAN interfaces onto two separate router ports with their own separate subnets.

You need to set up dhcpd.conf (or dnsmasq) for the two subnets (trusted LAN, and DMZ/whatever) as you usually would. You also have to set up Shorewall's SNAT file at /etc/shorewall/snat like this:

MASQUERADE 192.168.1.0/24 azirevpn-uk1 #Trusted LAN
MASQUERADE 192.168.2.0/24 enp3s0f0 #DMZ/other

Then rather than using wg-quick (which messes with your default routing tables) you need to add the WireGuard tunnel manually:

ip link add dev azirevpn-uk1 type wireguard
ip address add dev azirevpn-uk1 10.1.2.3/24 #Replace this with the tunnel address from your provider's .conf file.
wg setconf /etc/wireguard/azirevpn-uk1.conf #Make SURE that the Address and DNS entries are commented out of this file if you generated it from your provider.
ip link set up dev azirevpn-uk1

Now we can add a second routing table to re-route only the trusted part of the LAN through the WireGuard VPN tunnel:

echo "10 vpn" >> /etc/iproute2/rt_tables
ip rule add unicast iif enp3s0f0 table vpn
ip route add default dev azirevpn-uk1 via 10.1.2.3 table vpn

Voila. Anything plugged into your 'trusted LAN' port will route seamlessly via the VPN. Anything in your DMZ/other port(s) will route via your usual ISP and thus be fully accessible from the rest of the internet using your usual domain name. I can now access my Plex and other servers from anywhere even though the rest of my LAN is safely behind a VPN.

The firewall can access both interfaces at will. Don't forget to set up /etc/shorewall/interfaces and /policy to reflect the new additional WireGuard connection ('wg ipv4' in /zones, 'azirevpn-uk1' underneath the other interfaces in /interfaces. You'll finally need to add some lines to /policy to accept communication between wg and $FW, and vice versa. I added policy for wg to loc and back also to be safe. My network now LOOKS LIKE THIS, and it all works.

The one remaining (small) issue/inconvenience is that the WireGuard interface and default route for the VPN table are lost on reboot. It's a few seconds using memory to bring up the azirevpn-uk1 interface and add the default route to the VPN table again, but I'll have to learn the way to automate it using systemd in future. I hope this helps someone, as it gave me quite a number of grey hairs to learn haha!

Last edited by RainmakerRaw (2018-12-10 03:05:17)

Offline

#7 2018-12-15 09:42:24

RainmakerRaw
Member
Registered: 2015-03-30
Posts: 11

Re: [SOLVED] Arch router (Shorewall) DNAT not working under WireGuard

Final post to resolve the issue entirely. I wrote a couple of shell scripts and a systemd unit to bring up the WireGuard tunnel and add the relevant routes to tunnel the LAN traffic via WireGuard, and to allow the LAN to talk to the DMZ subnet while connected. Everything is now available on Github, so if you find this by searching a similar issue in future, the files on my repo should help you.

Offline

Board footer

Powered by FluxBB