Setting up a Router with iptables.

apoletti · 2016-02-17 19:20:56

Hi all! I'm rather new to Arch Linux but have used Linux in general for many years. I recently embarked on a project I've been wanting to do for years but haven't had the money/time to really do it: replace my consumer-grade router (a Belkin) with a low-power mini-ITX Linux system. After trying several distributions that I wasn't very happy with since they did not work well with the hardware or seemed bloated, I decided to try Arch. I love the minimalistic approach as I get to install just what I need, which is great for security and performance. OK, enough about that, it was merely an introduction as this is my first post.

Anyway, I've got most of my setup working. Here is a description of what I have:

enp3s0: The LAN interface.
- IP: 10.64.32.1
- Network: 10.64.32.0/24
- Services:
  - SSH
  - DNS (using BIND)
  - DHCP (using ISC DHCP Server)
  - HTTP (using Apache)
enp0s20u1u3: The WAN interface.
- IP/Network: Dynamically assigned by my cable modem.
- I will eventually set up OpenVPN to allow remote access.
- Forward port 443 (HTTPS) from WAN to 10.64.32.10 on the LAN.
iptables Rules (the default rules section comes from Arch's simple_firewall.rules exmaple configuration): http://pastebin.com/qXAF3y2G

DNS and DHCP are working. Computers on the LAN side can access each other, access the services on the router, and connect to services on the Internet.However, I am having extreme difficulty trying to forward a port to a web server on the LAN. I have a computer with an IP of 10.64.32.10 that is listening on port 443. I attempted to add the appropriate rules to the NAT and Filter tables but it just wasn't getting through. I even tried fruitlessly to expose the various services on the router to the WAN. Don't tell anyone, but for a brief moment I even set the default policies to all be ACCEPT--it didn't work so I set them back. Another issue I'm having is that Wake-on-LAN packets are not being sent correctly; If I am on the router (via a KVM or SSH, no difference), the wol is not actually waking up the computer.

A bit more info about where I was about two days ago to give some context:
Before starting this project. I had my Belkin N600 router and a Raspberry Pi running my network. The router had DHCP disabled and forwarded port 443 to 10.64.32.10 and port 1194 (OpenVPN) forwarded to 10.64.32.2 (the Raspberry Pi). The Raspberry Pi had the DNS, DHCP, and HTTP services. Everything worked fine, including the WoL. My goal, before doing anything else new, is to basically use this new Arch Linux Router as a drop-in replacement for the Belkin N600 and Raspberry Pi.

I'm sorry if this has already been asked or if my verbosity is too much, but I have been searching for an answer for hours and I'm sure this is a trivial thing to do, I'm probably just missing something. Let me know if you need more information. Thank you so much for any advice you can give me.

brebs · 2016-02-17 20:08:08

For debugging, use e.g.:

* tcpdump, to see the packets - run tcpdump on the PCs involved.
* iptables rate-limited logging, to gain insight into how the iptables rules are working. E.g.:

iptables -A INPUT -m limit --limit 10/min --limit-burst 10 -j LOG --log-level warning --log-prefix "blahblah: "

apoletti · 2016-02-18 06:20:42

Thanks for your response! I installed tcpdump and attempted to use it to capture what comes in on the WAN port, but I feel I may be using it incorrectly or at least misinterpreting the results. I'll continue trying with it, but I was hoping someone would be able to look over my iptables configuration and point out any glaring issues with how I set up the rules and/or things I should add in to make it more secure. I tried as best as I could with what research I did to try and set up the rules. I would like to know why my rules are failing and why something might fix it. I look forward to your reply and to learn from my mistakes.

ewaller · 2016-02-18 15:43:03

I know you provided the rules you input to your firewall, but can we see what rules it actually is using? Post the output of sudo iptables -L --verbose

apoletti · 2016-02-18 17:25:52

OK Here is the output of iptables -L -v:

Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
24535 1126K ACCEPT     tcp  --  enp3s0 any     10.64.32.0/24        anywhere             tcp dpt:ssh
  168  8959 ACCEPT     tcp  --  enp3s0 any     10.64.32.0/24        anywhere             tcp dpt:domain
17266 1157K ACCEPT     udp  --  enp3s0 any     10.64.32.0/24        anywhere             udp dpt:domain
 2643 1486K ACCEPT     udp  --  enp3s0 any     10.64.32.0/24        anywhere             udp spt:bootpc dpt:bootps
  193 24588 ACCEPT     tcp  --  enp3s0 any     10.64.32.0/24        anywhere             tcp dpt:http
    0     0 ACCEPT     udp  --  enp0s20u1u3 any     10.64.32.0/24        anywhere             udp spt:bootps dpt:bootps
 4026  354K ACCEPT     icmp --  any    any     anywhere             anywhere            
 983K  111M ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED
 8977  614K ACCEPT     all  --  lo     any     anywhere             anywhere            
 1649  275K REJECT     tcp  --  any    any     anywhere             anywhere             reject-with tcp-reset
 4407 1058K REJECT     udp  --  any    any     anywhere             anywhere             reject-with icmp-port-unreachable
 3216  112K REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-proto-unreachable

Chain FORWARD (policy DROP 195 packets, 9896 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  26M   25G ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED
    0     0 ACCEPT     tcp  --  any    any     anywhere             10.63.32.10          tcp dpt:https
    0     0 ACCEPT     tcp  --  enp0s20u1u3 enp3s0  anywhere             10.63.32.10          tcp dpt:https
44928 7389K ACCEPT     all  --  enp3s0 enp0s20u1u3  anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 2386K packets, 2886M bytes)
 pkts bytes target     prot opt in     out     source               destination

Also, here is the output of iptables -t nat -L -v as I figured it would help:

Chain PREROUTING (policy ACCEPT 64183 packets, 9818K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  195  9896 DNAT       tcp  --  enp0s20u1u3 any     anywhere             anywhere             tcp dpt:https to:10.64.32.10:443

Chain INPUT (policy ACCEPT 16973 packets, 2442K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 31756 packets, 2388K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 32274 packets, 2409K bytes)
 pkts bytes target     prot opt in     out     source               destination         
34208 5780K MASQUERADE  all  --  any    enp0s20u1u3  10.64.32.0/24        anywhere

It looks like in the NAT prerouting chain the DNAT rule is being hit; however in the Filter forward chain the HTTPS requests do not look like they are going to 10.64.32.10:443, so I feel that is where the disconnect is but I'm not sure.

ewaller · 2016-02-18 17:43:49

I'm really new at this (Brebs is you better go to person) but, I think that

26M   25G ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED

in your FORWARD section is your problem.
First, it is catching everything that has been established and is not letting requests through to your redirect. I think that line should be at the beginning of INPUT and that it should not be in forward.
I also think you want a DROP or REJECT by default at the end of the FORWARD chain.

EDIT: No, I think I am wrong. Ignore that for now

Edit2: Try removing this line from the INPUT chain. I think it is killing all TCP connections from the net. I could be wrong

1649  275K REJECT     tcp  --  any    any     anywhere             anywhere             reject-with tcp-reset

Perhaps that should even be an accept.

Beware: I am just learning about iptables myself.

Last edited by ewaller (2016-02-18 18:05:30)

apoletti · 2016-02-18 18:01:10

Hmm, interesting. I forgot where I saw the forward rule being recommended, but I will try removing it for now. As a side note after this is all working I'd like to secure my IPv6 side of things. Unfortunately I don't know much about IPv6 so I'll likely need to research it. Perhaps for now I should disable it for my WAN adapter.

ewaller · 2016-02-18 18:06:07

I was editing while you were posting. See my second edit also.

brebs · 2016-02-18 18:07:48

ewaller is right, the "FORWARD -m conntrack --ctstate RELATED,ESTABLISHED" rule is nuts - remove it.

I see no mention of the essential:

sysctl -q -w net.ipv4.ip_forward=1

For "I would like to know why my rules are failing", I'll repeat: tcpdump and iptables logging are *hugely* useful. With iptables logging, you can see the crucial details of the packets, as iptables sees them.

apoletti · 2016-02-18 18:36:34

I have IP forwarding set to 1; sorry about not mentioning that.Well, unfortunately if I remove the forwarding rule, I lose Internet connectivity; I had to add it back in so I could post this reply .

branch · 2016-02-18 19:34:59

ewaller wrote:

I'm really new at this (Brebs is you better go to person) but, I think that
26M   25G ACCEPT     all  --  any    any     anywhere             anywhere             ctstate RELATED,ESTABLISHED
in your FORWARD section is your problem.
First, it is catching everything that has been established and is not letting requests through to your redirect. I think that line should be at the beginning of INPUT and that it should not be in forward.

brebs wrote:

ewaller is right, the "FORWARD -m conntrack --ctstate RELATED,ESTABLISHED" rule is nuts - remove it.

That line is straight from the wiki [1]. I'm not sure I understand what ewaller is saying. The rule in question is a -j ACCEPT, so it is allowing packets through, not "catching" them. A similar rule is commonly put on the INPUT chain also for connections to/from the current host, but the OP's problem seems to be with connections forwarded from the WAN to the LAN.

apoletti wrote:

It looks like in the NAT prerouting chain the DNAT rule is being hit; however in the Filter forward chain the HTTPS requests do not look like they are going to 10.64.32.10:443, so I feel that is where the disconnect is but I'm not sure.

Spot on. PREROUTING DNAT rule matched 195 packets for 9896 bytes. FORWARD chain default DROP policy dropped 195 packets for 9896 bytes. The rule

-A FORWARD -i enp0s20u1u3 -o enp3s0 -p tcp -d 10.63.32.10 --dport 443 -j ACCEPT

is not matching the DNAT'ed packets. I note in the wiki [1] that the corresponding rule does not have the -i and -o matches, so I would try removing those.

EDIT: Compare 10.63.32.10 in the FORWARD rule with 10.64.32.10 in the DNAT rule.

[1] https://wiki.archlinux.org/index.php/Si … WARD_chain

Last edited by branch (2016-02-18 19:43:33)

apoletti · 2016-02-18 21:00:05

Yes! I got the port forwarding to work, and I think it was mostly due to the typo of 63 vs 64 I had. Anyway, my updated iptables configuration is here: http://pastebin.com/7sezsGAd. There is one more thing I have to get working, though I'm not sure what would be preventing it since the Filter output chain is set to accept by default, and that is sending a WoL packet from my router (the box we're working on) to 10.64.32.10 (obviously it has no IP when it's off, this is merely for ID purposes). The command I'm using is wol -i 255.255.255.255 D0:50:99:85:52:36. It was working before from the Raspberry Pi, so I'm thinking the issue is coming from the fact that I have two NICs but I'm not sure. By the way, thank you again for all the help you're giving me; I've learned so much over these last few days.

brebs · 2016-02-18 21:31:04

Sorry, I was overly harsh about "FORWARD -m conntrack --ctstate RELATED,ESTABLISHED" - it's probably OK, albeit subtle (is it considering the external-to-router route, or internal-to-router route, or external-to-interior route, or interior-to-exterior route, or all of them?). FORWARD is more complex than INPUT/OUTPUT.

WoL packet

See socat answer. Not as simple as hoped.

branch · 2016-02-18 22:01:58

brebs wrote:

Sorry, I was overly harsh about "FORWARD -m conntrack --ctstate RELATED,ESTABLISHED" - it's probably OK, albeit subtle (is it considering the external-to-router route, or internal-to-router route, or external-to-interior route, or interior-to-exterior route, or all of them?). FORWARD is more complex than INPUT/OUTPUT.

The flow chart [1] may help. It shows that ingress packets go to either INPUT or FORWARD, and egress packets come from either OUTPUT or FORWARD (assuming the XFRM features are not in use). So rules in the FORWARD chain apply to external-to-internal and internal-to-external, but not to external-to-router, router-to-external, internal-to-router, or router-to-internal.

[1] https://upload.wikimedia.org/wikipedia/ … t-flow.svg

fukawi2 · 2016-02-18 22:06:55

brebs wrote:

Sorry, I was overly harsh about "FORWARD -m conntrack --ctstate RELATED,ESTABLISHED" - it's probably OK,

It's not only OK, it's pretty much a required standard in a stateful-firewall (which iptables is)

apoletti · 2016-02-19 00:17:39

brebs: I'm not sure it needs to be that complicated; I just want to send a magic packet to that MAC address over the LAN NIC from the router itself. It doesn't seem like a big thing to accomplish.

branch · 2016-02-19 02:04:50

I don't see anything in the iptables rules that would filter egress WOL packets. I bet there is something about the packet that the RPi doesn't like. This is where tcpdump may help. Verify the packets are going out enp3s0, and capture a full packet with link header. Then plug the old router back in to the network and capture a WOL packet that it sends for comparison. Might your RPi require a WOL password?

In case you didn't find it before, the documentation for the tcpdump filter language is under "man pcap-filter".

apoletti · 2016-02-19 02:37:11

I'll try to use tcpdump to capture the WoL packet. Actually, you're misinterpreting my setup: the Raspberry Pi used to be the device that sent the WoL packet. One difference I just realized was on the Raspberry Pi I used a program called wakeonlan but on Arch I have wol. Maybe they send something slightly differently. The box receiving the magic packet is a regular desktop system.

Arch Linux

#1 2016-02-17 19:20:56

Setting up a Router with iptables.

#2 2016-02-17 20:08:08

Re: Setting up a Router with iptables.

#3 2016-02-18 06:20:42

Re: Setting up a Router with iptables.

#4 2016-02-18 15:43:03

Re: Setting up a Router with iptables.

#5 2016-02-18 17:25:52

Re: Setting up a Router with iptables.

#6 2016-02-18 17:43:49

Re: Setting up a Router with iptables.

#7 2016-02-18 18:01:10

Re: Setting up a Router with iptables.

#8 2016-02-18 18:06:07

Re: Setting up a Router with iptables.

#9 2016-02-18 18:07:48

Re: Setting up a Router with iptables.

#10 2016-02-18 18:36:34

Re: Setting up a Router with iptables.

#11 2016-02-18 19:34:59

Re: Setting up a Router with iptables.

#12 2016-02-18 21:00:05

Re: Setting up a Router with iptables.

#13 2016-02-18 21:31:04

Re: Setting up a Router with iptables.

#14 2016-02-18 22:01:58

Re: Setting up a Router with iptables.

#15 2016-02-18 22:06:55

Re: Setting up a Router with iptables.

#16 2016-02-19 00:17:39

Re: Setting up a Router with iptables.

#17 2016-02-19 02:04:50

Re: Setting up a Router with iptables.

#18 2016-02-19 02:37:11

Re: Setting up a Router with iptables.

Board footer