[SOLVED] How to perform iptables forwarding with two NAT servers?

nbd · 2019-01-07 20:05:03

Hi,
I setup a simple packet forwarding on a machine with two NICs, following ArchWiki:

https://wiki.archlinux.org/index.php/Internet_sharing

I intend it to be a firewall, but it has no firewall rules yet: just transparent packet forwarding. This machine has one NIC connected to my ISP and the other connected to a WiFi router:

[Wifi router]- - - - -[eth1]=[forwarding machine]=[ppp0]- - - - -[ISP]
|
|
|
[browser machine]

And I have a problem with connecting to archlinux.org site from a browser machine connected to Wifi router: in the browser status bar a line "Performing TLS handshaking..." appears and the site doesn't open. Other sites such as gmail.com, bing.com, etc. open normally. The 443 port is opened on Wifi router.

On the forwarding machine I have the following iptables rules:

root ~# iptables -nvL
Chain INPUT (policy ACCEPT 171K packets, 209M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 88 packets, 5601 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 239K  203M ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
 1947  104K ACCEPT     all  --  eth1   ppp0    0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 91987 packets, 10M bytes)
 pkts bytes target     prot opt in     out     source               destination

tcpdump on the forwarding machine shows the following:

root ~# tcpdump -i eth1
22:37:14.410156 IP myrouter.56971 > apollo.archlinux.org.https: Flags [S], seq 3665854004, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
22:37:14.412210 IP myrouter.56972 > apollo.archlinux.org.https: Flags [S], seq 1437676288, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0
22:37:14.463300 IP apollo.archlinux.org.https > myrouter.56971: Flags [S.], seq 778640681, ack 3665854005, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
22:37:14.464565 IP myrouter.56971 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, length 0
22:37:14.465225 IP apollo.archlinux.org.https > myrouter.56972: Flags [S.], seq 1466822147, ack 1437676289, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
22:37:14.466161 IP myrouter.56972 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, length 0
22:37:14.520608 IP apollo.archlinux.org.https > myrouter.56972: Flags [.], ack 169, win 237, length 0
22:37:14.520831 IP apollo.archlinux.org.https > myrouter.56971: Flags [.], ack 169, win 237, length 0
22:37:14.537392 IP apollo.archlinux.org.https > myrouter.56972: Flags [P.], seq 2921:4097, ack 169, win 237, length 1176
22:37:14.537586 IP apollo.archlinux.org.https > myrouter.56972: Flags [P.], seq 4097:4152, ack 169, win 237, length 55
22:37:14.539240 IP myrouter.56972 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, options [nop,nop,sack 1 {2921:4097}], length 0
22:37:14.539609 IP myrouter.56972 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, options [nop,nop,sack 1 {2921:4152}], length 0
22:37:14.543543 IP apollo.archlinux.org.https > myrouter.56971: Flags [P.], seq 2921:4097, ack 169, win 237, length 1176
22:37:14.543737 IP apollo.archlinux.org.https > myrouter.56971: Flags [P.], seq 4097:4152, ack 169, win 237, length 55
22:37:14.545073 IP myrouter.56971 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, options [nop,nop,sack 1 {2921:4097}], length 0
22:37:14.545474 IP myrouter.56971 > apollo.archlinux.org.https: Flags [.], ack 1, win 16425, options [nop,nop,sack 1 {2921:4152}], length 0

"myrouter" in the log is the Wifi router. Can someone advise from that information where the problem is?

EDIT: archlinux.org doesn't open from behind Wifi router neither on Windows (FireFox, IE11), nor on ArchLinux (FireFox). But it opens OK on the forwarding machine (Chromium).

EDIT2: It's needed to mention that the Wifi router works as NAT server, as well as forwarding machine. That is in this topology "NATting" is performed two times. Maybe this is the cause of problem?

Last edited by nbd (2019-01-11 03:45:39)

mich41 · 2019-01-09 22:13:39

Your FORWARD chain is moot, all rules are ACCEPT and the default policy is ACCEPT, might as well just have the policy and nothing more.
Check other tables (iptables -t nat/mangle/raw).

Run tcpdump on every interface of every machine involved (client, wifi, firewall) to see where packets are lost.

I think this belongs in "networking", not "kernel and hardware".

nbd · 2019-01-10 21:05:42

mich41 wrote:

Your FORWARD chain is moot, all rules are ACCEPT and the default policy is ACCEPT, might as well just have the policy and nothing more.
Check other tables (iptables -t nat/mangle/raw).

Thus was written in Wiki, other tables are empty.

It turns out that this problem with TLS handshake with only SOME sites (while https with others works OK) is well known:

https://www.thesslstore.com/blog/troubl … e-message/

But there may be different causes of it and the reason of differences is most probably in different server configurations. Is it so that some configurations are ideally correct and others are not ideally correct? If bing, google, reddit, youtube, stackoverflow are getting through two NATs OK and e.g. frozentux.net is not, then...?

Last edited by nbd (2019-01-10 21:06:32)

fukawi2 · 2019-01-10 22:04:21

Post your FULL iptables ruleset using `iptables-save`

nbd · 2019-01-10 22:13:44

@fukawi2

Below is the listing of iptables-save:

root ~# iptables-save
# Generated by iptables-save v1.6.0 on Fri Jan 11 01:10:28 2019
*filter
:INPUT ACCEPT [130673:151245679]
:FORWARD ACCEPT [345:23648]
:OUTPUT ACCEPT [56810:5953379]
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth1 -o ppp0 -j ACCEPT
COMMIT
# Completed on Fri Jan 11 01:10:28 2019
# Generated by iptables-save v1.6.0 on Fri Jan 11 01:10:28 2019
*nat
:PREROUTING ACCEPT [1034:95260]
:INPUT ACCEPT [219:46986]
:OUTPUT ACCEPT [415:45731]
:POSTROUTING ACCEPT [322:34275]
-A POSTROUTING -o ppp0 -j MASQUERADE
COMMIT
# Completed on Fri Jan 11 01:10:28 2019

fukawi2 · 2019-01-11 02:45:25

I see you're using a PPP connection. You may need to apply MSS Clamping [Note 0] rules to force the MSS lower.

To see what your MSS should be:

ping -Mdo -c3 -s1452 1.1.1.1

If you get ping replies, then 1452 is OK and you should run the command again, with a higher value for 1452 (eg, 1460). Find the highest number before you get "Message too long" or similar errors.

Once you find that number, add the following to your rules (replace 1452 with the appropriate number):

iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o ppp0 -j TCPMSS --set-mss 1452

I'm guessing yours might be 1452 [Note 1] but I'm quite likely wrong - experiment!

[Note 0] https://www.phildev.net/mss/blackhole_description.shtml
[Note 1] 1500 - 20 bytes for TCP - 20 bytes for IPv4 - 8 bytes for PPPoE = 1452)

nbd · 2019-01-11 03:32:34

@fukawi

My ISP actually using L2TP which has its own header, so the maximum MSS in my case is 1432:

root ~# ping -Mdo -c3 -s1432 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 1432(1460) bytes of data.
1440 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=20.5 ms
1440 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=20.4 ms
^C
--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 20.409/20.467/20.526/0.154 ms
root ~# ping -Mdo -c3 -s1433 1.1.1.1
PING 1.1.1.1 (1.1.1.1) 1433(1461) bytes of data.
ping: local error: Message too long, mtu=1460
ping: local error: Message too long, mtu=1460
^C
--- 1.1.1.1 ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1065ms

root ~# iptables -t mangle -A POSTROUTING -p tcp --tcp-flags SYN,RST SYN -o ppp0 -j TCPMSS --set-mss 1432

I run your command with 1432 instead of 1452 but unfortunately nothing has changed. Is the piece "--tcp-flags SYN,RST SYN" is correct syntax?

nbd · 2019-01-11 03:44:21

EDIT: with 1420 the problem has gone!

Many thanks fukawi! That was very cryptic problem.

fukawi2 · 2019-01-11 05:33:18

You're welcome. I've been bitten by that one several times over the years, and every time I spend hours running traceroutes and shouting at the DSL line before I remember about MSS Clamping. I had it recently and your post sounded very similar!

Arch Linux

#1 2019-01-07 20:05:03

[SOLVED] How to perform iptables forwarding with two NAT servers?

#2 2019-01-09 22:13:39

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#3 2019-01-10 21:05:42

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#4 2019-01-10 22:04:21

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#5 2019-01-10 22:13:44

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#6 2019-01-11 02:45:25

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#7 2019-01-11 03:32:34

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#8 2019-01-11 03:44:21

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

#9 2019-01-11 05:33:18

Re: [SOLVED] How to perform iptables forwarding with two NAT servers?

Board footer