complex ip-masquerading failure

Tyris · 2007-04-19 04:02:34

Hi guys,

I've got a rather unusual ip_masquerading / ip_forwarding problem in which a formerly stable gateway machine no longer wants to forward http requests for hosts inside my home network.

The problem machine has two internal ethernet ports, I use 1 for internal LAN, one for WAN to the ISP. I've recently also added an atheros wireless card in a free PCI slot but have not configured it and have backed out any kernel changes I did to reduce variables. When setup as the gateway, others on the internal network see the following symptoms:

1. not able fetch http requests in firefox or wget
2. SSH works
3. SCP works
4. telnet www.cnn.com 80 -- sorta works.

I have an established rc.firewall file that I've been using for 5 years now, and it (something!) suddenly fails me.
Internal networked machines have had no configuration changes (1 laptop, one mythtv box) and are known good.

I do have an older machine which takes my rc.firewall and routes as a gateway without issue, internal machines run just great with the old server is acting as gateway.

Now here's where Im absolutely stumped. Regardless of kernel (both my own or 2.6.20.7-arch), I am not able to get normal gateway operation when doing http requests. All other network requests work for the internal machines. Firefox/http requests run great on the server/gateway machine.

My now super-streamlined rc.firewall reads:

## Setup firewall -- clear all rules, start clean
/usr/sbin/iptables -F
/usr/sbin/iptables -F -t nat

## setup masq and forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward
#/usr/sbin/iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
#/usr/sbin/iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED -j ACCEPT
/usr/sbin/iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
/usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

ETH0 is my WAN interface, ETH1 is my INTERNAL interface.

My current iptables settings are as follows:

[root@monolith root]# iptables -L -t nat -v
Chain PREROUTING (policy ACCEPT 6 packets, 2159 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 8 packets, 898 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   10   602 MASQUERADE  0    --  any    eth0    anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 18 packets, 1500 bytes)
 pkts bytes target     prot opt in     out     source               destination         

[root@monolith root]# iptables -L -v
Chain INPUT (policy ACCEPT 416 packets, 184K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     0    --  eth1   eth0    anywhere             anywhere            

Chain OUTPUT (policy ACCEPT 349 packets, 35688 bytes)
 pkts bytes target     prot opt in     out     source               destination

I have tried this and variations of the gentoo and archlinux recommended stateful firewalls.
I can see that masquerading partially works, but HTTP is failing. Any idea where I should look to resolve this?

running out of ideas

-T

Last edited by Tyris (2007-04-19 04:10:54)

cactus · 2007-04-19 04:28:51

iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT

based on your table output, you don't appear to have a rule to allow stated traffic back in.
the above uses your eth0 as the outside, and eth1 as the inside..

EDIT: it looks like you have that rule..but it is commented out.
any reason why?

Tyris · 2007-04-19 04:42:09

cactus wrote:

iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
based on your table output, you don't appear to have a rule to allow stated traffic back in.
the above uses your eth0 as the outside, and eth1 as the inside..
EDIT: it looks like you have that rule..but it is commented out.
any reason why?

Enabling that now. Yes, eth0 is to the ISP, eth1 is to the internal network.

My rc.firewall now reads:

## setup masq and forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i eth0 -o eth1 -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

I've been staring at this so long my eyes are going crossed. I starting to wonder if I have the eth0 and eth1 crossed in some places.

Last edited by Tyris (2007-04-19 05:02:41)

Tyris · 2007-04-19 06:28:21

minor update:

current rc.firewall:

## setup masq and forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward
/usr/sbin/iptables -A FORWARD -i eth0 -o eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
/usr/sbin/iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
/usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

On the internal machine, ssh works perfectly to outside hosts -- so masquerading+nat is atleast partially working (I'd assume).
wget is able to connect, but that is as far as we go:

[root@paradox ~]# wget cnn.com
--22:23:26--  http://cnn.com/
           => `index.html'
Resolving cnn.com... 64.236.24.28, 64.236.29.120, 64.236.16.20, ...
Connecting to cnn.com|64.236.24.28|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: http://www.cnn.com/ [following]
--22:23:26--  http://www.cnn.com/
           => `index.html'
Resolving www.cnn.com... 64.236.91.22, 64.236.91.23, 64.236.91.24, ...
Connecting to www.cnn.com|64.236.91.22|:80... connected.
HTTP request sent, awaiting response...

Tomorrow I will try to get a hold of a friend's identical server and move my hard drive over there. If it works, my hardware is busted. If it fails, Im going to throw in the towel and reinstall arch on my machine.

Has anyone else ever seen a case where ssh worked but http requests hang?

thanks in advance

-T

Last edited by Tyris (2007-04-19 06:31:16)

Tyris · 2007-04-20 02:10:41

Goddamn. It finally works. Cactus, thank you for your suggestions and for listening to my problem.

I was getting ready to shoot this server if it turned out to be a hardware problem.

I studied some wireshark captures from my LAN a bit and made no forward progress so I brought over a friend's identical server (well, his has faster Xeons and more ram ) swapped my main drive for his. This resulted in an identical failure; so it had do be a software or configuration problem, hardware variable eliminated (else this would have gotten expensive instead of just time consuming.)

I installed firestarter, neat tool. Im tempted to give up my handwritten firewall scripts and just use this, but it seems a little 1 dimensional as I need to figure out how to hook up and route some services through my new wireless interface.

Even with firestarter handling my eth0 and eth1 forwarding was not working properly. I did note several errors being generated from firestarter as it tried to invoke iptables. After this I recompiled my kernel with everything and the kitchen sink related to iptables.

Suddenly everything worked, and the LAN machines were getting their http fetches.
I am happy again

-T

Arch Linux

#1 2007-04-19 04:02:34

complex ip-masquerading failure

#2 2007-04-19 04:28:51

Re: complex ip-masquerading failure

#3 2007-04-19 04:42:09

Re: complex ip-masquerading failure

#4 2007-04-19 06:28:21

Re: complex ip-masquerading failure

#5 2007-04-20 02:10:41

Re: complex ip-masquerading failure

Board footer