Been gone for a while but I got an extra computer recently and I wanted to install Arch on that.
I have five computers, three desktops (1 linux and two XP), one laptop (XP) and one router/server OpenBSD). Here's how I have them set up:
I have NAT set up on both OpenBSD (vi pf) and Archlinux (vi iptables, Arno's script) and internet works on all computers.
I cannot ping the OpenBSD box from the WinXP boxes connected to Arch Linux and vice versa however. Pinging the Openbsd box from Arch works fine and I can also ping Arch from both of the WinXP boxes and vice versa.
Arch linux > route Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface localhost.local * 255.255.255.255 UH 0 0 0 lo 192.168.4.0 * 255.255.255.0 U 0 0 0 eth2 192.168.3.0 * 255.255.255.0 U 0 0 0 eth1 192.168.1.0 * 255.255.255.0 U 0 0 0 eth0 default 192.168.1.100 0.0.0.0 UG 0 0 0 eth0 > ifconfig eth0 Link encap:Ethernet HWaddr 00:50:BF:A8:B7:47 inet addr:192.168.1.101 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:29816 errors:0 dropped:0 overruns:0 frame:0 TX packets:39043 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4014724 (3.8 Mb) TX bytes:27758852 (26.4 Mb) Interrupt:11 Base address:0xd800 eth1 Link encap:Ethernet HWaddr 00:50:BF:A8:B7:45 inet addr:192.168.3.100 Bcast:192.168.3.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:35212 errors:0 dropped:0 overruns:0 frame:0 TX packets:26429 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:27341736 (26.0 Mb) TX bytes:1545935 (1.4 Mb) Interrupt:3 Base address:0xdc00 eth2 Link encap:Ethernet HWaddr 00:10:4B:BB:8A:67 inet addr:192.168.4.100 Bcast:192.168.4.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:45 errors:0 dropped:0 overruns:0 frame:0 TX packets:28 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:4615 (4.5 Kb) TX bytes:2197 (2.1 Kb) Interrupt:10 Base address:0xe000 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:6 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:288 (288.0 b) TX bytes:288 (288.0 b)
The XP boxes have a default route pointing to their gateway (192.168.3.100 and 192.168.4.100 respectively).
I've checked my firewalls and as far as I can see they should let the packets through so I'm wondering if this is a routing issue I'm not grasping.
Any input is greatly appreciated.
Ugh, a NAT in a NAT? That's horrible. Why don't you just use a hub or switch to make one network between all the boxes, except the wireless one? That would also save two networkcards. If you don't want that then at least try to get rid of the second NAT, and only do the forwarding. Make sure the mask is correct on the BSD box, I think it should be 255.255.0.0.
Thanks. See I was supicious of my own attemps here since I don' t know a whole lot about networking.
I'd like to try your second approach. If I set up NAT on the BSD box, forwarding on the Linux box, change the mask for the BSD interface conected to my Linux box to 255.255.0.0, would that work? Am I understanding you correctly?
Yes, that's correct. I think it should work, but no guarantees. ;-)
NAT is an ugly construction to overcome IP shortage problems, as you can't talk to the big internet with your not very unique local ip address, hence you need something like NAT. But that isn't the case for the local networks, there you can use local IP addresses without problems, and using NAT to hide the subnets doesn't make sense.
Good deal. Thanks!
But my inution that each NIC needs its own subnet is correct no?
Cuz I'm trying this now and it turns out it's a pain to set up forwarding without NAT :-) Friggin' iptables!
Edit: btw I have ip forwarding enabled.
> cat /proc/sys/net/ipv4/ip_forward 1
Edit: I made a simple iptables log rule like this:
iptables -P FORWARD ACCEPT iptables -A FORWARD -j LOG iptables -A FORWARD -j ACCEPT
And I see packets from (for example) 192.168.4.101 destined for 192.168.1.100 show up in the dmesg on the linux box. After that, no sign of em though...
But my inution that each NIC needs its own subnet is correct no?
Yes, that's correct. Otherwise it's unclear to which interface traffic should be sent.
You can try to ping from one of the XP's to the BSD (or vice versa) and monitor the traffic to see if the data is sent. Can do the monitoring with simple ifconfig or something more advanced like Ethereal. If the data is passed along then the config of the Arch box is alright.
Did you change the mask fo all the other pc's too? That may be needed.
I would get it working with as few firewall rules as possible, to rule that out (and disable any on the XPs).
As far as I know enabling forwarding and setting default gateway should be enough to make it work, NAT only does extra work on top of that; changing the source ip address and keeping track of connections and stuff.
Hmm, perhaps it helps to change one of the ip addresses of the BSD cards, as they're now in the same subnet as the other pc's, which may be confusing. If a packets has as destiny 192.168.x.x then it's unclear to where it must be send for the BSD. What if you try using 10.0.0.X for aue0?
Ok great, I'm at work now but I'll play with it as soon as I get home.
Thanks for your help! I appreciate the conceptual clarification and the practical advice.
Well this is fun..geez :-)
The OpenBSD box is configured as 192.168.1.100 netmask 255.255.0.0 towards Linux
and 172.16.1.100 netmask 255.255.255.0 towards the windows XP box. In other words these shouldn' t interfere with each other now (they don' t appear to).
On linux I have the NIC connected to Openbsd configured as:
192.168.1.101 netmask 255.255.0.0
and the NICs going to the two windows boxes as:
192.168.2.100 netmask 255.255.255.0
192.168.3.100 netmask 255.255.255.0
Now OpenBSD talks to Linux and vice versa and Linux talks to the two XP boxes and vice versa and internet works on BSD and Linux (natted). When I ping the OpenBSD box (192.168.1.100) from one of the XP boxes connected to it through Linux I get a request time-out and Ethereal shows the packets as going through. And indeed, when I try to ping one of the XP boxes from OpenBSD (say 192.168.2.101) I get a " host is down" and Ethereal shows that it's looking in the arp table to find 192.168.2.101.
It appears my problem is now effectively reduced to getting OpenBSD to ping the windows boxes connected to it through Linux.
If this sounds at all intelligible perhaps someone could help me understand why OpenBSD can' t ping the XP boxes.
OpenBSD's route output shows that for 192.168.0.0/16 it goes to the NIC at 192.168.1.100 so at least that is alright...
Ah yes, ARP. Totally forgot about that. When you were using NAT that wasn't a problem because the packets appeared to come from the Linux pc. Looking around a bit, it looks like you need to enable proxy-ARP on the Arch pc. Very useful websites (especially the first one has awfully lot of info):
In short, it looks like you need to do the following:
$ echo 1 > /proc/sys/net/ipv4/conf/eth0/proxy_arp $ echo 1 > /proc/sys/net/ipv4/conf/eth1/proxy_arp $ echo 1 > /proc/sys/net/ipv4/conf/eth2/proxy_arp
While this is fun to get working, why not just putting the two networkcards into the BSD box? Cable problem?
AH HA! This works. I can now ping the OpenBSD box from XP but the other way around XP doesn't respond back (with the firewall off). To make this more interesting: I can ssh into 192.168.2.101 (Windows) from OpenBSD but not the other way around. Internet also works on the windows box. Maybe it's a problem with the firewall on OpenBSD but I'll fix that tomorrow (want to be really careful since that thing is hooked up to the net directly).
I think you saved the day! Thank you :-)
Why this particular setup?
1) The OpenBSD box and one WinXP box are one floor below my linux box and the the other two XP boxes. Hooking them all up to a router or switch would require more cable and probably more drilling through the floors. I'd like to avoid that.
2) The OpenBSD box is a Dell Optiplex with very little space inside so I can't stick another NIC in there.