[Solved] 1:1 nat with iptables

Metallion · 2011-03-25 06:25:48

I've got a host with 2 qemu virtual machines in it. They're set up bridged with a tap interface so they both have their own ip address and are accessible from the outside.

Their ips are:

VM1: 10.1.0.10
VM2: 10.1.0.11
Netmask for both: 255.255.255.0

Now I am trying to add iptables rules to the host machine to nat both virtual machines to subnet 172.16.0.0/24. I use the following rules for this.

iptables -P FORWARD DROP
iptables -A FORWARD -s 10.1.0.0/24 -j ACCEPT
iptables -A FORWARD -d 10.1.0.0/24 -j ACCEPT
iptables -A INPUT -s 10.1.0.0/24 -j ACCEPT
iptables -A INPUT -s 172.16.0.0/24 -j ACCEPT
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.1.0.10 -j SNAT --to 172.16.0.10
iptables -t nat -A POSTROUTING -s 10.1.0.11 -j SNAT --to 172.16.0.11

The host machine has 3 interfaces.

Eth0 which is the external interface connected to the internet
Tap0 which is the tap interface for the first VM
Tap1 which is the tap interface for the second VM
These are all added to a bridge called br0 that has the external connection set up.

When I try to ping google from inside VM1, I see this going through tap0.

10113.790379    10.1.0.10 -> 8.8.8.8      DNS Standard query A www.google.com

10113.834219 Cisco_42:4f:60 -> Broadcast    ARP Who has 172.16.0.10?  Tell 172.16.0.1

And this through eth0.

10348.090665  172.16.0.10 -> 8.8.8.8      DNS Standard query A www.google.com

10348.134424 Cisco_42:4f:60 -> Broadcast    ARP Who has 172.16.0.10?  Tell 172.16.0.1

So apparently the source nat is properly happening when the dns request for google goes out but then the response doesn't know where to find 172.16.0.10.

Does anyone know how to solve this? Perhaps through virtual interfaces? If possible, I would like to handle this on the host OS without tinkering with the VM's internal network settings.

Last edited by Metallion (2011-03-30 06:58:41)

cactus · 2011-03-25 06:32:41

should be as simple as

ip addr add 172.16.0.10/32 dev eth0

and repeat for the other address. (note. advisable to replace eth0 with the appropriate interface)

you could also use ip aliases, but iproute2 utils are the future (and also leaves you with less interface names!)

Metallion · 2011-03-25 06:43:03

cactus wrote:

should be as simple as
ip addr add 172.16.0.10/32 dev eth0
and repeat for the other address. (note. advisable to replace eth0 with the appropriate interface)
you could also use ip aliases, but iproute2 utils are the future (and also leaves you with less interface names!)

Doesn't work. This makes 172.16.0.10 point to my host machine but its packets aren't getting forwarded into VM1. When I ssh to it, I end up logging into my host.

cactus · 2011-03-25 06:56:40

oops. forgot the dnat rule.

you need a dnat rule to rewrite the traffic on the way back in, _as well as_ adding the ip to the host (otherwise you will not get that machine to arp reply).

iptables -t nat -A PREROUTING -d 172.16.0.10 -j DNAT --to-destination 10.1.0.10

sorry for the failboat on my end. oogah.

Last edited by cactus (2011-03-25 06:58:42)

Metallion · 2011-03-25 07:10:39

I believe iptables automatically sets up a corresponding dnat rule for ever snat. At least that's what I've been reading everywhere.

I tried your suggestion none the less though but still no result. Ssh into 172.16.0.10 just times out now. I see the ssh packets arriving on eth0 but they don't go through to tap0. I'm guessing it must be a forwarding rule somewhere.

fukawi2 · 2011-03-25 07:40:00

Post the current state of your rules:

iptables -nvL
iptables -t nat -nvL

Metallion · 2011-03-25 07:44:46

iptables -nvL

Chain INPUT (policy ACCEPT 367 packets, 38976 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      *       10.1.0.0/24          0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       172.16.0.0/24        0.0.0.0/0           

Chain FORWARD (policy DROP 209 packets, 60314 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  445  125K LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 
    0     0 ACCEPT     all  --  *      *       10.1.0.0/24          0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.1.0.0/24         
  196 53522 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 

Chain OUTPUT (policy ACCEPT 163 packets, 24684 bytes)
 pkts bytes target     prot opt in     out     source               destination

iptables -t nat -nvL

Chain PREROUTING (policy ACCEPT 4221 packets, 822K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    4   336 DNAT       all  --  *      *       0.0.0.0/0            172.16.0.10         to:10.1.0.10 

Chain OUTPUT (policy ACCEPT 114 packets, 8403 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 193 packets, 33094 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 SNAT       all  --  *      *       10.1.0.10            0.0.0.0/0           to:172.16.0.10

As you can see, I've set up logging for all the forwarded packets. The outgoing ones are showing up in the log but incoming ones are not. I tried setting up logging for the prerouting chain too but they still don't show up. Seems like they just aren't dnatted at all. Very strange since their destination clearly is 172.16.0.10.

Here are the relevant parts of the logs in case it helps. This is what shows when making a dns request for www.google.com

Mar 25 17:15:18 hanra kernel: [1886767.666360] IN=br0 OUT=br0 PHYSIN=tap0 PHYSOUT=eth0 SRC=10.1.0.10 DST=8.8.8.8 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14428 DF PROTO=UDP SPT=38635 DPT=53 LEN=40 
Mar 25 17:15:18 hanra kernel: [1886767.666395] IN= OUT=br0 PHYSIN=tap0 PHYSOUT=eth0 SRC=10.1.0.10 DST=8.8.8.8 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=14428 DF PROTO=UDP SPT=38635 DPT=53 LEN=40

In tshark it looks like this:

For eth0:

19649.108081  172.16.0.10 -> 8.8.8.8      DNS Standard query A www.google.com
19649.153407      8.8.8.8 -> 172.16.0.10  DNS Standard query response CNAME www.l.google.com A 74.125.235.82 A 74.125.235.80 A 74.125.235.84 A 74.125.235.83 A 74.125.235.81

For tap0

19414.807637    10.1.0.10 -> 8.8.8.8      DNS Standard query A www.google.com

Response arrives on eth0 but isn't dnatted to tap0.

Last edited by Metallion (2011-03-25 08:25:36)

cactus · 2011-03-25 20:44:36

Do you have ip forwarding enabled?
Seems like a very weird issue. I wonder if some weird interplay between bridging and iptables is going on here...

Metallion · 2011-03-28 03:07:29

Ip forwarding is enabled. It's weird indeed. I'm logging all natted packets and the response doesn't show up in the log at all. That means it's discarded before it reaches that point in the chain. I'm going to see what happens if I play with my filtering rules a bit.

I set all policies to accept and turned on logging on all chains. Still I don't see the response in iptables' logs. When I ping 172.16.0.10 from my laptop though, I do see it coming in but it still doesn't go through to tap0. It might indeed be the bridge that stops the response from ever reaching ip tables but then I still wonder why this ping request isn't natted either.

Last edited by Metallion (2011-03-28 03:43:23)

cactus · 2011-03-28 08:38:53

hmmm...what is the ip address of the router machine's bridge interface?
eg, the ip of br0

Metallion · 2011-03-28 09:06:05

it's 192.168.2.25. This was the host machine's eth0 address before I set up the bridge.

Edit: In my search I have learned that only the first packet of each connection (ie with state NEW) traverse the nat table. That explains why the response packets don't show up in the log. Still doesn't explain why they aren't natted though. Continuing my search.

Last edited by Metallion (2011-03-28 09:22:18)

cactus · 2011-03-29 02:01:53

hmm. So if your interface is 192.168.2.25, and the bridge is passing traffic for 10.1.0.x, then the machine has no route to get that traffic back onto the bridge. I wonder if you changed the interface for the bridge to something on the 10.1.0.x network, if it would start working.

I think alternately you could probably use ebtables to coerce the traffic onto the bridge, but that might be work than needed.

Worth a try at least..

i ran across this link too, might be useful.
http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html

Metallion · 2011-03-30 06:58:07

You're right! I added an 10.1.0.x ip to br0:0 and now the setup works! It looks so simple now but you just gotta know these things. Thank you very much for your help!

cactus · 2011-03-30 07:04:20

hooray!
Glad it got sorted out.

Narsimha · 2014-03-13 11:57:05

Hi Metallion,
Can you post the configuration completely that you have done in order to nat the ips.

Inxsible · 2014-03-26 18:27:46

Narsimha, the thread is 3 years old. Please do not necrobump threads -- https://wiki.archlinux.org/index.php/Fo … Bumping.22

Start your own thread and link to this one if you feel it is relevant. Also list what you have already tried in order to fix it.

Closing...

Arch Linux

#1 2011-03-25 06:25:48

[Solved] 1:1 nat with iptables

#2 2011-03-25 06:32:41

Re: [Solved] 1:1 nat with iptables

#3 2011-03-25 06:43:03

Re: [Solved] 1:1 nat with iptables

#4 2011-03-25 06:56:40

Re: [Solved] 1:1 nat with iptables

#5 2011-03-25 07:10:39

Re: [Solved] 1:1 nat with iptables

#6 2011-03-25 07:40:00

Re: [Solved] 1:1 nat with iptables

#7 2011-03-25 07:44:46

Re: [Solved] 1:1 nat with iptables

#8 2011-03-25 20:44:36

Re: [Solved] 1:1 nat with iptables

#9 2011-03-28 03:07:29

Re: [Solved] 1:1 nat with iptables

#10 2011-03-28 08:38:53

Re: [Solved] 1:1 nat with iptables

#11 2011-03-28 09:06:05

Re: [Solved] 1:1 nat with iptables

#12 2011-03-29 02:01:53

Re: [Solved] 1:1 nat with iptables

#13 2011-03-30 06:58:07

Re: [Solved] 1:1 nat with iptables

#14 2011-03-30 07:04:20

Re: [Solved] 1:1 nat with iptables

#15 2014-03-13 11:57:05

Re: [Solved] 1:1 nat with iptables

#16 2014-03-26 18:27:46

Re: [Solved] 1:1 nat with iptables

Board footer