You are not logged in.

#1 2013-02-11 20:10:07

Sin.citadel
Member
Registered: 2008-01-22
Posts: 267

[Unsolved] Single Destination, Multiple Gateways

This question is not arch specific, i was wondering if it is possible to use multiple network connections with separate public ip addresses, for connecting to the same destination.

Suppose i have two ppp connections

ppp0: 192.168.0.2,
ppp1: 192.168.1.2

Using ip routes, i have created appropriate tables and entries, so that whenever packets are generated with source address 192.168.0.2, they use the device gateway of ppp0, similarly for 192.168.1.2.

now adding a route for the destination, i.e, 11.22.33.44

route add 11.22.33.44 dev ppp0
or
route add 11.22.33.44 dev ppp1

The default route is ppp0

This will forward all traffic for that destination to any of these interfaces, depending on the route, or through the default route if any of the routes are not present. What i want, is that the first connection uses ppp0, the second ppp1, the third ppp1, and on and on.

For this i did the following:

supposing that ip route tables consists of two new tables named table1 and table2,
ip route add default dev ppp0 table table1
ip route add default dev ppp0 table table2

Adding rules
ip rule add from 192.168.0.2 table table1
ip rule add from 192.168.1.2 table table2

iptables -t mangle -A PREROUTING -d 11.22.33.44 -m state --state ESTABLISHED,RELATED -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -d 11.22.33.44 -m state --state NEW -m statistic --mode nth --every 2 --packet 0 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -d 11.22.33.44 -m state --state NEW -m statistic --mode nth --every 2 --packet 1 -j MARK --set-mark 2
iptables -t mangle -A PREROUTING -d 11.22.33.44 -j CONNMARK --save-mark

The above 4 rules, allow the first syn packet to that destination to be marked as 1, and then using iptables connmark, any subsequent packets for that connections will be set to mark 1, the second one will be set to mark 2, the third again set to mark 1 and on and on.

Now we add the final rules for routing

ip rule add fwmark 1 table table1
ip rule add fwmark 2 table table2

So now, any packets with mark 1 are sent through ppp0, and any packets with mark 2 are sent through ppp1.

The Problem occurs with the iptables connection tracking, the mangle tables and the ip rules work fine as i can see that initial connection is made through ppp0, the second one through ppp1. The problem is the outgoing ip. the packets going through both interfaces use only one ip, either 192.168.0.2 or 192.168.1.2.

If the packets use 192.168.0.2, then the packets through ppp0 are properly replied and the connection works, but since the packets through ppp1 also use the source address 192.168.0.2, those packets aernt replied, i.e, only 50% of the connections work, the others dont.

Even when using SNAT in iptables such as the following, i am still unable to set outgoing ip address.

iptables -t nat -A POSTROUTING -o ppp0 -j SNAT --to 192.168.0.2
iptables -t nat -A POSTROUTING -o ppp1 -j SNAT --to 192.168.1.2

I was wondering if anyone else has tried something like this, i.e, multiple gateways, but only a single destination. or if it is even possible to do it.

Last edited by Sin.citadel (2013-03-01 20:43:50)

Offline

#2 2013-02-12 23:43:02

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: [Unsolved] Single Destination, Multiple Gateways

Yes, you can load balance Out-bound TCP and UDP streams over two interfaces. The reply will come back on the interface the were sent from. To do this you will need to use the tc command.

The easiest way to load balance over two interfaces is to use arno-iptables-firewall. It has a plug-in to do it for you.

/etc/arno-iptables-firewall/plugins

# ------------------------------------------------------------------------------
#             -= Arno's iptables firewall - Multirouting plugin =-
# ------------------------------------------------------------------------------

# To actually enable this plugin make ENABLED=1:
# ------------------------------------------------------------------------------
ENABLED=0

# RP_FILTER settings (Only set this to 0, if you know what you're doing)
# ------------------------------------------------------------------------------
RP_FILTER_DISABLE=1

# Settings for the first interface:
# ------------------------------------------------------------------------------
MULTIROUTE_EXT_IF1=eth0
MULTIROUTE_EXT_ROUTER1=1.2.3.5
MULTIROUTE_EXT_IP1=5.6.7.9
MULTIROUTE_EXT_WEIGHT1=1

# Settings for the second interface:
# ------------------------------------------------------------------------------
MULTIROUTE_EXT_IF2=eth1
MULTIROUTE_EXT_ROUTER2=1.2.3.4
MULTIROUTE_EXT_IP2=5.6.7.8
MULTIROUTE_EXT_WEIGHT2=1

# Optional settings if you also use an internal interface (for your LAN/gateway)
# ------------------------------------------------------------------------------
MULTIROUTE_INT_IP=192.168.0.1
MULTIROUTE_INT_NET=192.168.0.0/24

https://aur.archlinux.org/packages/arno … -firewall/
https://aur.archlinux.org/packages/syst … -firewall/

Last edited by hunterthomson (2013-02-13 00:26:03)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#3 2013-02-13 14:32:25

Sin.citadel
Member
Registered: 2008-01-22
Posts: 267

Re: [Unsolved] Single Destination, Multiple Gateways

Can you tell as to how i would go about it using tc? i have used tc in the past but only for traffic shaping.

In the arno iptables multiroute script, it seems that it uses the nexthop parameter of ip route. What it will do is allow the kernel to cache the destination for a specific interface.

So for example, using the equalize parameter, connections to destination 111.111.111.111 will go through ppp0, and connections to 222.222.222.222 will go through ppp1, and since the routes will be cached by the kernel, it will send packets to that destination through those interfaces, regardless of any rules. So i wont be able to split connections evenly to a single destination.

Offline

#4 2013-02-18 00:47:35

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: [Unsolved] Single Destination, Multiple Gateways

Awe, Okay, so if you have 2 computers, each computer with 2 IP's and you want to use both links at the same time for communication between the 2 computers... and you have Root and Kernel access on both computers.

You could do this....



                                                                         IP Addr                  IP Addr
        10.0.0.1/24  /(Tap0)-----<OpenVPN>------(ppp0)=======(ppp0)-----<OpenVPN>-----(Tap0)\  10.0.0.2/24
Host----(Bond-0)|                                                                                                                                  |(Bond-0)----Host
                            \(Tap1)-----<OpenVPN>------(ppp1)=======(ppp1)-----<OpenVPN>-----(Tap1)/
                                                                         IP Addr                   IP Addr


Basically, You could make an OpenVPN connection over each of your WAN links to the remote Host.
Use Layer 2 Tap devices with OpenVPN and do NOT set IP's on the Tap devices.
Now, use Bond to create a Layer 2 Bonding between the two Layer 2 Tap devices.
Finally, put an IP address on the Bond devices on each Host, both in the same IP subnet.

Now, the Bond will alternate between Tap0 and Tap1 Ethernet Frame by Ethernet Frame. That setup would send a single TCP Session or UDP Stream between the two Hosts over both WAN Links at the same time.

Last edited by hunterthomson (2013-02-18 01:33:13)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#5 2013-02-20 13:55:41

Sin.citadel
Member
Registered: 2008-01-22
Posts: 267

Re: [Unsolved] Single Destination, Multiple Gateways

This setup will work, but it requires a server (controlled by me) at the remote end to recombine frames and then send them to the original destination. So i would need the remote end to have a higher bandwidth than both of my links combined, and need a publicly accessible server as well.

Any other way?

Offline

#6 2013-02-20 23:12:31

hunterthomson
Member
Registered: 2008-06-22
Posts: 794
Website

Re: [Unsolved] Single Destination, Multiple Gateways

Boy, if I think of something I'll post, but I really don't see how. It is just the way TCP/IP works. You would have to drop to the Link Layer in order to send a single TCP connection over two links.

https://en.wikipedia.org/wiki/Internet_protocol_suite

You can load balance multiple TCP connections over multiple links to a single destination no problem.

Last edited by hunterthomson (2013-02-20 23:17:38)


OpenBSD-current Thinkpad X230, i7-3520M, 16GB CL9 Kingston, Samsung 830 256GB
Contributor: linux-grsec

Offline

#7 2013-03-01 20:41:26

Sin.citadel
Member
Registered: 2008-01-22
Posts: 267

Re: [Unsolved] Single Destination, Multiple Gateways

Yeah, you can load balance, but not to a single destination if multiple links have different ip addresses.

Anyway, looks like there is no solution to this. i'll mark it as unsolved.

Thanks for your help hunterthomson.

Offline

Board footer

Powered by FluxBB