How to create an internal OVS network for VMs with world access?

srulop · 2016-12-19 19:56:08

Hi!

I have several VMs connected using OpenVSwitch. I want them to be able to connect to the external (corporate) network, but not directly exposed to it.
Meaning: I could just add the physical interface to the OVS, and make the external connection through the bridge itself, but that would make the VMs exposed directly to the corporate network, and I don't want that. I want the corporate network to see only one computer - the host.
The VMs are using tap interface.

I thought it would be a good idea to setup some kind of routing, like so:

+------------------------------------------------+
|  +---------+   +---------+        +---------+  |
|  |   VM1   |   |   VM2   | ...    | HOST NET|  |
|  +----+----+   +----+----+        +----+----+  |
|       |             |                  |       |
|       +-------------+--+---------------+       |
|                        |                       |
|                   +----+----+                  |
|                   | ovs-br0 |                  |
|                   +----+----+                  |
|                        |          PHYSICAL     |
|                   +----+----+       HOST       |
|                   | ROUTER? |                  |
|                   +----+----+                  |
|                        |                       |
|                   +----+----+                  |
|                   |  eno1   |                  |
+------------------------+-----------------------+
                         |
                      INTERNET

But I found no helpful howtos for that, and moreover, have read that since OVS uses OpenFlow, the whole "router" idea might be redundant.

My question is: what is the proper, "good practice", way to achieve my goal? That's my first attempt to use OVS, so I'm a bit clueless here.

Thanks!!

alphaniner · 2016-12-20 18:35:38

Naively, I assume you'd just follow the Internet Sharing wiki with eno1 and ovs-br0 in place of internet0 and net0 respectively. Minus the OVS/OpenFlow, that's pretty much what I did at my workplace. Granted, I know nothing of OVS/OF so I don't know how they will influence things. Searching "ovs internet sharing" might yield some useful info.

OTOH, maybe it would be better to just use a separate network for internet access.

srulop · 2016-12-20 18:52:19

alphaniner wrote:

Naively, I assume you'd just follow the Internet Sharing wiki with eno1 and ovs-br0 in place of internet0 and net0 respectively. Minus the OVS/OpenFlow, that's pretty much what I did at my workplace. Granted, I know nothing of OVS/OF so I don't know how they will influence things. Searching "ovs internet sharing" might yield some useful info.
OTOH, maybe it would be better to just use a separate network for internet access.

Thanks for your reply.
What I'm trying to do at this very moment is to implement the Internet Sharing wiki between a port connected to an additional bridge that is connected to eno1.
I think I'll lose the external internet connection if I'll try to do it with eno1 directly, as there will be no device exposed to the outside network, am I correct?

What do you mean by "use a separate network for internet access"?

alphaniner · 2016-12-20 19:55:39

It sounds like you're already working towards using "a separate network for internet access" with the additional bridge. Assuming, that is, you plan to add a separate interface to each VM to connect to this bridge?

srulop · 2016-12-21 23:55:49

OK, I achieved the desired result, but not marking "solved" yet, as I would greatly appreciate if someone will go over what I did, to make sure I didn't screw something up (security wise, or "good practices" wise).

So here goes:

Permanent settings:

These are the permanent changes I did to the system.

1)

sudo pacman -S openvswitch dnsmasq

2) Write the following to: /etc/netctl/ovs-br0-eno1

Description="OpenvSwitch connection"
Interface=ovs-br0
Connection=openvswitch
BindsToInterfaces=(eno1)

## Ensure that the bridge gets assigned the MAC address of eno1 (corporate network leases IP only to familiar MACs)
IP=no
ExecUpPost="ip link set dev ovs-br0 address $(cat /sys/class/net/eno1/address); IP=dhcp; ip_set"
ExecDownPre="IP=dhcp"

## Ignore (R)STP and immediately activate the bridge
SkipForwardingDelay=yes

3) Write the needed dnsmasq settings (mind the IPs and the second bridge name):

sudo sed -i 's/^#*dhcp-range=[0-9.]\+,[0-9.]\+,[0-9]\+h/dhcp-range=10.0.0.2,10.0.0.50,12h/;s/^#*interface=.*/interface=ovs-br1/' /etc/dnsmasq.conf

4) Enable IP forwarding (will be active after networking restart)

sudo sh -c "echo net.ipv4.ip_forward=1 > /etc/sysctl.d/30-ipforward.conf"

5) Stop and disable all network services.
6) Start and enable relevant services:

sudo systemctl start ovs-vswitchd.service
sudo systemctl enable ovs-vswitchd.service
sudo netctl start ovs-br0-eno1
sudo netctl enable ovs-br0-eno1

5) Edit /etc/iptables/iptables.rules to the following. Do NOT copy this blindly if you wanna try yourself! Your firewall needs may differ! Also you will NOT be able to connect remotely with these settings.

*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.0.0.0/24 -o ovs-br0 -j MASQUERADE
COMMIT
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:TCP - [0:0]
:UDP - [0:0]
:fw-interfaces - [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p icmp -m icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -p udp -m conntrack --ctstate NEW -j UDP
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j fw-interfaces
-A FORWARD -j REJECT --reject-with icmp-host-unreachable
-A TCP -p tcp -m tcp --dport 53 -s 10.0.0.0/24 -j ACCEPT
-A UDP -p udp -m udp --dport 67 -i ovs-br1 -j ACCEPT
-A UDP -p udp -m udp --dport 53 -s 10.0.0.0/24 -j ACCEPT
-A fw-interfaces -i ovs-br1 -j ACCEPT
COMMIT

Temporary settings:

Each time when a script launches the VMs, it does:

ovs-vsctl add-br ovs-br1
ip link set ovs-br1 up
ip addr add 10.0.0.1/24 dev ovs-br1
systemctl start dnsmasq.service
# And then the ports for each VM are created on ovs-br1 ...

Each time when the VMs stop working, the following happens:

systemctl stop dnsmasq.service
ip link set ovs-br1 down
ovs-vsctl del-br ovs-br1

So, I think that I got the following:
(* = temporary)

          +-----------------------------------------+
          |          +--------+     ***** *****     |
          |          |IP STACK|     *VM1* *VM2* ... |
          |          +----+---+     ***** *****     |
          |               |           *     *       |
          +------+   +----+----+    *************   |
INTERNET--+ eno1 +---+ ovs-br0 |    *  ovs-br1  *   |
          +------+   +----+----+    *************   |
          |               |               |         |
          |              ++---------------++        |
          |              |     ROUTING     |        |
          |              +-----------------+        |
          +-----------------------------------------+

Does this sound reasonable?

Thanks!

Arch Linux

#1 2016-12-19 19:56:08

How to create an internal OVS network for VMs with world access?

#2 2016-12-20 18:35:38

Re: How to create an internal OVS network for VMs with world access?

#3 2016-12-20 18:52:19

Re: How to create an internal OVS network for VMs with world access?

#4 2016-12-20 19:55:39

Re: How to create an internal OVS network for VMs with world access?

#5 2016-12-21 23:55:49

Re: How to create an internal OVS network for VMs with world access?

Board footer