Netctl, Bridges and nspawn

lufti · 2017-12-19 18:13:37

Hello,

I tried to fiddle a bit with bridging, networking and other things.

Scenario: I want to use systemd-nspawn as chroot on steroids for several webservices (teamspeak, dont starve, mail, openvpn, etc etc etc)

* I also want to seperate each instance regarding the network
* I want to use netctl to manage devices and connections

I ran into some strange behavior.

Profile for bridge:

Description="Bridge to ens3"
Interface=br0
Connection=bridge
BindsToInterfaces=(ens3)
IP=static
Address=('xxx.xxx.xxx.xxx/xx')
Gateway='xxx.xxx.xxx.xxx'
DNS=('xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx')
SkipNoCarrier=yes
SkipForwardingDelay=yes

Profile for ens3:

Description='A basic static ethernet connection'
Interface=ens3
Connection=ethernet
IP=static
Address=('xxx.xxx.xxx.xxx/xx')
Gateway='xxx.xxx.xxx.xxx'
DNS=('xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx' 'xxx.xxx.xxx.xxx')
SkipNoCarrier=yes

I let it run on a live system. After first trying to start the bridge, it failed, always, because br0 was not found as device. Installing bridge-utils and rebooting helped here. I do not now if rebooting alone or installing bridge-utils, then rebooting solved it, but maybe someone else knows about this behavior.

So far so good, but now only the br0 gets an IP assigned. ens3 not. In another tutorial, which is not using netctl and seems to use debian, the ethernet device and the bridge get assigned the same IP. So far everything works fine.

Only if I try to us br0 as network bridge for a arch-spawn, suddenly every connection of the server fails.

I used

systemd-nspawn -b -D /srv/helloWorld --network-bridge=br0

After the system is booted, the server no longer responds with ssh or other network related services.

I want to understand the behavior a bit better, because I want to harden my system by using some chroot like approach...

Would be glad if someone could explain some details or hint me where to find them (the wiki seemed not deep enough for me)

BR,
Lufti.

gdkags · 2017-12-20 13:58:29

lufti wrote:

* I also want to seperate each instance regarding the network
* I want to use netctl to manage devices and connections

I've used systemd-nspawn several times for different applications and consequently switched my server (and workstations) to use netctl. If you'd explain your goal with a bit more information (only specific ports opened for the containers, all/some ports accessible from outside of localhost, socket activation?) I could possibly help you with that.

lufti wrote:

I want to understand the behavior a bit better, because I want to harden my system by using some chroot like approach...
Would be glad if someone could explain some details or hint me where to find them (the wiki seemed not deep enough for me)

Is there any specific reason for configuring the bridging yourself? AFAIR starting a machine with

systemd-nspawn -b -D /var/lib/machines/machine

will provide internet access to that machine since interfaces are shared between machines. Hence all ports opened by the conatiner will be accessible as usual.

lo1 · 2017-12-20 14:04:24

Please post each step from when you boot (into the host) to when you expect the connection to be up. Then post

systemctl status netctl@ens3

In another tutorial, which is not using netctl and seems to use debian, the ethernet device and the bridge get assigned the same IP.

It's not long that I got into bridges, tuntap and VM-like environments, but two interfaces (of any kind) sharing the same address doesn't sound good at all to me.
I don't know if simply changing one of the IP address will do the trick, but are you 200% sure this is the way you want to go?

Last edited by lo1 (2017-12-20 14:05:43)

lufti · 2017-12-20 20:31:07

gdkags wrote:

Is there any specific reason for configuring the bridging yourself? AFAIR starting a machine with
systemd-nspawn -b -D /var/lib/machines/machine
will provide internet access to that machine since interfaces are shared between machines. Hence all ports opened by the conatiner will be accessible as usual.

I am more trying to refresh and improve my understanding about bridging in general and knowing what is going on.

If I understood systemd-nspawn correctly, your suggestion would give the machine access to the hosts network. This is a usable approach, and good on its own, but as I said, I want to try out some things and understand, how certain things happen, and why they happen in that way.

I also played around in my mind, that, if I would set up a tap interface with VPN (no LAN, public server), I could bridge some of the service provided by my spawns via this interface only. (Can also be achieved by configuring iptables, but I like to try out things).

What I want: currently just a hardened game/teamspeak-server. More for educating myself and get known to systemd better. I played around with the idea of later using the containers to maybe having a mean to comfortably moving services like TS or minecraft servers or similar around, if I see that one server is no longer enough to handle it, or to share such a image with others as some sort of preconfigured image, ready to deplay and run.... but thats more of an idea and nothing carved out in special.

lo1 wrote:

Please post each step from when you boot (into the host) to when you expect the connection to be up. Then post
systemctl status netctl@ens3
In another tutorial, which is not using netctl and seems to use debian, the ethernet device and the bridge get assigned the same IP.
It's not long that I got into bridges, tuntap and VM-like environments, but two interfaces (of any kind) sharing the same address doesn't sound good at all to me.
I don't know if simply changing one of the IP address will do the trick, but are you 200% sure this is the way you want to go?

It looked to me strange as well, and I also did not like that idea very much. I will do what you said, when I have again a bit more time, and set up a local system... my TS-Users already complained a bit

Thanks to both of you, will post on the weekend again.

Arch Linux

#1 2017-12-19 18:13:37

Netctl, Bridges and nspawn

#2 2017-12-20 13:58:29

Re: Netctl, Bridges and nspawn

#3 2017-12-20 14:04:24

Re: Netctl, Bridges and nspawn

#4 2017-12-20 20:31:07

Re: Netctl, Bridges and nspawn

Board footer