LXC bridge, netctl, systemd and tutti quanti

ianux · 2014-07-02 02:12:43

Hi archers,

I have rented a dedicated server at a provider which offer Arch64 (with 3.10-lts kernel) installation. The system is now up-to-date (netctl-1.8, systemd-214).

I would like to use it for running several services, each one running in a LXC container.

Each container should have access to the Internet, and I have only one public IP address, so it seems pretty obvious I need a bridge to achieve that.

The thing is, I have no access to a KVM, so playing with network configuration is a bit tricky.

So, I have one physical network interface with static IP, and I would like the LXC containers to belong to a private network (like 10.0.0.0/24), with access to the outside world.

I've read a lot of documentation, including Arch wiki, and I'm a little confused about how to achieve all of this. I have read about systemd-networkd, but I would like to stick to netctl, since it appears to me simpler to revert back to a working configuration in case of error (I have broken the network once, following https://wiki.archlinux.org/index.php/Linux_Containers and https://wiki.archlinux.org/index.php/Bridge_with_netctl, leading to reinstallation).

So far, I have a DNS server up and running on the host with my own zone, ip forwarding enabled and no iptables rules.

The things I understand is that I should assign the static public IP address to the br0 bridge, but then what about eth0 configuration, and how do I define a 10.0.0.0/24 private network for my containers (which will be named in my DNS zone. Static IP, no DHCP. The Arch wiki talks about being careful with /etc/resolv.conf, without further explanation)? The Arch wiki, and even the discussions I have read on the Arch ML, are a bit confusing.

Tarqi · 2014-07-02 03:10:06

The easiest might be to use systemd-containers in combination with systemd-networkd.

Since you just have one public ip, you can't bridge your public interface. What you need is the following:

        |         Host          |
        |                       |
WAN <-> eth0 - iptables/NAT - br0 <-> LAN
         |                     |
      PublicIP              10.0.0.254/24


|      Container1    |
ve-container1 <-> host0
    |               |
   br0           10.0.0.1/24 (via 10.0.0.254)


|      Container2    |
ve-container2 <-> host0
    |               |
   br0           10.0.0.2/24 (via 10.0.0.254)

...

Steps:
1) Use systemd-networkd on the host to setup the bridge.
2) Setup iptables/NAT on the host.
3) Start your systemd-nspawn container with the appropriate options. This will create the neccessary network interfaces on the host and the container.
4) Use systemd-networkd on the host to add the (hostside) container interface to the bridge.
5) Use systemd-networkd inside the container to give the (containerside) network interface an ip, the gateway and your dns.
6) Repeat 3-5 for each container.

Thats not to hard, just see man systemd-nspawn and man machinectl.

Note: While systemd-networkd can be used with more magic, I described it this way, as it is easier to understand what to do.

WonderWoofy · 2014-07-02 04:42:00

+1 for systemd-networkd. It makes things just sooo damn simple.

But systemd-nspawn might not be the best idea for containers. The systemd devs have said repeatedly that nspawn is not intended for securing services in the same way as other container solutions like LXC and docker. I have played with nspawn a bit just to try it out, and it is really quite amazing in terms of usability. Also, machinectl and the integration with the other systemd tools is kick ass awesome. But just don't think that it is going to give you the same level of security as those other solutions.

ianux · 2014-07-02 07:14:52

OK then, how do I migrate from netctl to systemd-networkd on a remote host, without losing ssh connection and ensuring network configuration is not irremediably broken?

And I would like to stick to LXC, since I want to do this also to learn something valuable in the IT world.

Tarqi · 2014-07-02 16:23:54

WonderWoofy wrote:

But systemd-nspawn might not be the best idea for containers. The systemd devs have said repeatedly that nspawn is not intended for securing services in the same way as other container solutions like LXC and docker. I have played with nspawn a bit just to try it out, and it is really quite amazing in terms of usability. Also, machinectl and the integration with the other systemd tools is kick ass awesome. But just don't think that it is going to give you the same level of security as those other solutions.

I didn't realize that security is such a strong requirement. As it is obvious that the OP lacks in-deep knowledge of networking (and probably other things), I thought this would be a good idea to get him kicked in.

ianux wrote:

OK then, how do I migrate from netctl to systemd-networkd on a remote host, without losing ssh connection and ensuring network configuration is not irremediably broken?
And I would like to stick to LXC, since I want to do this also to learn something valuable in the IT world.

If you use LXC, just stay with netctl, as it is coverd in the wikipage you linked. However, the diagram above still shows the topology you need, regardless of LXC.

WonderWoofy · 2014-07-02 17:05:48

Tarqi wrote:

WonderWoofy wrote:
But systemd-nspawn might not be the best idea for containers. The systemd devs have said repeatedly that nspawn is not intended for securing services in the same way as other container solutions like LXC and docker. I have played with nspawn a bit just to try it out, and it is really quite amazing in terms of usability. Also, machinectl and the integration with the other systemd tools is kick ass awesome. But just don't think that it is going to give you the same level of security as those other solutions.
I didn't realize that security is such a strong requirement. As it is obvious that the OP lacks in-deep knowledge of networking (and probably other things), I thought this would be a good idea to get him kicked in.

I don't know that it is a strong requirement. I just wanted to make sure that the OP was aware of this before (s)he made a decision of what to use. nspawn really is a great tool, and I use it myself for a couple things. But you shouldn't, for example, use it in the hopes of securing an internet facing http service.

Runiq · 2014-07-02 17:45:09

WonderWoofy wrote:

[
I don't know that it is a strong requirement. I just wanted to make sure that the OP was aware of this before (s)he made a decision of what to use. nspawn really is a great tool, and I use it myself for a couple things. But you shouldn't, for example, use it in the hopes of securing an internet facing http service.

I got curious, googled that for a second and found a fairly detailed e-mail by the man himself on that topic.

Arch Linux

#1 2014-07-02 02:12:43

LXC bridge, netctl, systemd and tutti quanti

#2 2014-07-02 03:10:06

Re: LXC bridge, netctl, systemd and tutti quanti

#3 2014-07-02 04:42:00

Re: LXC bridge, netctl, systemd and tutti quanti

#4 2014-07-02 07:14:52

Re: LXC bridge, netctl, systemd and tutti quanti

#5 2014-07-02 16:23:54

Re: LXC bridge, netctl, systemd and tutti quanti

#6 2014-07-02 17:05:48

Re: LXC bridge, netctl, systemd and tutti quanti

#7 2014-07-02 17:45:09

Re: LXC bridge, netctl, systemd and tutti quanti

Board footer