Setting up LXC containers. Unable to bridge network via dnsmasq

bunnybooboo · 2018-01-29 13:31:04

I've been following the wiki on Linux Containers and Dnsmasq to get them running on my Arch Linux laptop. My settings seem to have a problem specifically with dnsmasq. I'm in over my head though as I came to the wiki for a simple how-to.

My steps to implementation I documented over at Github, complete with screen shots of the systemctl statuses for each of the 3 services: NetworkManager, dnsmasq, and lxc-net. https://github.com/bunnybooboo/learning … h-start.md

Ultimately, I end up with an error in dnsmasq.service "dnsmasq: failed to create listening socket for 127.0.0.1: Address already in use". The other service are all green. Running `brctl show` shows the bridge and its associated container.

Hope someone might be able to guide me towards better understanding of what is going wrong.

I've had a variety of suggestion the last few days:

"dnsmasq is calling 127.0.0.1 twice"
"attempting 2 dnsmasqs"
"once as IPv4 and again as IPv6"
"bridging wifi does not work as ethernet. you would need 4addr mode on both client(bridge) and accesspoint (hodtapd can do that). but you can create internal network, bridge that and do nat on the wifi"

This is day 5 of trying to find a solution.

Leonid.I · 2018-01-29 21:21:35

You know, it usually helps to not solve all issues at once... I can only comment on dnsmasq because I use it daily. However, I don't have ipv6 or NM. Also, chattr +i /etc/resolv.conf is BAD. You don't need to do it.

So, dnsmasq has to listen on the bridge, hence:

$ cat /etc/dnsmasq.conf
domain-needed
bogus-priv
local=/t-lan/
interface=br0
listen-address=10.137.0.1,127.0.0.1
bind-interfaces
domain=t-lan
dhcp-range=10.137.0.10,10.137.0.50,24h

As it follows, I called the bridge interface "br0" and assigned an address 10.137.0.1 to it. It hosts a network 10.137.0.0/24, aka "t-lan".

Once this setup is working, you can move on to setting up NM and ipv6...

EDIT: Oh, and my resolv.conf is

search t-lan colorado.edu
nameserver 128.138.140.5

EDIT2: And here is a netctl profile that creates the bridge

Description="Bridge"
Connection=bridge
Interface=br0
SkipNoCarrier="yes"
BindsToInterfaces=(eno1)
IP=static
Address=(10.137.0.1/24)

"eno1" is a physical NIC (the machine has two of them) that is a part of the bridge, so the workstation can be used as a router for other machines.

Last edited by Leonid.I (2018-01-29 21:32:25)

Elizine · 2018-02-01 05:35:52

Below are the 3 steps to bridge network via dnsmasq -

1. Create the bridge device.
2. Assign an IP address to the bridge device.
3. Configure dnsmasq to listen on the bridge interface IP address.

Creating the bridge device is simple:

sudo apt-get install bridge-utils
sudo brctl addbr br0
sudo brctl addif br0 eth0 eth1

If you want it created automatically, modify /etc/network/interfaces:

iface eth0 inet manual
iface eth1 inet manual

# Bridge setup
iface br0 inet static
    bridge_ports eth0 eth1
    address 192.168.0.2
    netmask 255.255.255.0
    gateway 192.168.0.1

For all intents and purposes, the newly-created bridge interface (br0) is just another interface. In this example, the RPi will be assigned 192.168.0.2 and use 192.168.0.1 as its default gateway.

Configuring dnsmasq is done just as with any other interface. dnsmasq supports a lot of options, but one setting is essential: Define a dhcp range:

dhcp-range=192.168.0.128,192.168.0.191,72h

Once you have dnsmasq configured, it will hand out dhcp leases to devices connected to either interface in the bridge pair. dhcp requests received on either eth0 or eth1 will be assigned addresses by dnsmasq. Be sure this is what you want! If you want to limit dnsmasq to only eth1, you'll need to do some additional work, and bridging may not be the best answer.

For devices not using dhcp, simply point them to your RPi's IP address (192.168.0.2 in this example) for their DNS server.

bunnybooboo · 2018-02-01 17:44:34

Hey thanks for helping Elizine but Debian/Ubuntu instructions are not what I'm looking for in an Arch Linux forum. I'm following solutions provided in Arch Linux wiki for these specific technologies.

Leonid yeah sorry for the dump of info. The frustration was tangible.

Right, I've made some progress. These NetworkManager configurations were superfluous. They had to be commented out or removed. Looks to be a plugin alternative to dnsmasq.conf, so produced a duplication.

/etc/NetworkManager/NetworkManager.conf
------------------------------------------------------------------
#[main]
#dns=dnsmasq

I did the same with

/etc/NetworkManager/dnsmasq.d/ipv6_listen.conf
-------------------------------------------------------------------------
#listen-address=::1

Then Dnsmasq needing editing

/etc/dnsmasq.conf
--------------------------------
...
bind-dynamic
...

This now gives both NetworkManager.service and dnsmasq.service green active/running statuses.

However, lxc-net.service moans a little

● lxc-net.service - LXC network bridge setup
   Loaded: loaded (/usr/lib/systemd/system/lxc-net.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2018-02-01 14:41:49 GMT; 2h 56min ago
  Process: 2622 ExecStart=/usr/lib/lxc/lxc-net start (code=exited, status=1/FAILURE)
 Main PID: 2622 (code=exited, status=1/FAILURE)

Feb 01 14:41:49 archlinux systemd[1]: Starting LXC network bridge setup...
Feb 01 14:41:49 archlinux lxc-net[2622]: dnsmasq: cannot set --bind-interfaces and --bind-dynamic
Feb 01 14:41:49 archlinux lxc-net[2622]: Failed to setup lxc-net.
Feb 01 14:41:49 archlinux lxc-net[2622]: Failed to setup lxc-net.
Feb 01 14:41:49 archlinux systemd[1]: lxc-net.service: Main process exited, code=exited, status=1/FAILURE
Feb 01 14:41:49 archlinux systemd[1]: lxc-net.service: Failed with result 'exit-code'.
Feb 01 14:41:49 archlinux systemd[1]: Failed to start LXC network bridge setup.

I've just searched in all the code and can only ONE reference to bind-dynamic, which is in dnsmasq.conf. There is no longer any reference to 'bind-interfaces', so I feel like there is something, somewhere, overriding these particular settings.

Last edited by bunnybooboo (2018-02-01 17:46:15)

Leonid.I · 2018-02-01 23:39:51

Ah, I see your problem now

Anyway, you are using lxc-net script that attempts to set up a bridge, iptables NAT and start dnsmasq -- just look in it. I only quickly scanned the script, and it seems that you need to make dnsmasq.conf blank because lxc-net puts all options to dnsmasq via cmdline (I don't know whether those options are actually understood by dnsmasq, if they are not, it's a bug in lxc-net). You also need to make sure that 10.0.3.0/24 network does not exist on any interfaces of your system, because that's what the script uses by default (or you need to override it via /etc/default/lxc I guess).

In general, I don't like this approach because it silently manipulates your firewall, sysctl settings and in general it is too complex. I prefer to decouple things and build the setup from individual services: enable ip_forward via sysctl.d/, make the bridge with netctl, start dnsmasq and then just start lxc containers. This gives an additional advantage of being able to reuse the same bridge for other things, like nspawn containers or virtual machines (which I run a lot).

Arch Linux

#1 2018-01-29 13:31:04

Setting up LXC containers. Unable to bridge network via dnsmasq

#2 2018-01-29 21:21:35

Re: Setting up LXC containers. Unable to bridge network via dnsmasq

#3 2018-02-01 05:35:52

Re: Setting up LXC containers. Unable to bridge network via dnsmasq

#4 2018-02-01 17:44:34

Re: Setting up LXC containers. Unable to bridge network via dnsmasq

#5 2018-02-01 23:39:51

Re: Setting up LXC containers. Unable to bridge network via dnsmasq

Board footer