Arch Linux

7nm

Member

Registered: 2019-02-08

Posts: 15

systemd-networkd fails when using bridges and tap devices

I'm using the latest version of Arch Linux with 4.20.6-arch1-1-ARCH and systemd 240.

I'm using bridges and tap devices for giving network access to my VMs which I start using QEMU.
After I tried to start a VM today, systemd-networkd failed and gave me the following error log -

Assertion 'IN_SET(link->state, LINK_STATE_CONFIGURING, LINK_STATE_FAILED, LINK_STATE_LINGER)' failed at ../systemd-stable/src/network/networkd-link.c:934, function address_h>
systemd-networkd.service: Main process exited, code=dumped, status=6/ABRT
systemd-networkd.service: Failed with result 'core-dump'.
systemd-networkd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
systemd-networkd.service: Scheduled restart job, restart counter is at 1.
Stopped Network Service.
Starting Network Service...
tap1: TUNSETIFF failed on tun dev: Device or resource busy
Could not load configuration files: Device or resource busy

The relevant network files are -

$ cat /etc/systemd/network/40-br0.netdev
[NetDev]
Description="A bridge for VMs and containers"
Name=br0
Kind=bridge
MACAddress=14:CF:92:A3:31:E5

$ cat /etc/systemd/network/40-br0.network
[Match]
MACAddress=14:CF:92:A3:31:E5
Name=br0

[Link]
MACAddress=14:CF:92:A3:31:E5
RequiredForOnline=no

[Network]
Description="A bridge for VMs and containers"
DHCPServer=yes
LinkLocalAddressing=no
LLMNR=no
IPv6AcceptRA=no

[Address]
Address=192.168.1.1/24
Broadcast=192.168.1.255

[DHCPServer]
PoolOffset=100
PoolSize=50

$ cat /etc/systemd/network/31-tap1.netdev
[NetDev]
Description="This is a tap device for a VM or a container"
Name=tap1
Kind=tap

[Tap]
User=user
Group=user

$ cat /etc/systemd/network/31-tap1.network
[Match]
Name=tap1

[Link]
RequiredForOnline=no

[Network]
Description="This is a tap device for a VM or a container"
LinkLocalAddressing=no
LLMNR=no
IPv6AcceptRA=no
Bridge=br0

I've setup the following IPtables rules for the NAT routing from 192.168.0.1/24 on the host to 192.168.1.1/24 on the bridge and the guests -

# Generated by iptables-save v1.6.2 on Fri Sep  7 01:20:10 2018
*nat
:PREROUTING ACCEPT [1:360]
:INPUT ACCEPT [1:360]
:OUTPUT ACCEPT [1666:225694]
:POSTROUTING ACCEPT [226:17150]
-A POSTROUTING -o enp3s0 -j MASQUERADE
COMMIT
# Completed on Fri Sep  7 01:20:10 2018
# Generated by iptables-save v1.6.2 on Fri Sep  7 01:20:10 2018
*filter
:INPUT ACCEPT [100505:108196414]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [77813:6196574]
-A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i br0 -o enp3s0 -j ACCEPT
COMMIT
# Completed on Fri Sep  7 01:20:10 2018

I think I've also set the relevant sysctl parameters -

net.ipv4.ip_forward=1
net.ipv4.conf.enp3s0.proxy_arp=1
net.ipv4.conf.tap0.proxy_arp=1
net.ipv4.conf.tap1.proxy_arp=1

Any ideas? dmartins on #archlinux IRC suggested that this might be the cause of this error. If it is, can the fix committed to master be backported to Arch? Or is this another issue? Please help!

chr0mag

Member

From: Vancouver, Canada

Registered: 2017-02-02

Posts: 94

Re: systemd-networkd fails when using bridges and tap devices

I see this on one of my machines as well where I'm using a bridge device, but no TAP device. In my case systemd-networkd successfully recovers after the core dump.

My relevant config files look like this:
/etc/systemd/network/1-br0.netdev

[NetDev]
Name=br0
Kind=bridge

/etc/systemd/network/20-br0.network

[Match]
Name=br0

[Network]
Address=10.14.14.1/24

[Route]
Destination=192.168.4.0/24
Gateway=10.14.14.14

/etc/systemd/network/40-br0-wired-slaves.network

[Match]
Name=lan*

[Network]
Bridge=br0

/etc/systemd/network/50-br0-wireless-slaves.network

[Match]
Name=wifi*

[Network]
Bridge=br0

[Bridge]
HairPin=true
MulticastToUnicast=true

That fix is slated to appear in v241 so I was going to sit tight until then. Since networkd doesn't seem to recover at all for you, you could temporarily revert to usermode networking. You could also try downgrading systemd. It was after the 239.303-1 -> 239.370-1 update that I first saw this problem.

Last edited by chr0mag (2019-02-08 23:50:49)

dmartins

Member

Registered: 2006-09-23

Posts: 360

Re: systemd-networkd fails when using bridges and tap devices

I manged to replicate this running a systemd-nspawn container, although like chr0mag mentioned, systemd-networkd did respawn and the container obtained an IP address.

If you want to try rebuilding systemd, I updated the package from the core repo to include the necessary commits from https://github.com/systemd/systemd/pull/11274/commits

To compile systemd with these backports:
# pacman -S asp
$ asp checkout systemd
$ cd systemd/repos/core-x86_64

Save this patch to systemd/repos/core-x86_64
https://gist.github.com/bikefrivolously … 2c1edc6b98

$ patch -p3 < systemd-240.34-3-issue-11272.patch
$ makepkg -sr

Install the updated version of systemd, libsystemd, systemd-sysvcompat, and if you use it systemd-resolvconf using pacman -U

If it fixes your issue, let me know and maybe this can be included in the next version of systemd coming through testing (prior to 241).

7nm

Member

Registered: 2019-02-08

Posts: 15

Re: systemd-networkd fails when using bridges and tap devices

I manged to replicate this running a systemd-nspawn container, although like chr0mag mentioned, systemd-networkd did respawn and the container obtained an IP address.

If you want to try rebuilding systemd, I updated the package from the core repo to include the necessary commits from https://github.com/systemd/systemd/pull/11274/commits

To compile systemd with these backports:
# pacman -S asp
$ asp checkout systemd
$ cd systemd/repos/core-x86_64

Save this patch to systemd/repos/core-x86_64
https://gist.github.com/bikefrivolously … 2c1edc6b98

$ patch -p3 < systemd-240.34-3-issue-11272.patch
$ makepkg -sr

Install the updated version of systemd, libsystemd, systemd-sysvcompat, and if you use it systemd-resolvconf using pacman -U

If it fixes your issue, let me know and maybe this can be included in the next version of systemd coming through testing (prior to 241).

I tried building systemd with your patch but the build fails during check() with this file being the culprit -

350/514 test-socket-util                        FAIL     0.01 s (killed by signal 6 SIGABRT)

Here's the log.

7nm

Member

Registered: 2019-02-08

Posts: 15

Re: systemd-networkd fails when using bridges and tap devices

I can confirm that the bug isn't present in systemd-239.370-1. I've downgraded systemd, libsystemd, and systemd-sysvcompat for now.