[SOLVED] Docker breaks existing bridge I use for KVM/QEMU

mwilson · 2018-01-19 19:22:35

Hi there,

I have my Arch box set up, using systemd to control network interfaces, with a bridge (br0) that contains my physical network interface (eno1). The bridge is configured with static IP info. So my Arch system has network connectivity. I run a VM using KVM/QEMU attached to the bridge using the "-net bridge,br=br0" QEMU option. With that setup, the VM has network connectivity.

My problem is that when I start the Docker service, my VM no longer has network connectivity. The Arch host still does, though. Stopping the Docker service doesn't fix it--I have to reboot to get systemd to set up the network properly again for my VM. I tried doing some research on how Docker is setting up its bridge to understand what might be going on, but I haven't been able to grok enough to figure out how to configure Docker not to break my existing bridge.

Here's how I have my /etc/systemd/network files set up, which works great before Docker is enabled.

br0.netdev:

[NetDev]
Name=br0
Kind=bridge

br0.network:

[Match]
Name=br0

[Network]
Address=10.110.8.13/23
Gateway=10.110.8.1
DNS=127.0.0.1

uplink.network:

[Match]
Name=eno1

[Network]
Bridge=br0

If anyone has any hints for further troubleshooting or Docker configuration, I'd really appreciate it.

Last edited by mwilson (2018-01-19 22:26:24)

mwilson · 2018-01-19 22:25:43

After more research I learned that I was barking up the wrong tree thinking it was specifically the bridge setup that was being broken. Instead, it was an iptables problem. By default docker messes with iptables when it starts up, and was changing the FORWARD chain to a default policy of deny and adding some of its own rules. That, apparently, was preventing the QEMU tap interface in the br0 bridge from working correctly.

Solved by adding the --iptables=false option to the docker daemon, and now all works as it should.

Edit 2018-01-26 for completeness in case people find this in the future looking for a solution:

It turns out just adding --iptables=false ends up making the docker containers not be able to reach the network. So I removed that flag and looked at what it was doing to iptables. Docker appears to change the default policy for FORWARD from accept to reject, so for expediency, I just do `iptables -P FORWARD ACCEPT` to change it back. I'm assuming I'm opening some security hole here, but the host in question is in a small, protected VLAN, so I'm fine with it for the moment.

Last edited by mwilson (2018-01-26 21:01:14)

puithove · 2018-02-18 14:52:05

Thanks for posting your findings. It get me going quicker. Something must have changed somewhat recently. I hadn't been running any vms on this host in a while, but I didn't have this issue previously. Just fired up a vm and ran into this issue.

My previous thoughts were that since this was a bridged connection (working on a different layer), it would not be filtering IP traffic. A little research shows that IP traffic is still captured and passed through iptables even on bridges.

I wanted to find a finer grained solution than yours, so I added a post-up command to add a rule for the bridged connection. So in my case (using netctl), I added this to my bridged connection's profile:

ExecUpPost="/usr/bin/iptables -A FORWARD -p all -i bridge0 -j ACCEPT"
ExecDownPre="/usr/bin/iptables -D FORWARD -p all -i bridge0 -j ACCEPT"

The rule gets added before docker starts, and is in effect to allow forwarding for anything coming in on that bridge.

pklaus · 2019-07-05 16:04:18

I'd like to say thanks to the two of you! I had the exact same problem.

About my use case:
I'm bridging the two 10G ports of an Intel X550-T2, avoiding an expensive 10G switch if only 3 physical hosts are needed in the network.
Just like you, I'm running Docker containers in parallel.

The Docker service seems to be the culprit loading br_netfilter and altering the firewall as you described. So I thought I'd alter the Docker service file. I used the systemctl edit command to alter the file /etc/systemd/system/docker.service.d/override.conf:

EDITOR=vim sudo -E systemctl edit docker

[Service]
ExecStartPre=/usr/bin/iptables -A FORWARD -p all -i bridge0 -j ACCEPT
#ExecStopPost=/usr/bin/iptables -D FORWARD -p all -i bridge0 -j ACCEPT

And restart docker using:

sudo systemctl daemon-reload
sudo systemctl stop docker
sudo systemctl start docker

The ExecStopPost line is commented out because Docker leaves the problematic iptables rules in place after exiting.

Only problem: Every restart of systemd adds the rule -A FORWARD -p all -i bridge0 -j ACCEPT once more to the firewall...

VoodaGod · 2019-10-02 08:27:52

pklaus wrote:

Only problem: Every restart of systemd adds the rule -A FORWARD -p all -i bridge0 -j ACCEPT once more to the firewall...

I fixed that by deleting the rule before adding it:

[Service]
ExecStartPre=/usr/bin/iptables -D FORWARD -p all -i br0 -j ACCEPT
ExecStartPre=/usr/bin/iptables -A FORWARD -p all -i br0 -j ACCEPT

WorMzy · 2019-10-02 09:44:06

Thanks for sharing. I'm going to go ahead and close this old topic now.

Closing.

Arch Linux

#1 2018-01-19 19:22:35

[SOLVED] Docker breaks existing bridge I use for KVM/QEMU

#2 2018-01-19 22:25:43

Re: [SOLVED] Docker breaks existing bridge I use for KVM/QEMU

#3 2018-02-18 14:52:05

Re: [SOLVED] Docker breaks existing bridge I use for KVM/QEMU

#4 2019-07-05 16:04:18

Re: [SOLVED] Docker breaks existing bridge I use for KVM/QEMU

#5 2019-10-02 08:27:52

Re: [SOLVED] Docker breaks existing bridge I use for KVM/QEMU

#6 2019-10-02 09:44:06

Re: [SOLVED] Docker breaks existing bridge I use for KVM/QEMU

Board footer