[Solved] Network Bridging Issues with Qemu/KVM VMs and Host

stewbond · 2016-08-17 17:18:36

Hi All,

I am trying to bridge a Qemu/kvm VM guest so that it has an IP address and can work with the rest of my 192.168.1.x network. I've followed dozens of guides, and everything is either incomplete, uses ifconfig/brctl/tunctl/init.d instead of ip link, ip tuntap, systemd, or uses sudo (which other guides say is bad), and none of them have given me a hint of success yet. Could someone point me in the correct direction? My main issues are (in order of decreasing priority):
1) Host internet is lost after making a bridge, Solved below with netctl
2) Bridge does not persist after a reboot Solved below with netctl
3) Windows guest doesn't have a network adapter. Solved by not putting this machine in bridged mode.

Host: Arch
Guests: Arch x2, Windows 10 x1

Guides I've been using:
http://www.linux-kvm.org/page/Networkin … ual_Bridge
http://wiki.qemu.org/Features-Done/HelperNetworking
https://sfxpt.wordpress.com/2011/02/08/ … ic-bridge/
https://wiki.archlinux.org/index.php/Network_bridge
https://wiki.archlinux.org/index.php/QEMU

I've tried several methods, but am having the most success with the qemu-bridge-helper method with this tweak/explanation. That method requires a bridge (br0) to already be available so I need to get that done first. Here's how I do it:

[stew@Romulus]$ ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether f0:79:59:dc:c4:bf brd ff:ff:ff:ff:ff:ff
3: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether f0:79:59:dc:c3:75 brd ff:ff:ff:ff:ff:ff
[stew@Romulus]$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether f0:79:59:dc:c4:bf brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.64/24 brd 192.168.1.255 scope global enp11s0
       valid_lft forever preferred_lft forever
    inet6 fe80::daf4:be2b:2a25:3b16/64 scope link 
       valid_lft forever preferred_lft forever
3: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f0:79:59:dc:c3:75 brd ff:ff:ff:ff:ff:ff
[stew@Romulus]$ sudo ip link add name br0 type bridge
[stew@Romulus]$ sudo ip link set br0 up
[stew@Romulus]$ sudo ip link set enp11s0 master br0

Host internet is lost at this point. That's my major concern. Meanwhile, continuing with the guests I just confirm that a few of the qemu-bridge-helper settings are prepared:

[stew@Romulus]$ bridge link
2: enp11s0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 
[stew@Romulus]$ ls -la /etc/qemu/bridge.conf
-rw-r--r-- 1 root kvm 10 Aug 17 17:21 /etc/qemu/bridge.conf
[stew@Romulus]$ cat /etc/qemu/bridge.conf
allow br0

They look good so now we start the VMs (line breaks added for clarity):

[stew@Romulus]$ /usr/bin/qemu-system-x86_64 -boot c -m 512 \
                                            -hda ApacheServer.qcow2 \
                                            -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,id=hn0 -device virtio-net-pci,netdev=hn0,id=nic1,mac=54:52:00:00:12:52 \
                                            -localtime --enable-kvm &
[stew@Romulus]$ /usr/bin/qemu-system-x86_64 -boot c -m 512 \
                                            -hda emptyArch.qcow2 \
                                            -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,id=hn0 -device virtio-net-pci,netdev=hn0,id=nic1,mac=54:52:00:00:12:51 \
                                            -localtime --enable-kvm &
[stew@Romulus]$ /usr/bin/qemu-system-x86_64 -boot c -m 3500 \
                                            -hda WindowsKVM1.qcow2 -hdb WindowsKVM2.qcow2 \
                                            -netdev tap,helper=/usr/lib/qemu/qemu-bridge-helper,id=hn0 -device virtio-net-pci,netdev=hn0,id=nic1,mac=54:52:00:00:12:53 \
                                            -localtime --enable-kvm &

The first two VMs (Arch guests) worked just fine. They got DHCP addresses (which I then successfully made static) and could communicate with the rest of the world. The third VM (Windows 10 guest) doesn't seem to have a network adapter installed. When I look at available connections, there are none. Using connecting a new network via the "Network and Sharing Center" I get a very plain message: "Windows did not detect any networking hardware". I suspect that this is something Qemu/kvm needs to make available to the guest. Here is what other information I gathered:

C:\Users\Stewart>ipconfig /all
Windows IP Configuration

   Host Name . . . . . . . . . . . . : Win10VM
   Primary DNS Suffix  . . . . . . . : 
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No

C:\Users\Stewart>ping 8.8.8.8
Pinging 8.8.8.8 with 32 bytes of data:
PING: transmit failed. General failure
PING: transmit failed. General failure

Ping statistics for 8.8.8.8:
    Packets: Send = 2, Received = 0, Lost = 2 (100% loss),
Control-C
^C

To keep moving forward, we can check the bridge link status:

[stew@Romulus]$ bridge link
2: enp11s0 state UP : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 
5: tap0 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
6: tap1 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
7: tap2 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100

To disable the bridge and return to a working internet for my host (so I can write this post) I either reboot or run these commands:

[stew@Romulus ~]$ sudo ip link set enp11s0 nomaster
[stew@Romulus ~]$ sudo ip link set enp11s0 down
[stew@Romulus ~]$ sudo ip link delete br0 type bridge
[stew@Romulus ~]$ sudo ip link set enp11s0 up
[stew@Romulus ~]$ sudo systemctl start dhcpcd@enp11s0.service

Of course, once I have the host back online with the VMs, I'd like to make these changes persistant and that is the other issue that needs to be solved. Everything I've seen just refers to /etc/network/interfaces or /etc/sysconfig/network-scripts/ifcfg-eth0 which are both unused in Arch. I suspect /etc/dhcpcd.conf will be involved and there may be some systemctl enable dhcpcd@br0.service needed.

Thanks in advance for your help.

Last edited by stewbond (2016-08-18 11:51:46)

alphaniner · 2016-08-17 18:00:50

Is netctl an option? That's what I always used to create a fixed bridge for use with Qemu/KVM. These days I use virt-manager, but I know I had it working with qemu command line back when I used that.

$ cat /etc/netctl/net0b
Description="lolnet bridge"
Interface=net0b
Connection=bridge
BindsToInterfaces=(net0)
IP=dhcp

stewbond · 2016-08-17 19:59:01

I'm not opposed to netctl at all. It's in Arch's base package, so it sounds like it is meant to be used (unlike netcfg which a lot of people seem to use) and shouldn't conflict with system.d or dhcpcd too badly. This is how I'm going to try to do it:

/etc/netctl/bridge

Description="VM Bridge connection"
Interface=br0
Connection=bridge
BindsToInterfaces=(enp11s0)
IP=dhcp
#SkipForwardingDelay=yes  (Should I play with this?)

Services:
Need to systemctl disable dhcpcd@enp11s0.service
Need to netctl start bridge & netctl enable bridge

Edit:
When doing some research I found this file. Could this be my problem for issue #2? No because it's not loaded. See "netctl status ethernet-noip"
/etc/netctl/ethernet-noip

Description="VM Ethernet bridge"
Interface=enp11s0
Connection=ethernet
IP=no

Last edited by stewbond (2016-08-17 21:03:02)

brebs · 2016-08-17 20:30:21

Use tcpdump to debug your network config.

alphaniner · 2016-08-17 20:44:58

When doing some research I found this file. Could this be my problem for issue #2?
/etc/netctl/ethernet-noip

Profiles don't have any effect unless they're started.

stewbond · 2016-08-17 21:02:23

Haha!!! You guys are great! My host and Arch guests have simultaneous connections. Posts above are updated with solutions. My last issue is regarding the lack of a network adapter provided to my windows guest.

Proof of success (parts 1 & 2):

[stew@Romulus ~]$ netctl status bridge
● netctl@bridge.service - VM Bridge connection
   Loaded: loaded (/etc/systemd/system/netctl@bridge.service; enabled; vendor pr
   Active: active (exited) since Wed 2016-08-17 23:03:31 CEST; 44s ago
     Docs: man:netctl.profile(5)
  Process: 389 ExecStart=/usr/lib/network/network start %I (code=exited, status=
 Main PID: 389 (code=exited, status=0/SUCCESS)
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/system-netctl.slice/netctl@bridge.service
           └─476 dhcpcd -4 -q -t 30 -L br0

Warning: Journal has been rotated since unit was started. Log output is incomple
[stew@Romulus ~]$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp11s0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP group default qlen 1000
    link/ether f0:79:59:dc:c4:bf brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f279:59ff:fedc:c4bf/64 scope link 
       valid_lft forever preferred_lft forever
3: eno1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f0:79:59:dc:c3:75 brd ff:ff:ff:ff:ff:ff
4: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether f0:79:59:dc:c4:bf brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.64/24 brd 192.168.1.255 scope global br0
       valid_lft forever preferred_lft forever
    inet6 fe80::f279:59ff:fedc:c4bf/64 scope link 
       valid_lft forever preferred_lft forever
[stew@Romulus Windows Clone]$ bridge link
2: enp11s0 state UP : <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 4 
5: tap0 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100 
7: tap1 state UNKNOWN : <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 100

Got my ubuntu server working too! Only part 3 above is outstanding.

Last edited by stewbond (2016-08-17 21:40:01)

alphaniner · 2016-08-17 21:46:04

Looks like you're using virtio nics, which probably require drivers in Windows. Theres virtio-win in AUR, but I have no idea if WX support is there yet. Or just try another type. Search /usr/share/doc/qemu/qemu-doc.html for "Network options" if you have no idea what I'm talking about.

stewbond · 2016-08-17 22:14:03

Interesting, the virtio stuff I got form either qemu-launcher, or a website somewhere. Searching around, the internet is full of people with virtio Windows VM issues. I'll do some investigating tomorrow. For now it's off to bed.

Thanks alphaniner. Sometimes it's really just a few keywords they get you really going towards a solution. In this case you gave me netctl and virtio. Netctl got me reading and that came to a solution. Let's hope virtio does the same! I'm going to mark this as solved. I'll update with what I did when I find a solution, or make a link to a new thread if I need more help.

Here I've been trying to install the virtio-win drivers

git clone https://aur.archlinux.org/virtio-win.git
cd virtio-win
makepkg -sri

Now I have a virtio-win.iso which I insert into my VM from qemu's compatmonitor0 (CTRL+ALT+2) like so:

(qemu) change ide1-cd0 virtio-win.iso

From there I drop into the Windows command prompt

d:\> guest-agent/qemu-ga-x64.msi
d:\> rundll32 NetKVM/w10/amd64/netkvmco.dll
d:\> netsh netkvm show devices
The following command was not found: netkvm show devices.

There isn't much else I can do with this package. It doesn't seem to do what it says it does. So I need to look for another solution. Will post when I figure out what I'm going to do next.

Next option1: Run my birdge with something other than virtio
Next option2: Since this particular guest isn't a server, I could try and get it running without the bridge (user-mode)

Option 2 worked like a charm. Startup command for this machine is now:

/usr/bin/qemu-system-x86_64 -boot c -m 3500 
                            -hda '/home/stew/VirtualBox VMs/Windows Clone/WindowsKVM1.qcow2' 
                            -hdb '/home/stew/VirtualBox VMs/Windows Clone/WindowsKVM2.qcow2' 
                            -net nic,vlan=0,macaddr=54:52:00:00:12:51 -net user,vlan=0 
                            -localtime 
                            -soundhw all 
                            -smp 4 
                            --enable-kvm &

Last edited by stewbond (2016-08-18 11:51:03)

Arch Linux

#1 2016-08-17 17:18:36

[Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#2 2016-08-17 18:00:50

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#3 2016-08-17 19:59:01

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#4 2016-08-17 20:30:21

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#5 2016-08-17 20:44:58

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#6 2016-08-17 21:02:23

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#7 2016-08-17 21:46:04

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

#8 2016-08-17 22:14:03

Re: [Solved] Network Bridging Issues with Qemu/KVM VMs and Host

Board footer