You are not logged in.

#1 2023-09-06 21:12:53

oysterpingu
Member
Registered: 2023-09-06
Posts: 3

[KVM/QEMU/Libvirt] NAT `default` network not working as expected

I am using QEMU/KVM with libvirt and virt-manager. The virtual machine runs just fine, but it is not able to work out its ip. I can provide one manually, which would work except that it doesn't connect to the internet either (Can't ping 8.8.8.8).

The packages I have installed are: qemu-full, dnsmasq, libvirt, virt-manager and iptables-nft (just as suggested by the libvirt page at the wiki).

journalctl -b -u libvirtd

Sep 06 22:53:16 archpc systemd[1]: Starting Virtualization daemon...
Sep 06 22:53:16 archpc systemd[1]: Started Virtualization daemon.
Sep 06 22:53:17 archpc dnsmasq[664]: started, version 2.89 cachesize 150
Sep 06 22:53:17 archpc dnsmasq[664]: compile time options: IPv6 GNU-getopt DBus no-UBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset nftset au>
Sep 06 22:53:17 archpc dnsmasq-dhcp[664]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
Sep 06 22:53:17 archpc dnsmasq-dhcp[664]: DHCP, sockets bound exclusively to interface virbr0
Sep 06 22:53:17 archpc dnsmasq[664]: no servers found in /etc/resolv.conf, will retry
Sep 06 22:53:17 archpc dnsmasq[664]: read /etc/hosts - 0 names
Sep 06 22:53:17 archpc dnsmasq[664]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 names
Sep 06 22:53:17 archpc dnsmasq-dhcp[664]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Sep 06 22:53:24 archpc dnsmasq[664]: reading /etc/resolv.conf
Sep 06 22:53:24 archpc dnsmasq[664]: using nameserver 80.58.61.250#53
Sep 06 22:53:24 archpc dnsmasq[664]: using nameserver 80.58.61.254#53

cat /etc/nsswitch.conf

# Generated by NetworkManager
nameserver 80.58.61.250
nameserver 80.58.61.254

ip address show virbr0

4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:9e:af:69 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever

brctl show

bridge name	bridge id		STP enabled	interfaces
virbr0		8000.5254009eaf69	yes		

journalctl -xeu dnsmasq.service

No entries

sudo virsh net-dhcp-leases default

Expiry Time   MAC address   Protocol   IP address   Hostname   Client ID or DUID
-----------------------------------------------------------------------------------

sudo virsh net-dumpxml default

<network>
  <name>default</name>
  <uuid>06b8f2ad-353b-40ca-b8c8-2d5bb4efbbf5</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr0' stp='on' delay='0'/>
  <mac address='52:54:00:9e:af:69'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>

sudo virsh dumpxml gnsysserv

<domain type='kvm'>
  <name>gnsysserv</name>
  <uuid>8bf1be6c-ed4b-43ee-b1e8-ee45592e0115</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://ubuntu.com/ubuntu/20.04"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-8.1'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'/>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' discard='unmap'/>
      <source file='/var/lib/libvirt/images/gnsysserv-1.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='sda' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x18'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='10' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='10' port='0x19'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x1a'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x1b'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
    </controller>
    <controller type='pci' index='13' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='13' port='0x1c'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
    </controller>
    <controller type='pci' index='14' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='14' port='0x1d'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
    </controller>
    <controller type='pci' index='15' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='15' port='0x1e'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x6'/>
    </controller>
    <controller type='pci' index='16' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:9e:a3:3d'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='unix'>
      <target type='virtio' name='org.qemu.guest_agent.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='2'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <audio id='1' type='spice'/>
    <video>
      <model type='virtio' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <watchdog model='itco' action='reset'/>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </memballoon>
    <rng model='virtio'>
      <backend model='random'>/dev/urandom</backend>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </rng>
  </devices>
</domain>

systemctl –type=service

  UNIT                                                  LOAD   ACTIVE SUB     DESCRIPTION
  accounts-daemon.service                               loaded active running Accounts Service
  bolt.service                                          loaded active running Thunderbolt system service
  colord.service                                        loaded active running Manage, Install and Generate Color Profiles
  dbus.service                                          loaded active running D-Bus System Message Bus
  gdm.service                                           loaded active running GNOME Display Manager
  kmod-static-nodes.service                             loaded active exited  Create List of Static Device Nodes
  ldconfig.service                                      loaded active exited  Rebuild Dynamic Linker Cache
  lvm2-monitor.service                                  loaded active exited  Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling
  NetworkManager.service                                loaded active running Network Manager
  polkit.service                                        loaded active running Authorization Manager
  rtkit-daemon.service                                  loaded active running RealtimeKit Scheduling Policy Service
  systemd-backlight@backlight:intel_backlight.service   loaded active exited  Load/Save Screen Backlight Brightness of backlight:intel_backlight
  systemd-backlight@leds:tpacpi::kbd_backlight.service  loaded active exited  Load/Save Screen Backlight Brightness of leds:tpacpi::kbd_backlight
  systemd-fsck@dev-disk-by\x2duuid-F50D\x2dD237.service loaded active exited  File System Check on /dev/disk/by-uuid/F50D-D237
  systemd-journal-catalog-update.service                loaded active exited  Rebuild Journal Catalog
  systemd-journal-flush.service                         loaded active exited  Flush Journal to Persistent Storage
  systemd-journald.service                              loaded active running Journal Service
  systemd-logind.service                                loaded active running User Login Management
  systemd-machined.service                              loaded active running Virtual Machine and Container Registration Service
  systemd-modules-load.service                          loaded active exited  Load Kernel Modules
  systemd-random-seed.service                           loaded active exited  Load/Save OS Random Seed
  systemd-remount-fs.service                            loaded active exited  Remount Root and Kernel File Systems
  systemd-sysctl.service                                loaded active exited  Apply Kernel Variables
  systemd-sysusers.service                              loaded active exited  Create System Users
  systemd-timesyncd.service                             loaded active running Network Time Synchronization
  systemd-tmpfiles-setup-dev.service                    loaded active exited  Create Static Device Nodes in /dev
  systemd-tmpfiles-setup.service                        loaded active exited  Create Volatile Files and Directories
  systemd-udev-trigger.service                          loaded active exited  Coldplug All udev Devices
  systemd-udevd.service                                 loaded active running Rule-based Manager for Device Events and Files
  systemd-update-done.service                           loaded active exited  Update is Completed
  systemd-update-utmp.service                           loaded active exited  Record System Boot/Shutdown in UTMP
  systemd-user-sessions.service                         loaded active exited  Permit User Sessions
  systemd-vconsole-setup.service                        loaded active exited  Virtual Console Setup
  udisks2.service                                       loaded active running Disk Manager
  upower.service                                        loaded active running Daemon for power management
  user-runtime-dir@1000.service                         loaded active exited  User Runtime Directory /run/user/1000
  user@1000.service                                     loaded active running User Manager for UID 1000
  wpa_supplicant.service                                loaded active running WPA supplicant

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
38 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

sudo nft list ruleset

table inet filter {
	chain input {
		type filter hook input priority filter; policy drop;
		ct state invalid drop comment "early drop of invalid connections"
		ct state { established, related } accept comment "allow tracked connections"
		iifname "lo" accept comment "allow from loopback"
		ip protocol icmp accept comment "allow icmp"
		meta l4proto ipv6-icmp accept comment "allow icmp v6"
		tcp dport 22 accept comment "allow sshd"
		meta pkttype host limit rate 5/second counter packets 15 bytes 1440 reject with icmpx admin-prohibited
		counter packets 93 bytes 4056
	}

	chain forward {
		type filter hook forward priority filter; policy drop;
	}
}
table ip filter {
	chain LIBVIRT_INP {
		iifname "virbr0" udp dport 53 counter packets 0 bytes 0 accept
		iifname "virbr0" tcp dport 53 counter packets 0 bytes 0 accept
		iifname "virbr0" udp dport 67 counter packets 0 bytes 0 accept
		iifname "virbr0" tcp dport 67 counter packets 0 bytes 0 accept
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		counter packets 948 bytes 394024 jump LIBVIRT_INP
	}

	chain LIBVIRT_OUT {
		oifname "virbr0" udp dport 53 counter packets 0 bytes 0 accept
		oifname "virbr0" tcp dport 53 counter packets 0 bytes 0 accept
		oifname "virbr0" udp dport 68 counter packets 0 bytes 0 accept
		oifname "virbr0" tcp dport 68 counter packets 0 bytes 0 accept
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 926 bytes 91707 jump LIBVIRT_OUT
	}

	chain LIBVIRT_FWO {
		iifname "virbr0" ip saddr 192.168.122.0/24 counter packets 0 bytes 0 accept
		iifname "virbr0" counter packets 0 bytes 0 xt target "REJECT"
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump LIBVIRT_FWX
		counter packets 0 bytes 0 jump LIBVIRT_FWI
		counter packets 0 bytes 0 jump LIBVIRT_FWO
	}

	chain LIBVIRT_FWI {
		oifname "virbr0" ip daddr 192.168.122.0/24 xt match "conntrack" counter packets 0 bytes 0 accept
		oifname "virbr0" counter packets 0 bytes 0 xt target "REJECT"
	}

	chain LIBVIRT_FWX {
		iifname "virbr0" oifname "virbr0" counter packets 0 bytes 0 accept
	}
}
table ip nat {
	chain LIBVIRT_PRT {
		ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 0 bytes 0 return
		ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
		meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 xt target "MASQUERADE"
		meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 xt target "MASQUERADE"
		ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 xt target "MASQUERADE"
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		counter packets 206 bytes 17770 jump LIBVIRT_PRT
	}
}
table ip mangle {
	chain LIBVIRT_PRT {
		oifname "virbr0" udp dport 68 counter packets 0 bytes 0 xt target "CHECKSUM"
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
		counter packets 932 bytes 92979 jump LIBVIRT_PRT
	}
}
table ip6 filter {
	chain LIBVIRT_INP {
	}

	chain INPUT {
		type filter hook input priority filter; policy accept;
		counter packets 266 bytes 18732 jump LIBVIRT_INP
	}

	chain LIBVIRT_OUT {
	}

	chain OUTPUT {
		type filter hook output priority filter; policy accept;
		counter packets 155 bytes 12836 jump LIBVIRT_OUT
	}

	chain LIBVIRT_FWO {
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump LIBVIRT_FWX
		counter packets 0 bytes 0 jump LIBVIRT_FWI
		counter packets 0 bytes 0 jump LIBVIRT_FWO
	}

	chain LIBVIRT_FWI {
	}

	chain LIBVIRT_FWX {
	}
}
table ip6 nat {
	chain LIBVIRT_PRT {
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		counter packets 15 bytes 1344 jump LIBVIRT_PRT
	}
}
table ip6 mangle {
	chain LIBVIRT_PRT {
	}

	chain POSTROUTING {
		type filter hook postrouting priority mangle; policy accept;
		counter packets 155 bytes 12836 jump LIBVIRT_PRT
	}
}

Offline

#2 2023-09-06 21:47:53

Sarig
Member
Registered: 2020-12-03
Posts: 4

Re: [KVM/QEMU/Libvirt] NAT `default` network not working as expected

I had exact same problem. (systemd-networkd)
sudo nano /etc/resolv.conf
  domain lan
  nameserver 192.168.1.1
  nameserver fd8a:35ce:18f2::3
sudo systemctl restart libvirtd.service

Offline

#3 2023-09-07 08:30:24

oysterpingu
Member
Registered: 2023-09-06
Posts: 3

Re: [KVM/QEMU/Libvirt] NAT `default` network not working as expected

I am using NetworkManager,  that auto fills /etc/resolv.conf with

sudo nano /etc/resolv.conf
# Generated by NetworkManager
nameserver 8.8.8.8
nameserver 8.8.4.4

Last edited by oysterpingu (2023-09-07 08:31:36)

Offline

#4 2023-09-07 10:57:06

oysterpingu
Member
Registered: 2023-09-06
Posts: 3

Re: [KVM/QEMU/Libvirt] NAT `default` network not working as expected

I statically configured my vm with an ip and pinged 8.8.8.8. If I use tcpdump to view, from my host, what's being received from virbr0:

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on virbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
12:52:48.885346 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28
12:52:49.911881 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28
12:52:50.935670 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28
12:52:51.960132 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28
12:52:52.983881 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28
12:52:54.007699 ARP, Request who-has 8.8.8.8 tell 192.168.122.0, length 28

This looks like the vm is connected to the network just fine, however the host isnt properly redirecting the 8.8.8.8 request.

The vm output when doing `ping 8.8.8.8` is destination host unreachable.

Last edited by oysterpingu (2023-09-07 10:59:00)

Offline

Board footer

Powered by FluxBB