[SOLVED] libvirt NAT network doesn't forward traffic

gtarch · 2025-08-20 08:29:10

I have this problem with virt-manager where network traffic doesn't reach the bridge device and I can't get an IP in the guest.

This is the network configuration.

<interface type="network">
  <mac address="52:54:00:89:da:c6"/>
  <source network="default" portid="3673f38b-e2a1-49dd-929c-c18d0fa960f3" bridge="virbr0"/>
  <target dev="vnet0"/>
  <model type="virtio"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
</interface>

I have dnsmasq running:

ss -tlnp                                                                                                                                       
State          Recv-Q         Send-Q                 Local Address:Port                  Peer Address:Port        Process
LISTEN         0              32                     192.168.122.1:53                         0.0.0.0:*            users:(("dnsmasq",pid=704404,fd=6))

The host network is:

ip a
6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb state DOWN group default qlen 1000
    link/ether 52:54:00:6c:01:f2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
41: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fe:54:00:89:da:c6 brd ff:ff:ff:ff:ff:ff

The DHCP requests from the guest show up on the guest interface

tcpdump  -ni vnet0
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
11:23:31.105080 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 52:54:00:89:da:c6, length 285

but not on the bridge device

The netfilter rules are:

nft list ruleset
table ip libvirt_network {
	chain forward {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump guest_cross
		counter packets 0 bytes 0 jump guest_input
		counter packets 0 bytes 0 jump guest_output
	}

	chain guest_output {
		ip saddr 192.168.122.0/24 iif "virbr0" counter packets 0 bytes 0 accept
		iif "virbr0" counter packets 0 bytes 0 reject
	}

	chain guest_input {
		oif "virbr0" ip daddr 192.168.122.0/24 ct state established,related counter packets 0 bytes 0 accept
		oif "virbr0" counter packets 0 bytes 0 reject
	}

	chain guest_cross {
		iif "virbr0" oif "virbr0" counter packets 0 bytes 0 accept
	}

	chain guest_nat {
		type nat hook postrouting priority srcnat; policy accept;
		ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 0 bytes 0 return
		ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
		meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
		meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
		ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade
	}
}
table ip6 libvirt_network {
	chain forward {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump guest_cross
		counter packets 0 bytes 0 jump guest_input
		counter packets 0 bytes 0 jump guest_output
	}

	chain guest_output {
	}

	chain guest_input {
	}

	chain guest_cross {
	}

	chain guest_nat {
		type nat hook postrouting priority srcnat; policy accept;
	}
}

and ip_forward is enabled.

What else can I check?

Last edited by gtarch (2025-08-22 12:13:19)

tekstryder · 2025-08-20 14:22:09

You didn't supply your inet filter table. Not sure if you're obfuscating for privacy?

Please provide the full output of sudo nft list ruleset.

I'll just guess this might be missing from the input chain:

  chain input {
    iifname virbr0 udp dport {53, 67} accept comment "allow libvirt NAT dhcp/dns requests to host"
}

and add to forward chain:

  chain forward {
    iifname virbr0 accept
    oifname virbr0 accept
  }

gtarch · 2025-08-21 07:36:30

tekstryder wrote:

You didn't supply your inet filter table. Not sure if you're obfuscating for privacy?
Please provide the full output of sudo nft list ruleset.

It's the full output. I just diff-ed it with the output on a machine that works and they are identical - no forward nor input chains other than the listed on either.

Last edited by gtarch (2025-08-21 07:39:13)

tekstryder · 2025-08-21 11:36:46

Please provide:

$ cat /etc/nftables.conf

EDIT: and...

# iptables -nvL

Last edited by tekstryder (2025-08-21 12:15:45)

gtarch · 2025-08-21 12:35:16

#!/usr/bin/nft -f
# vim:set ts=2 sw=2 et:

# IPv4/IPv6 Simple & Safe firewall ruleset.
# More examples in /usr/share/nftables/ and /usr/share/doc/nftables/examples/.

destroy table inet filter
table inet filter {
  chain input {
    type filter hook input priority filter
    policy drop

    ct state invalid drop comment "early drop of invalid connections"
    ct state {established, related} accept comment "allow tracked connections"
    iif lo accept comment "allow from loopback"
    iifname virbr0 udp dport {53, 67} accept comment "allow VM dhcp/dns requests to host"
    ip protocol icmp accept comment "allow icmp"
    meta l4proto ipv6-icmp accept comment "allow icmp v6"
    tcp dport ssh accept comment "allow sshd"
    pkttype host limit rate 5/second counter reject with icmpx type admin-prohibited
    counter
  }
  chain forward {
    type filter hook forward priority filter
    policy drop

    iifname virbr0 accept
    oifname virbr0 accept
  }
}

Chain INPUT (policy ACCEPT 2520K packets, 12G bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain OUTPUT (policy ACCEPT 1739K packets, 230M bytes)
 pkts bytes target     prot opt in     out     source               destination

Last edited by gtarch (2025-08-21 12:38:51)

tekstryder · 2025-08-21 12:47:05

Okay, that actually looks good and seems the rules simply weren't applied.

# nft -f /etc/nftables.conf

Then, ensure they are present with

# nft list ruleset

If the inet filter table is now enabled, fire up the VM and see if NAT is working.

If not, you may need to restart libvirt's network stack. Depending on whether you're using the monolithic or modular daemons, this can be tricky to walk you through.

EDIT: typo: .conf

Last edited by tekstryder (2025-08-21 12:58:43)

gtarch · 2025-08-21 13:04:41

tekstryder wrote:

Okay, that actually looks good and seems the rules simply weren't applied.

They aren't because the service isn't enabled nor needed. Nevertheless I ran

# nft -f /etc/nftables
# nft list ruleset
table ip libvirt_network {
	chain forward {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump guest_cross
		counter packets 0 bytes 0 jump guest_input
		counter packets 0 bytes 0 jump guest_output
	}

	chain guest_output {
		ip saddr 192.168.122.0/24 iif "virbr0" counter packets 0 bytes 0 accept
		iif "virbr0" counter packets 0 bytes 0 reject
	}

	chain guest_input {
		oif "virbr0" ip daddr 192.168.122.0/24 ct state established,related counter packets 0 bytes 0 accept
		oif "virbr0" counter packets 0 bytes 0 reject
	}

	chain guest_cross {
		iif "virbr0" oif "virbr0" counter packets 0 bytes 0 accept
	}

	chain guest_nat {
		type nat hook postrouting priority srcnat; policy accept;
		ip saddr 192.168.122.0/24 ip daddr 224.0.0.0/24 counter packets 0 bytes 0 return
		ip saddr 192.168.122.0/24 ip daddr 255.255.255.255 counter packets 0 bytes 0 return
		meta l4proto tcp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
		meta l4proto udp ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade to :1024-65535
		ip saddr 192.168.122.0/24 ip daddr != 192.168.122.0/24 counter packets 0 bytes 0 masquerade
	}
}
table ip6 libvirt_network {
	chain forward {
		type filter hook forward priority filter; policy accept;
		counter packets 0 bytes 0 jump guest_cross
		counter packets 0 bytes 0 jump guest_input
		counter packets 0 bytes 0 jump guest_output
	}

	chain guest_output {
	}

	chain guest_input {
	}

	chain guest_cross {
	}

	chain guest_nat {
		type nat hook postrouting priority srcnat; policy accept;
	}
}
table inet filter {
	chain input {
		type filter hook input priority filter; policy drop;
		ct state invalid drop comment "early drop of invalid connections"
		ct state { established, related } accept comment "allow tracked connections"
		iif "lo" accept comment "allow from loopback"
		iifname "virbr0" udp dport { 53, 67 } accept comment "allow VM dhcp/dns requests to host"
		ip protocol icmp accept comment "allow icmp"
		meta l4proto ipv6-icmp accept comment "allow icmp v6"
		tcp dport 22 accept comment "allow sshd"
		meta pkttype host limit rate 5/second burst 5 packets counter packets 0 bytes 0 reject with icmpx admin-prohibited
		counter packets 171 bytes 46224
	}

	chain forward {
		type filter hook forward priority filter; policy drop;
		iifname "virbr0" accept
		oifname "virbr0" accept
	}
}

tekstryder wrote:

If not, you may need to restart libvirt's network stack. Depending on whether you're using the monolithic or modular daemons, this can be tricky to walk you through.

I don't know whether I'm running a monolithic daemon. It's a default install without manual config changes. I did restart libvirtd, but the network still doesn't work.

tekstryder · 2025-08-21 13:23:05

See my edit: nft -f /etc/nftables.conf

gtarch wrote:

They aren't because the service isn't enabled nor needed.

AFAIK, you won't have NAT available without one of the firewall backends.

gtarch · 2025-08-21 13:26:21

tekstryder wrote:

See my edit: nft -f /etc/nftables.conf

I did that. The results are in the previous post.

tekstryder wrote:

AFAIK, you won't have NAT available without one of the firewall backends.

That's correct, but the libvirt rules already do the NAT in my understanding. At least this is how it works on another machine.

tekstryder · 2025-08-21 13:44:40

So, given you understand that... try enabling nftables service, restarting libvirt networking services and see if it NAT works.

gtarch · 2025-08-21 13:49:16

tekstryder wrote:

So, given you understand that... try enabling nftables service, restarting libvirt networking services and see if it NAT works.

All the nftables service does is

nft -f /etc/nftables.conf

, which I already did manually. On other machines the NAT networkning works without it and on this one it doesn't even with it.

tekstryder · 2025-08-21 14:01:12

tekstryder wrote:

restarting libvirt networking services

You need the virtual network to go down and up.

How you achieve that (the teardown procedure/sequence) is up to how you've configured your services / sockets.

You might be able to brute-force it with virsh, but I've never needed, nor tried this:

virsh net-destroy default
virsh net-start default

Note: This is assuming default, and that other VMs/users aren't actively using the network in question.

EDIT:

As mentioned in your other libvirt thread, using modular daemons here.

In my case this is as simple as:

# killall dnsmasq
# systemctl stop virtnetworkd.service virtnetworkd.socket

Starting the same service/socket combo will reestablish the network.

EDIT2:

Ah, so I recreated your scenario by flushing my ruleset and the letting the default virtnetworkd netfilter rules apply solely.

I cannot reproduce your issue. NAT is working.

I was under the impression that you'd intended for the "simple firewall" to be active, so went down the wrong avenue.

One difference from your OP is that I'm emulating the e1000e driver rather than virtio:

<interface type="network">
  <mac address="52:54:00:4e:66:ad"/>
  <source network="default" portid="8f2d322d-1eea-4eca-af08-406942b6282d" bridge="virbr0"/>
  <target dev="vnet0"/>
  <model type="e1000e"/>
  <alias name="net0"/>
  <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
</interface>

<network connections="1">
  <name>default</name>
  <uuid>f9ef967a-a323-4ad6-8133-a53d722c39a4</uuid>
  <forward mode="nat">
    <nat>
      <port start="1024" end="65535"/>
    </nat>
  </forward>
  <bridge name="virbr0" stp="on" delay="0"/>
  <mac address="52:54:00:c6:ab:19"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
</network>

Last edited by tekstryder (2025-08-21 19:11:40)

gtarch · 2025-08-22 08:23:12

tekstryder wrote:

restarting libvirt networking services
You need the virtual network to go down and up.
How you achieve that (the teardown procedure/sequence) is up to how you've configured your services / sockets.
You might be able to brute-force it with virsh, but I've never needed, nor tried this:
virsh net-destroy default
virsh net-start default
Note: This is assuming default, and that other VMs/users aren't actively using the network in question.

Tried that, still nothing.

tekstryder wrote:

One difference from your OP is that I'm emulating the e1000e driver rather than virtio:

I tried the e1000e driver, but still nothing.

Where do you get this information from?

tekstryder wrote:

<network connections="1">
  <name>default</name>
  <uuid>f9ef967a-a323-4ad6-8133-a53d722c39a4</uuid>
  <forward mode="nat">
    <nat>
      <port start="1024" end="65535"/>
    </nat>
  </forward>
  <bridge name="virbr0" stp="on" delay="0"/>
  <mac address="52:54:00:c6:ab:19"/>
  <ip address="192.168.122.1" netmask="255.255.255.0">
    <dhcp>
      <range start="192.168.122.2" end="192.168.122.254"/>
    </dhcp>
  </ip>
</network>

I couldn't find it on either my working machine or the not working one.

What drives me crazy is that I have a number of machines with the same setup and the network problem is present only on one of them. I can't find any differences in the configuration. It's a frustrating mystery.

Anyway, thank you for your support!

tekstryder · 2025-08-22 11:40:08

gtarch wrote:

Where do you get this information from?

Launch Virt Manager and highlight a VM > Edit > Connection Details > Virtual Networks > XML

gtarch wrote:

What drives me crazy is that I have a number of machines with the same setup

Is the guest image(s) shared with / accessed by other systems, a known image propagated to multiple systems, or a standalone / unique guest VM?

Could there be an issue with the guest?

The next thing to look at on the host is the system journal while the network comes up.

You'll need to ensure all services are down first to start from a clean slate...

$ ps -ef | grep [v]irt

...should return exactly nothing.

Then, capture the journal while initializing the virtual network. In my case that's:

# systemctl start virtnetworkd.service

Please provide the system journal snippet of this process from net init until guest running.

gtarch · 2025-08-22 12:12:15

tekstryder wrote:

Please provide the system journal snippet of this process from net init until guest running.

Yes! I found the problem.

systemd-networkd[522231]: vnet0: Configuring with /etc/systemd/network/20-wired.network.

Apparently systemd was taking over the interface and preventing libvirt from configuring it properly. I had

[Match]
Type=ether

in 20-wired.network. I changed that to Name=<ethname> and now everything works properly!

Thank you so much for assisting me with this issue!

Last edited by gtarch (2025-08-22 12:12:53)

Arch Linux

#1 2025-08-20 08:29:10

[SOLVED] libvirt NAT network doesn't forward traffic

#2 2025-08-20 14:22:09

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#3 2025-08-21 07:36:30

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#4 2025-08-21 11:36:46

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#5 2025-08-21 12:35:16

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#6 2025-08-21 12:47:05

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#7 2025-08-21 13:04:41

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#8 2025-08-21 13:23:05

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#9 2025-08-21 13:26:21

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#10 2025-08-21 13:44:40

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#11 2025-08-21 13:49:16

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#12 2025-08-21 14:01:12

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#13 2025-08-22 08:23:12

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#14 2025-08-22 11:40:08

Re: [SOLVED] libvirt NAT network doesn't forward traffic

#15 2025-08-22 12:12:15

Re: [SOLVED] libvirt NAT network doesn't forward traffic

Board footer