No network after suspend with Broadcom BCM57406 ethernet NIC

grbrady · 2025-01-21 04:47:44

I'm having an issue where when resuming from suspend I lose my network connection. The only way I can get it back is by rebooting. I'm running an AMD Ryzen 9 5900X on an Asus Pro WS X570-ACE motherboard with 128GB of RAM and a 3080Ti GPU, along with the aforementioned Broadcom BCM57406 NIC.

Here is the output of lshw -C network while everything is working correctly:

  *-network:0               
       description: Ethernet interface
       product: BCM57406 NetXtreme-E 10GBASE-T Ethernet Controller
       vendor: Broadcom Inc. and subsidiaries
       physical id: 0
       bus info: pci@0000:04:00.0
       logical name: enp4s0f0np0
       version: 01
       serial: 00:0a:f7:97:e3:48
       capacity: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm vpd msi msix pciexpress bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=bnxt_en driverversion=6.12.10-arch1-1 firmware=212.0.162.0/pkg 21.60.22.11 latency=0 link=no multicast=yes port=twisted pair
       resources: irq:11 memory:e2130000-e213ffff memory:e2120000-e212ffff memory:e2142000-e2143fff memory:fc580000-fc5fffff
  *-network:1
       description: Ethernet interface
       product: BCM57406 NetXtreme-E 10GBASE-T Ethernet Controller
       vendor: Broadcom Inc. and subsidiaries
       physical id: 0.1
       bus info: pci@0000:04:00.1
       logical name: enp4s0f1np1
       version: 01
       serial: 00:0a:f7:97:e3:49
       size: 10Gbit/s
       capacity: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm vpd msi msix pciexpress bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=bnxt_en driverversion=6.12.10-arch1-1 duplex=full firmware=212.0.162.0/pkg 21.60.22.11 ip=192.168.1.248 latency=0 link=yes multicast=yes port=twisted pair speed=10Gbit/s
       resources: irq:11 memory:e2110000-e211ffff memory:e2100000-e210ffff memory:e2140000-e2141fff memory:fc500000-fc57ffff

and to contrast, when things aren't working correctly after resuming from suspend:

  *-network:0               
       description: Ethernet interface
       product: BCM57406 NetXtreme-E 10GBASE-T Ethernet Controller
       vendor: Broadcom Inc. and subsidiaries
       physical id: 0
       bus info: pci@0000:04:00.0
       logical name: enp4s0f0np0
       version: 01
       serial: 00:0a:f7:97:e3:48
       capacity: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm vpd msi msix pciexpress bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=bnxt_en driverversion=6.12.10-arch1-1 firmware=212.0.162.0 latency=0 link=no multicast=yes port=twisted pair
       resources: irq:11 memory:e2130000-e213ffff memory:e2120000-e212ffff memory:e2142000-e2143fff memory:fc580000-fc5fffff
  *-network:1
       description: Ethernet interface
       product: BCM57406 NetXtreme-E 10GBASE-T Ethernet Controller
       vendor: Broadcom Inc. and subsidiaries
       physical id: 0.1
       bus info: pci@0000:04:00.1
       logical name: enp4s0f1np1
       version: 01
       serial: 00:0a:f7:97:e3:49
       size: 10Gbit/s
       capacity: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm vpd msi msix pciexpress bus_master cap_list rom ethernet physical tp 1000bt-fd 10000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=bnxt_en driverversion=6.12.10-arch1-1 duplex=full firmware=212.0.162.0 latency=0 link=yes multicast=yes port=twisted pair speed=10Gbit/s
       resources: irq:11 memory:e2110000-e211ffff memory:e2100000-e210ffff memory:e2140000-e2141fff memory:fc500000-fc57ffff

Here is what I see in the journal when the suspend is starting

Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.6518] manager: sleep: sleep requested (sleeping: no  enabled: yes)
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.6519] device (enp4s0f0np0): state change: unavailable -> unmanaged (reason 'unmanaged-sleeping', managed-type: 'full')
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7253] manager: NetworkManager state is now ASLEEP
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7255] device (enp4s0f1np1): state change: activated -> deactivating (reason 'sleeping', managed-type: 'full')
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7746] device (enp4s0f1np1): state change: deactivating -> disconnected (reason 'sleeping', managed-type: 'full')
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7749] dhcp4 (enp4s0f1np1): canceled DHCP transaction
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7749] dhcp4 (enp4s0f1np1): activation: beginning transaction (timeout in 45 seconds)
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.7749] dhcp4 (enp4s0f1np1): state changed no lease
Jan 20 22:55:50 archie NetworkManager[647]: <info>  [1737431750.8607] device (enp4s0f1np1): state change: disconnected -> unmanaged (reason 'unmanaged-sleeping', managed-type: 'full')

And the most pertinent entries after resuming

Jan 20 22:56:43 archie kernel: bnxt_en 0000:04:00.0 enp4s0f0np0: NIC Link is Down
Jan 20 22:58:59 archie NetworkManager[647]: <warn>  [1737431939.6865] device (enp4s0f1np1): Activation: failed for connection 'Wired connection 2'
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.6868] device (enp4s0f1np1): state change: failed -> disconnected (reason 'none', managed-type: 'full')
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7050] dhcp4 (enp4s0f1np1): canceled DHCP transaction
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7050] dhcp4 (enp4s0f1np1): activation: beginning transaction (timeout in 45 seconds)
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7050] dhcp4 (enp4s0f1np1): state changed no lease
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7056] policy: auto-activating connection 'Wired connection 2' (9f42dc18-9f93-3ee6-8eb5-ecbce36da3c2)
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7059] device (enp4s0f1np1): Activation: starting connection 'Wired connection 2' (9f42dc18-9f93-3ee6-8eb5-ecbce36da3c2)
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7060] device (enp4s0f1np1): state change: disconnected -> prepare (reason 'none', managed-type: 'full')
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7062] manager: NetworkManager state is now CONNECTING
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7064] device (enp4s0f1np1): state change: prepare -> config (reason 'none', managed-type: 'full')
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7067] device (enp4s0f1np1): state change: config -> ip-config (reason 'none', managed-type: 'full')
Jan 20 22:58:59 archie NetworkManager[647]: <info>  [1737431939.7070] dhcp4 (enp4s0f1np1): activation: beginning transaction (timeout in 45 seconds)
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.6921] device (enp4s0f1np1): state change: ip-config -> failed (reason 'ip-config-unavailable', managed-type: 'full')
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.6924] manager: NetworkManager state is now DISCONNECTED
Jan 20 22:59:44 archie NetworkManager[647]: <warn>  [1737431984.6926] device (enp4s0f1np1): Activation: failed for connection 'Wired connection 2'
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.6927] device (enp4s0f1np1): state change: failed -> disconnected (reason 'none', managed-type: 'full')
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.7183] dhcp4 (enp4s0f1np1): canceled DHCP transaction
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.7183] dhcp4 (enp4s0f1np1): activation: beginning transaction (timeout in 45 seconds)
Jan 20 22:59:44 archie NetworkManager[647]: <info>  [1737431984.7183] dhcp4 (enp4s0f1np1): state changed no lease

Things I have tried to resolve without working:

Getting a new DHCP lease with

 networkctl renew enp4s0f1np1

Restarting networking with

systemctl restart systemd-networkd

and

systemctl restart systemd-resolvd

Bringing the link up and down with

 ip link set enp4s0f1np1 down

and

 ip link set enp4s0f1np1 up

Restarting the driver with

 modprobe -r bnxt_en

and

 modprobe bnxt_en

Removing and reloading the PCI device for the ethernet devices

echo '1' > /sys/bus/pci/devices/0000\:04\:00.0/remove
echo '1' > /sys/bus/pci/devices/0000\:04\:00.1/remove
echo '1' > /sys/bus/pci/devices/rescan

Turning Energy Efficient Ethernet and LPI on and off doesn't seem to make any difference

ethtool --set-eee enp4s0f0np0 eee off tx-lpi off
ethtool --set-eee enp4s0f0np0 eee on tx-lpi on

I'm out of ideas right now.

One interesting data point: I have very similar behavior on this hardware running Windows 11 when I put the system to sleep and then wake it I can't get a network connection until I reboot. Windows isn't great about supplying diagnostic information so I have no further information to provide, but they seem like they could be related.

Last edited by grbrady (2025-01-21 04:49:55)

-thc · 2025-01-21 07:10:40

grbrady wrote:

One interesting data point: I have very similar behavior on this hardware running Windows 11[...]

So this is a dual boot system?

And you know that a normal Windows (8+) shutdown is not a real shutdown but a "hybrid" one and this hibernation is keeping the WiFi hardware in a non-fully powered down state?

What the Wiki article section fails to emphasize is that "hybrid shutdown" with hibernation is the Windows default now.

Try disabling that.

Last edited by -thc (2025-01-21 07:11:11)

grbrady · 2025-01-22 02:14:17

I tried another 10 Gbe NIC and it seems to be able to resume from suspend with a network connection with that card. I guess the Broadcom card has some firmware or other issue that causes a problem with suspend which, to be fair, as a server card probably wasn't a significant design requirement. I tried to update the firmware but this model was originally OEM for Dell. The firmware update tool is thus provided by Dell and does not seem to be able to run on non-original, non-Dell hardware. I wasn't able to make the Broadcom utility for similar cards to work, either.

So my particular issue is solved, but the Broadcom BCM57406 is probably not the best choice for general desktop use.

The card I replaced it with is a "noname" Chinese card branded NICGIGA is using the Marvell/Aquantia AQtion AQC-113 chipset. So far it seems quite well-behaved and checks out with speeds just less than 10 Gbps.

Last edited by grbrady (2025-01-22 02:15:00)

seth · 2025-01-22 07:55:16

I guess the Broadcom card has … issue

https://wiki.archlinux.org/title/Broadcom_wireless (though hat's wireless specific)

This btw. doesn't remedy the dual-boot concerns, the hibernating windows leaves the HW in an unpredictably state causing random behavior.

I have very similar behavior on this hardware running Windows 11 when I put the system to sleep and then wake it I can't get a network connection until I reboot

Is *extremely* suspcicious already.

Getting a new DHCP lease with
networkctl renew enp4s0f1np1
Restarting networking with
systemctl restart systemd-networkd
and
systemctl restart systemd-resolvd

Your journal snippets show you running NM - do not run multiple services concurrently.
Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Finally if you want more informed comments (well, potentially - wdk what's there) please post your complete system journal for a boot covering the loss, eg.

sudo journalctl -b -3 | curl -F 'file=@-' 0x0.st

for three boots ago.

Something™ brought the NIC down and you're apparently timing out on the dhcp request (what's not very surprising giving the NIC is down)

Beyond all of that and for a pot. workaround: have you tried to disconnect the network and unload the bnxt_en module before the suspend and loading + reconnecting after the suspend (eg. w/ a sleep hook)?

Arch Linux

#1 2025-01-21 04:47:44

No network after suspend with Broadcom BCM57406 ethernet NIC

#2 2025-01-21 07:10:40

Re: No network after suspend with Broadcom BCM57406 ethernet NIC

#3 2025-01-22 02:14:17

Re: No network after suspend with Broadcom BCM57406 ethernet NIC

#4 2025-01-22 07:55:16

Re: No network after suspend with Broadcom BCM57406 ethernet NIC

Board footer