systemd-networkd-wait-online.service failure, but network is still up

shrum · 2024-08-12 11:09:57

I noticed that systemd-networkd-wait-online.service is dead:

 $ systemctl status systemd-networkd-wait-online.service
× systemd-networkd-wait-online.service - Wait for Network to be Configured
     Loaded: loaded (/usr/lib/systemd/system/systemd-networkd-wait-online.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Mon 2024-08-12 11:52:33 BST; 8min ago
 Invocation: e8d6b3bc91174af6ba2ca7e16a9a1731
       Docs: man:systemd-networkd-wait-online.service(8)
    Process: 276139 ExecStart=/usr/lib/systemd/systemd-networkd-wait-online (code=exited, status=1/FAILURE)
   Main PID: 276139 (code=exited, status=1/FAILURE)
   Mem peak: 1.5M
        CPU: 10ms

But I still have access to the internet via wlan0.

Looking at the config it seems to run:

/usr/lib/systemd/systemd-networkd-wait-online

And this ELF binary is behaving quite surprisingly for me:

 $  /usr/lib/systemd/systemd-networkd-wait-online --interface=wlan0 --timeout=2
<success>
 $  /usr/lib/systemd/systemd-networkd-wait-online --any --timeout=2
Timeout occurred while waiting for network connectivity.
<failure>

My setup is pretty basic I think:

 $ ls /sys/class/net
lo  wlan0

The service starts failing after coming back from hibernation.

Does anyone have any idea what might be wrong here?

Last edited by shrum (2024-08-12 11:15:19)

seth · 2024-08-12 15:32:21

Are you using networkd tbw?
Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Also see the caveat in https://wiki.archlinux.org/title/System … ait-online because the service likely doesn't do what you want it to do anyway.
And from there

waits for all links managed by systemd-networkd

=> https://wiki.archlinux.org/title/System … tion_files - do you have any stale links configured?

shrum · 2024-08-12 16:13:11

$ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
bluetooth.service                        | bluetooth.target.wants
dbus-org.bluez.service                   | system
dbus-org.freedesktop.network1.service    | system
dbus-org.freedesktop.resolve1.service    | system
dhcpcd.service                           | multi-user.target.wants
getty@tty1.service                       | getty.target.wants
iwd.service                              | multi-user.target.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire-pulse.socket                    | sockets.target.wants
pipewire-session-manager.service         | user
pipewire.socket                          | sockets.target.wants
remote-fs.target                         | multi-user.target.wants
systemd-networkd.service                 | multi-user.target.wants
systemd-networkd.socket                  | sockets.target.wants
systemd-networkd-wait-online.service     | network-online.target.wants
systemd-network-generator.service        | sysinit.target.wants
systemd-resolved.service                 | sysinit.target.wants
wireplumber.service                      | pipewire.service.wants
xdg-user-dirs-update.service             | default.target.wants

It doesn't look like I have some other links, although the IDX being 3 for wlan0 looks suspicious.

$ networkctl list --all
IDX LINK  TYPE     OPERATIONAL SETUP    
  1 lo    loopback carrier     unmanaged
  3 wlan0 wlan     routable    unmanaged

2 links listed.

I will just update the service to call the binary with --interface=wlan0 for now, as I don't really use other interfaces for online connectivity.

seth · 2024-08-12 16:20:34

Disable dhcpcd.service - b/c of the questioable index counting, don't rely on some networkctl output, check the files in the config dirs.

shrum · 2024-08-13 11:01:26

I disabled dhcpcd.service.

I think I rely on the config provided with the package:

 $ ls /etc/systemd/network/
<empty>

 $ ls /usr/lib/systemd/network/*.{network,link}
/usr/lib/systemd/network/80-6rd-tunnel.link              /usr/lib/systemd/network/80-container-ve.link     /usr/lib/systemd/network/80-namespace-ns.network
/usr/lib/systemd/network/80-6rd-tunnel.network           /usr/lib/systemd/network/80-container-ve.network  /usr/lib/systemd/network/80-vm-vt.link
/usr/lib/systemd/network/80-container-host0.network      /usr/lib/systemd/network/80-container-vz.link     /usr/lib/systemd/network/80-vm-vt.network
/usr/lib/systemd/network/80-container-host0-tun.network  /usr/lib/systemd/network/80-container-vz.network  /usr/lib/systemd/network/80-wifi-adhoc.network
/usr/lib/systemd/network/80-container-vb.link            /usr/lib/systemd/network/80-iwd.link              /usr/lib/systemd/network/99-default.link
/usr/lib/systemd/network/80-container-vb.network         /usr/lib/systemd/network/80-namespace-ns.link

($ ls | xargs -I {} pacman -Qo {} says all files are owned by systemd)

The only relevant config ( (ignoring the filed with "v", as I'm not running this in any virtual environment) seems to be:

$ cat  /usr/lib/systemd/network/80-iwd.link
[Match]
Type=wlan

[Link]
NamePolicy=keep kernel

 $ cat 80-wifi-adhoc.network
# SPDX-License-Identifier: MIT-0
#
# This config file is installed as part of systemd.
# It may be freely copied and edited (following the MIT No Attribution license).
#
# To make local modifications, use "networkctl edit". See networkctl(1) for details.
# This file should not be edited in place, because it'll be overwritten on upgrades.

[Match]
Type=wlan
WLANInterfaceType=ad-hoc

[Network]
LinkLocalAddressing=yes
IPv6AcceptRA=no
MulticastDNS=yes

$ cat  99-default.link
# SPDX-License-Identifier: MIT-0
#
# This config file is installed as part of systemd.
# It may be freely copied and edited (following the MIT No Attribution license).
#
# To make local modifications, use "networkctl edit". See networkctl(1) for details.
# This file should not be edited in place, because it'll be overwritten on upgrades.

[Match]
OriginalName=*

[Link]
NamePolicy=keep kernel database onboard slot path
AlternativeNamesPolicy=database onboard slot path
MACAddressPolicy=persistent

I don't really fully understand this configuration.

seth · 2024-08-13 22:41:03

Since explicitly selecting the interface works, what about explicitly ignoring the loopback device (should™ be default, though)

/usr/lib/systemd/systemd-networkd-wait-online --any --ignore=lo --timeout=30 # also raise the timeout a bit

Because iwd and the NIC situation, do you maybe https://wiki.archlinux.org/title/Iwd#En … figuration
Did you reboot after disabling dhcpcd?

shrum · 2024-08-14 08:36:16

I tried

 $ /usr/lib/systemd/systemd-networkd-wait-online --any --ignore=lo --timeout=30

before reboot and it also failed.

After disabling dhcpd.service and a reboot the problem didn't reoccur yet and I am not able to reproduce it (I have no idea what triggers it).

$ networkctl list --all
IDX LINK  TYPE     OPERATIONAL SETUP     
  1 lo    loopback carrier     unmanaged
  3 wlan0 wlan     routable    configured

2 links listed.

wlan0 still appears with index 3.

seth · 2024-08-14 08:41:23

shrum wrote:

After disabling dhcpd.service and a reboot the problem didn't reoccur yet and I am not able to reproduce it (I have no idea what triggers it).

The cross-test would be to re-enable dhcpcd and then run into it again pretty fast.

If you don't and unless it shows up again in the next couple of days, please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

Arch Linux

#1 2024-08-12 11:09:57

systemd-networkd-wait-online.service failure, but network is still up

#2 2024-08-12 15:32:21

Re: systemd-networkd-wait-online.service failure, but network is still up

#3 2024-08-12 16:13:11

Re: systemd-networkd-wait-online.service failure, but network is still up

#4 2024-08-12 16:20:34

Re: systemd-networkd-wait-online.service failure, but network is still up

#5 2024-08-13 11:01:26

Re: systemd-networkd-wait-online.service failure, but network is still up

#6 2024-08-13 22:41:03

Re: systemd-networkd-wait-online.service failure, but network is still up

#7 2024-08-14 08:36:16

Re: systemd-networkd-wait-online.service failure, but network is still up

#8 2024-08-14 08:41:23

Re: systemd-networkd-wait-online.service failure, but network is still up

Board footer