Strange systemd behavior with network-online.target

amhairghin · 2024-12-09 09:27:15

To rule out any problem I have done the following, I have raised a virtual machine with Arch Linux completely clean and booting the network with systemd-networkd and, when querying the status of network-online.target I see the following:

[vagrant@archlinux ~]$ systemctl status network-online.target
○ network-online.target - Network is Online
     Loaded: loaded (/usr/lib/systemd/system/network-online.target; static)
     Active: inactive (dead)
       Docs: man:systemd.special(7)
             https://systemd.io/NETWORK_ONLINE

I find it surprising that it appears inactive, could it be some kind of bug?

I ask this mainly because this is affecting other things such as podman.

More info:

[vagrant@archlinux ~]$ systemctl status systemd-networkd
● systemd-networkd.service - Network Configuration
     Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-12-09 08:58:44 UTC; 1min 11s ago
 Invocation: a0b0da8a6f76411289d70f8303bf65dc
TriggeredBy: ● systemd-networkd.socket
       Docs: man:systemd-networkd.service(8)
             man:org.freedesktop.network1(5)
   Main PID: 283 (systemd-network)
     Status: "Processing requests..."
      Tasks: 1 (limit: 530)
   FD Store: 0 (limit: 512)
     Memory: 3.4M (peak: 3.7M)
        CPU: 22ms
     CGroup: /system.slice/systemd-networkd.service
             └─283 /usr/lib/systemd/systemd-networkd


[vagrant@archlinux ~]$ systemctl --version
systemd 256 (256.9-1-arch)
+PAM +AUDIT -SELINUX -APPARMOR -IMA +SMACK +SECCOMP +GCRYPT +GNUTLS +OPENSSL
+ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS
+LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK +XKBCOMMON
+UTMP -SYSVINIT +LIBARCHIVE

seth · 2024-12-09 13:06:04

I find it surprising that it appears inactive, could it be some kind of bug?

Should™ say "Active: active since …"
Seems podman specific, though?
https://github.com/containers/podman/is … 2527822425

amhairghin · 2024-12-09 13:18:12

seth wrote:

I find it surprising that it appears inactive, could it be some kind of bug?
Should™ say "Active: active since …"
Seems podman specific, though?
https://github.com/containers/podman/is … 2527822425

Yes, that issue was created by me. I'm trying to figure out who has the problem. From my point of view it seems that systemd has it, but it was to know other opinions.

The systemd is basically because even if you have user units that depend on network-online.target this is never enabled unless you start a system unit that depends on network-online.target (basically because making user units dependent on system units is useless).

In the end I think it all comes down to this bug: https://github.com/systemd/systemd/issues/3312

Last edited by amhairghin (2024-12-09 13:36:08)

seth · 2024-12-09 13:33:02

https://systemd.io/NETWORK_ONLINE/ wrote:

Network connectivity has been established: network-online.target
network-online.target is a target that actively waits until the network is “up”, where the definition of “up” is defined by the network management software. Usually it indicates a configured, routable IP address of some kind. Its primary purpose is to actively delay activation of services until the network has been set up.
It is an active target, meaning that it may be pulled in by the services requiring the network to be up, but is not pulled in by the network management service itself. By default all remote mounts defined in /etc/fstab make use of this service, in order to make sure the network is up before attempts to connect to a network share are made. Note that normally, if no service requires it and if no remote mount point is configured, this target is not pulled into the boot, thus avoiding any delays during boot should the network not be available. It is strongly recommended not to make use of this target too liberally: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up). Its primary purpose is network client software that cannot operate without network.
For more details about those targets, see the systemd.special(7) man page.

So basically you've a (user) service that depends on network-online, but no network service enabled but want the target to trigger some service?
Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

amhairghin · 2024-12-09 13:46:44

The problem here is clearly how systemd works.

Since you cannot make a user unit depend on a system unit (or what is the same, you cannot make a user unit depend on `network-online.target`) the people who make podman have come up with the idea of creating a dummy unit that simply checks if `network-online.target` is active. They do this very simply by running a “sh -c ‘until systemctl is-active network-online.target; do sleep 0.5; done’”. The problem is that by default network-online.target is not active unless you have a system unit that depends on this target. And of course, if you don't have such a unit then network-online.target is never activated and therefore the podman check fails.

Is the problem with podman? Yes and no. Yes because of how they do the checking and no because it is really systemd that is causing the problem by not being able to make dependencies between user units and system.

seth · 2024-12-09 14:11:40

It is strongly recommended not to make use of this target too liberally: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up). Its primary purpose is network client software that cannot operate without network.

There's more to this, starting by "the wait-online services by NM and systemd-networkd by default don't do what you'd expect"
I'd simply not use this target as indicator of anything itfp.
If you need to wait for some remote access to become available, loop some ping until success, see eg. https://bbs.archlinux.org/viewtopic.php?id=285097
This issue frequently comes up and for your entertainment: https://ewontfix.com/15/

amhairghin · 2024-12-09 14:35:35

seth wrote:

This issue frequently comes up and for your entertainment: https://ewontfix.com/15/

Completely agree. Well, at least this is written here in case someone else has this same problem and knows what options he/she has.

Arch Linux

#1 2024-12-09 09:27:15

Strange systemd behavior with network-online.target

#2 2024-12-09 13:06:04

Re: Strange systemd behavior with network-online.target

#3 2024-12-09 13:18:12

Re: Strange systemd behavior with network-online.target

#4 2024-12-09 13:33:02

Re: Strange systemd behavior with network-online.target

#5 2024-12-09 13:46:44

Re: Strange systemd behavior with network-online.target

#6 2024-12-09 14:11:40

Re: Strange systemd behavior with network-online.target

#7 2024-12-09 14:35:35

Re: Strange systemd behavior with network-online.target

Board footer