[SOLVED] netctl-auto starts dhcpcd service implicitly

Kanehekili · 2024-08-07 21:50:02

I have a problem understanding a "rogue" behavior of my netctl-auto configuration:
Checking device A with

systemctl status netctl..

gives me:

     Loaded: loaded (/usr/lib/systemd/system/netctl-auto@.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-08-07 06:30:07 CEST; 16h ago
       Docs: man:netctl.special(7)
    Process: 299 ExecStart=/usr/bin/netctl-auto start wlan0 (code=exited, status=0/SUCCESS)
      Tasks: 7 (limit: 380)
        CPU: 4.838s
     CGroup: /system.slice/system-netctl\x2dauto.slice/netctl-auto@wlan0.service
             ├─357 wpa_supplicant -q -B -P /run/wpa_supplicant-wlan0.pid -i wlan0 -D nl80211,wext -c/run/netctl/wpa_supplic>
             ├─361 wpa_cli -i wlan0 -p /run/wpa_supplicant -B -a /usr/lib/netctl/auto.action
             ├─389 "dhcpcd: wlan0 [ip4]"
             ├─390 "dhcpcd: [privileged proxy] wlan0 [ip4]"
             ├─391 "dhcpcd: [control proxy] wlan0 [ip4]"

which leads to a failed activated dhcpcd service (since obviously that service already runs)

On other devices with the same configuration my dhcpcd service is running and netctl-auto does not seem to start dhcpcd itself:

     Loaded: loaded (/usr/lib/systemd/system/netctl-auto@.service; enabled; preset: disabled)
     Active: active (running) since Wed 2024-08-07 06:30:10 CEST; 17h ago
       Docs: man:netctl.special(7)
    Process: 298 ExecStart=/usr/bin/netctl-auto start wlan0 (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 380)
        CPU: 4.552s
     CGroup: /system.slice/system-netctl\x2dauto.slice/netctl-auto@wlan0.service
             ├─356 wpa_supplicant -q -B -P /run/wpa_supplicant-wlan0.pid -i wlan0 -D nl80211,wext -c/run/netctl/wpa_supplicant-wlan0.co>
             └─360 wpa_cli -i wlan0 -p /run/wpa_supplicant -B -a /usr/lib/netctl/auto.action

Comparing the services between device A and the other devices showed no difference.
The config files in /etc/netctl are identical , /etc/resolv.conf links to the resolve-stub.conf. Something must be different - but I'm out of ideas where to look.

Last edited by Kanehekili (2024-08-28 21:41:33)

-thc · 2024-08-08 07:06:22

Kanehekili wrote:

which leads to a failed activated dhcpcd service (since obviously that service already runs)

Why is that obvious? To whom?

1. Your code tags contain nothing obvious (beside the fact that netctl starts dhcpcd in one case)
2. When installed, dhcpcd is disabled by default
3. Neither the netctl nor the dhcpcd wiki page mention activating/starting dhcpcd

Kanehekili · 2024-08-08 12:28:07

1) Code tag 1 shows that dhcpcd is already running in a slice group. That seems to be the reason that the enabled systemd service dhcpcd@wlan0 fails to start.

2) dhcpcd was enabled explicitly, because netctl usually does not start dhcpcd.

So what makes netctl start the dhcpcd slice in 2 of my 7 devices?

shulamy · 2024-08-08 12:37:05

the wiki says that switching to dhclient may help

ezik

Head_on_a_Stick · 2024-08-08 12:58:05

shulamy wrote:

switching to dhclient may help

That project was abandoned quite some time ago. I think dhcpcd is a better option.

@OP: do you have ExcludeAuto set in any of the profiles? That determines whether a DHCP server is started or not.

Kanehekili · 2024-08-08 13:12:43

No ExcludeAuto. The devices are cloned, so all profiles are identical.

seth · 2024-08-08 13:21:43

dhcpcd was enabled explicitly, because netctl usually does not start dhcpcd.

Where did you get that idea?

Edit: post your netctl profile, obviously obfuscate any passwords.

Last edited by seth (2024-08-08 13:22:22)

Kanehekili · 2024-08-08 21:39:53

Hi Seth, thanks to chime in.
The dhcpcd was enabled explicitly by **me**, because until now I've not seen that netctl started its own dhcpcd.slice
For the record, the profile:

Interface=wlan0
Connection=wireless
Security=wpa
ESSID=XXXNet
IP=dhcp
Key=##########

systemd-cgls shows:

  ├─system-netctl\x2dauto.slice
  │ └─netctl-auto@wlan0.service
  │   ├─353 wpa_supplicant -q -B -P /run/wpa_supplicant-wlan0.pid -i wlan0 -D n>
  │   ├─357 wpa_cli -i wlan0 -p /run/wpa_supplicant -B -a /usr/lib/netctl/auto.>
  │   ├─382 dhcpcd: wlan0 [ip4]
  │   ├─383 dhcpcd: [privileged proxy] wlan0 [ip4]
  │   ├─384 dhcpcd: [control proxy] wlan0 [ip4]
  │   ├─386 dhcpcd: [BPF ARP] wlan0 x
  │   └─438 dhcpcd: [BOOTP proxy] x

On the devices that behave "normally":

Interface=wlan0
Connection=wireless
Security=wpa
ESSID=XxxNet
IP=dhcp
Key=######

but systemd-cgls looks different:

├─system-netctl\x2dauto.slice
  │ └─netctl-auto@wlan0.service
  │   ├─355 wpa_supplicant -q -B -P /run/wpa_supplicant-wlan0.pid -i wlan0 -D n>
  │   └─359 wpa_cli -i wlan0 -p /run/wpa_supplicant -B -a /usr/lib/netctl/auto.>

seth · 2024-08-08 22:11:20

This is on "different devices", though?

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Disable all dhcpcd services, netctl will start the daemonitself.
If not (check "ps aux", not systemd control groups) we'll have to look into why that is.

The difference between the systems might be a race condition where one NIC shows up way earlier than the other and spawns its dhcpcd process before the service starts while on the other system, the global dhcpcd service starts before the NIC shows up.

Wrt "wlan0", in case you've iwd installed, make sure that's not enabled (in doubt post the output of the previous find command for the systems) as it'll not only conflict w/ wpa_supplicant but maybe even run its own dhcp request.

Kanehekili · 2024-08-08 23:35:28

Yes, this is on "different devices". I've created an iso image, changed hostname, rm machineId, ssh stuff for each device and propagated it to 7 different devices (raspi 3a). So all should have the same configuration. The goal of my quest is to understand the different behavior of netctl. I could just disable the dhcpcd service on those devices, but I want to understand why - that is the beauty of linux - usually everything can be explained.
iwd is not installed. The "find" command produces the same services on both devices (those with dhcpcd slices and those without):

ctrl-alt-del.target                      | system
dbus-org.freedesktop.home1.service       | system
dbus-org.freedesktop.resolve1.service    | system
dbus-org.freedesktop.timesync1.service   | system
dhcpcd@wlan0.service                     | multi-user.target.wants
getty@tty1.service                       | getty.target.wants
machines.target                          | multi-user.target.wants
netctl-auto@wlan0.service                | sys-subsystem-net-devices-wlan0.device.wants
nightly.timer                            | timers.target.wants
p11-kit-server.socket                    | sockets.target.wants
remote-cryptsetup.target                 | multi-user.target.wants
remote-fs.target                         | multi-user.target.wants
sshd.service                             | multi-user.target.wants
systemd-boot-update.service              | sysinit.target.wants
systemd-homed-activate.service           | systemd-homed.service.wants
systemd-homed.service                    | multi-user.target.wants
systemd-journald-audit.socket            | sockets.target.wants
systemd-journald-audit.socket            | systemd-journald.service.wants
systemd-pstore.service                   | sysinit.target.wants
systemd-resolved.service                 | sysinit.target.wants
systemd-timesyncd.service                | sysinit.target.wants
systemd-userdbd.socket                   | sockets.target.wants
tsvaccess.service                        | multi-user.target.wants

Then I compared "/etc/nsswitch.conf". (Which I altered)
hosts: files myhostname dns resolve mymachines

Same on all devices, so that could not be the culprit.
Since the devices are productive and Wi-Fi only, I'll check if that is a NIC problem on a locally accessible device by simply disabling dhcpcd.

Last edited by Kanehekili (2024-08-08 23:51:08)

seth · 2024-08-09 07:50:03

but I want to understand why - that is the beauty of linux - usually everything can be explained

Looks at the system journal and the process list, It's most likely going to be a race condition.

I could will just disable the dhcpcd service on those devices

Ftfy
Don't misconfigure the system and expect sane or deterministic outcome. Concurrent network services are a primary way to cause weird behavior, intermittent failure and random network issues.

Kanehekili · 2024-08-17 22:05:54

I didn't need to "misconfigure" anything. The single device suddenly behaves like the other devices. No updates nor any other changes were executed. That means, netctl currently doesn't have those observed slices and the dhcpcd@wlan0 service runs again. Seems to be a non-solvable riddle. To make it this clear: each device reboots every night via a timer. I'm not setting it solved, since I don't understand why netctl may be able to activate dhcpcd on rare occasions.

Last edited by Kanehekili (2024-08-17 22:07:20)

seth · 2024-08-17 22:13:40

Do. Not. Enable. Any. Dhcpcd. Service. when using netctl.

netctl *will* by default start dhpcd. if an explicit dhcpcd service is enabled, those will collide and the behavior become completely no-determistic.
Concurrent network services are the most popular way to screw ones network, there's really nothing special about that.

Kanehekili · 2024-08-28 21:36:44

OK, tested it and it works. netctl implicitly starts the dhcpcd service indeed - a big thank you for solving the problem since is is not mentioned anywhere. Marking this thread as solved

ewaller · 2024-08-29 21:37:59

Kanehekili wrote:

netctl implicitly starts the dhcpcd service indeed

Not trying to be pedantic here, but it does not. netctl starts the dhcpcd daemon. The dhcpcd service, when started, also starts an instance of the dhcpcd daemon. The problem is 'Bad Things'™ happen when there are multiple dhcpcd daemons running.

Kanehekili · 2024-08-30 21:13:52

This makes sense. Explains the slices if seen on my "rogue machine". Turning off the dhcpcd.service does not interfere with starting the daemon anymore since boot time decreased on all machines (where the dhcpcd.service was always on top of systemd-analyze blame). I want to thank everyone for clearing this up for me.

Arch Linux

#1 2024-08-07 21:50:02

[SOLVED] netctl-auto starts dhcpcd service implicitly

#2 2024-08-08 07:06:22

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#3 2024-08-08 12:28:07

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#4 2024-08-08 12:37:05

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#5 2024-08-08 12:58:05

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#6 2024-08-08 13:12:43

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#7 2024-08-08 13:21:43

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#8 2024-08-08 21:39:53

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#9 2024-08-08 22:11:20

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#10 2024-08-08 23:35:28

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#11 2024-08-09 07:50:03

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#12 2024-08-17 22:05:54

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#13 2024-08-17 22:13:40

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#14 2024-08-28 21:36:44

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#15 2024-08-29 21:37:59

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

#16 2024-08-30 21:13:52

Re: [SOLVED] netctl-auto starts dhcpcd service implicitly

Board footer