Arch Linux

Inspector Gadget

Member

Registered: 2023-04-09

Posts: 3

[SOLVED] Not able to spawn virtual functions (VF/SR-IOV) via systemd..

Hi there,

I am having an issue with a self written service. I have a network adapter able to spawn virtual functions (SR-IOV). It uses the intel ixgbe driver. There are two methods to spawn the VFs:
1. User a kernel parameter: ixgbe.max_vfs=N
2. echo N > /sys/class/net/NETWORKDEVICE/device/sriov_numvfs

I want to schedule a few of my services to start after the VFs have been spawned so I'd like to use the second method to spawn VFs and use this in a service.

Now I wrote a service for this task:

[Unit]
Description=Spawn virtual functions on interface %i.
Requires=network.target multi-user.target

[Service]
Type=oneshot
RemainAfterExit=true
# Use SR-IOV capabilities and spawn VF's
ExecStart=/usr/bin/bash -c "/usr/bin/echo 63 > /sys/class/net/%i/device/sriov_numvfs && sleep 30"

[Install]
WantedBy=custom-startup.target

However, during boot, the start of this service fails. After boot I investigated and it seems the service is unable to access "/sys/class/net/%i/device/sriov_numvfs" during boot, since I outputs a message that the file can't be found. After I login, after the boot procedure, I can start the very same service without issues. Does anyone know what dependencies "Requires=" or "After=" argument I need to set for the service to be able to find the file?

Thank you.

Last edited by Inspector Gadget (2023-04-09 21:34:23)

Trilby

Inspector Parrot

Registered: 2011-11-29

Posts: 29,550

Website

Re: [SOLVED] Not able to spawn virtual functions (VF/SR-IOV) via systemd..

Just do it properly with systemd tmpfiles.

EDIT: but I also don't see why the kernel parameters wouldn't work as other services most certainly will only start after the kernel is running.

Last edited by Trilby (2023-04-09 19:10:46)

---

"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Inspector Gadget

Member

Registered: 2023-04-09

Posts: 3

Re: [SOLVED] Not able to spawn virtual functions (VF/SR-IOV) via systemd..

Just do it properly with systemd tmpfiles.

This is a creative use of tmp-files, but does not really help me as far as I understand. I need subsequent services to fail when this operation is not successful. A tmp-file will be created regardless of the fact that the VFs actually spawn or not. While when "/sys/class/net/%i/device/sriov_numvfs" is present and I echo to it I can be as certain as I can be that they will be spawned. For example the tmp-file would be created even when the network adapter is not present at all. I am also unsure how I am supposed to time subsequent services, what let I depend them on to know when the VFs have been instructed to spawn? I would like to prefer to find out at which point the path to the files becomes available, but maybe I am not understanding how tmp-files work in this context.

EDIT: but I also don't see why the kernel parameters wouldn't work as other services most certainly will only start after the kernel is running.

Because since an actual hardware device is involved their creation is not instant. The kernel parameter seems to be particularly slow. When using the kernel parameter the boot process can reach the graphical.target and even after having logged in they are sometimes not yet present.

Inspector Gadget

Member

Registered: 2023-04-09

Posts: 3

Re: [SOLVED] Not able to spawn virtual functions (VF/SR-IOV) via systemd..

I figured it out!

I can recommend the tool systemd-analyze to debug these kinds of problems. The following command was in particular helpful since it provides a visualization of the boot process.

systemd-analyze plot > /home/USER/start.svg

Turns out the initialization of devices like network cards are represented by systemd *.device files. In the case of a network adapter available under enp8s0f0 this would result in the following file be part of the boot process.

sys-subsystem-net-devices-enp8s0f0.device

So the service file needs to be adjusted. Here lies my second mistake. The usage of "Requires=" in my service file is wrong, correct would be the following here.

[Unit]
Description=Spawn virtual functions on interface %i.
After=sys-subsystem-net-devices-%i.device
Wants=sys-subsystem-net-devices-%i.device

[Service]
Type=oneshot
RemainAfterExit=true
# Use SR-IOV capabilities and spawn VF's
ExecStart=/usr/bin/bash -c "/usr/bin/echo 63 > /sys/class/net/%i/device/sriov_numvfs && sleep 30"

[Install]
WantedBy=custom-startup.target