Arch Linux

neo-fetch

Member

Registered: 2021-12-06

Posts: 15

[SOLVED] Docker doesn't run with NVIDIA GPUs

I'm trying to run Nvidia Modulus (Previously known as NVIDIA simnet). Using the instructions given here.

Unfortunately, I get the following error:

Failed to initialize NVML: Unknown Error

I then search and find the solution to the exact issue online: https://bbs.archlinux.org/viewtopic.php?id=266915

On executing Method 1, I now get the new error:

docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #1:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: container error: cgroup subsystem devices not found: unknown.
ERRO[0000] error waiting for container: context canceled

my /proc/cmdline

root=PARTUUID=af2931ff-5938-49a0-b9c1-b335ccc22211 rw add_efi_memmap initrd=boot\initramfs-linux.img systemd.unified_cgroup_hierarchy=false

(the file above resets after reboot)

my /etc/nvidia-container-runtime/config.toml

isable-require = false
#swarm-resource = "DOCKER_RESOURCE_GPU"
#accept-nvidia-visible-devices-envvar-when-unprivileged = true
#accept-nvidia-visible-devices-as-volume-mounts = false

[nvidia-container-cli]
#root = "/run/nvidia/driver"
#path = "/usr/bin/nvidia-container-cli"
environment = []
#debug = "/var/log/nvidia-container-toolkit.log"
#ldcache = "/etc/ld.so.cache"
load-kmods = true
no-cgroups = false
#user = "root:video"
ldconfig = "@/sbin/ldconfig"

[nvidia-container-runtime]
#debug = "/var/log/nvidia-container-runtime.log"

Apologies in advance if there is something obvious I haven't done. I am pretty new to this after all, but please let me know what I can do.
Any recommendations and/or constructive criticism is welcome.

# EDIT_1: I have already followed this procedure before doing anything stupid: https://wiki.archlinux.org/title/docker … commended)

Last edited by neo-fetch (2021-12-12 14:03:17)

neo-fetch

Member

Registered: 2021-12-06

Posts: 15

Re: [SOLVED] Docker doesn't run with NVIDIA GPUs

Did something that works for now:

I clean built every package (libnvidia, container runtime, container toolkit, docker). Then changed the kernel paramaters. Since hijacking the proc/cmdline didn't exactly work, I looked for the alternate ways specified in the kernel parameters wiki.

Since I use refind, I added the systemd param using the refind menu. Booted using the newly added params, changed the config.toml, and started the docker service.

Works now, but it's really annoying to do this every time I boot. I'd like to say a lot of things to Nvidia, but since I hold this forum sacred, I shall hold my tongue and live to fight another day. Godspeed whoever is reading this.