Linux 6.12 on AMD 5950X broke my VM

Ranguvar · 2024-12-05 21:17:57

6.11.9.zen1-1 still works well. 6.12.1.zen1-1 isn't working, and neither is 6.12.2.arch1-1, 6.13rc1-1, or 6.12.0.zen1-1.

The VM starts, but locks up as Windows 11 begins to load.
(It works fine even in the Windows Boot Manager, it locks up presumably loading the kernel, when the loading circle icon begins to spin.)

I'm using vfio to pass it my RTX 3090, an NVMe drive, and a PCIe USB controller. I already removed all other disks to narrow down the issue.
This worked fine for kernels up to 6.12.

I have isolated CPU cores for the VM. Those lock at 100% usage once the issue starts until the VM is off.
Interestingly, new processes also stall, and spawn only once I force the VM off.

No output in dmesg other than:

[Thu Dec  5 14:43:37 2024] hrtimer: interrupt took 1389 ns
[Thu Dec  5 14:43:44 2024] sched: DL replenish lagged too much

I'm using iommu=pt in kernel cmdline.

I tried kvm.enable_virt_at_load=0 as recommended for testing here but it did not change anything.
kvm_amd.nested=0 also did not help.

I can run a normal VM without issue.
Might need to try creating another custom VM using the isolated cores, and if that works, adding the PCIe devices.

Only other thing to do is git bisect. It's been a while since I've tried that.
Probably won't get the time for a few weeks yet. For now, reverting to 6.11.9.zen1-1.

EDIT: Tracking the issue at physicsBTW's VFIO gpu passthrough kernel regression thread.

Last edited by Ranguvar (2024-12-09 21:00:34)

gromit · 2024-12-05 22:31:33

Do you have any out of tree module loaded (nvidia, virtualbox etc.)?

If you want I can provide you prebuilt kernel images for the bisection, just ping me if you want that help once you have some time to test

Ranguvar · 2024-12-05 22:51:24

gromit wrote:

Do you have any out of tree module loaded (nvidia, virtualbox etc.)?
If you want I can provide you prebuilt kernel images for the bisection, just ping me if you want that help once you have some time to test

I do have ZFS git master, zenpower3, apfs-rw, hid-xpadneo, and nvidia-open-dkms 566.77. No others.

None of the NVIDIA modules are actually being permitted to load, I’ve checked.

Just in case, I tried removing them all with pacman and rebooted. Same issue though.

Thank you for the offer, I will ping you if I get to that point.

Also, something interesting. Most new processes on the Arch host start up fine during this bug. However, alacritty doesn’t create a window, kitty creates a broken window, and glxinfo stalls and can’t be Ctrl+C’d _after_ showing one line, “name of display: :0”.

Last edited by Ranguvar (2024-12-05 22:56:35)

Lone_Wolf · 2024-12-06 10:37:57

I'm using iommu=pt in kernel cmdline.

Typically that parameter is only needed in very specific cases and often causes issues if your system is not one of those cases.
Try without that parameter.

Arch Linux

#1 2024-12-05 21:17:57

Linux 6.12 on AMD 5950X broke my VM

#2 2024-12-05 22:31:33

Re: Linux 6.12 on AMD 5950X broke my VM

#3 2024-12-05 22:51:24

Re: Linux 6.12 on AMD 5950X broke my VM

#4 2024-12-06 10:37:57

Re: Linux 6.12 on AMD 5950X broke my VM

Board footer