Stuck at triggering uevents after 6.6.13 update

johnramsden · 2024-01-26 18:26:43

After the 6.6.13 update (linux-lts) my computer no longer boots and gets stuck at "triggering events". I revert back to 6.1.71-1-lts and everything works just fine.

I think it might be something to do with my GPU (AMD ATI Radeon RX 7900 XT/7900 XTX), since if I enable nomodeset I can boot successfully.

I have tried enabling

loglevel=7 debug

, but there is no output after the triggering events, it is just stuck there forever.

Even before the upgrade, that was the stage of boot that took the longest (about 20 seconds once I hit triggering) but after the upgrade it just stays there forever (i've waited an hour).

loqs · 2024-01-26 19:17:24

Have you tried any of the series between 6.1 and 6.6 to see which introduced the issue? Please post the full journal from the boot with nomodeset and also anything recorded in the journal from the boot with the issue.

johnramsden · 2024-01-29 00:59:17

I was on linux-lts so there was no in between. I also attempted switching to the stable kernel and the same issue occurs.

I read some related posts:

* https://www.reddit.com/r/archlinux/comm … g_uevents/
* https://bbs.archlinux.org/viewtopic.php?id=282114
* https://bbs.archlinux.org/viewtopic.php?id=151259

But I couldn't find anything that helped.

I also followed the AMD page https://wiki.archlinux.org/title/AMDGPU carefully, and I attempted adding modules to my mkinitcpio.conf MODULES=(amdgpu radeon).

Nothing really helped so far. And I'm kind of lost. I attached some logs for the nomodeset boot, but there was nothing there that stood out to me, and it just seemed to work fine with nomodeset.

* journal: http://sprunge.us/irxB8m
* dmesg: http://sprunge.us/2vkj1R

loqs · 2024-01-29 20:01:50

Is the system not using the kms hook? Which I would have expected to pull amdgpu into the initdrd. Does booting into rescue.target or emergency.target still trigger the issue? You can obtain old version of the linux package from the ALA to test the 6.2 to 6.5 series.

johnramsden · 2024-01-29 21:33:28

It is:

HOOKS=(base udev autodetect modconf kms keyboard zfs keymap consolefont block filesystems)

I was just checking if adding it to the modules would help since the wiki recommended trying that.

I will give the other targets a try.

johnramsden · 2024-02-04 20:08:33

I just came across this thread, I have a feeling this might be the cause of my issue https://bbs.archlinux.org/viewtopic.php?id=290707

I have attempted fallback. I'm not able to revert to old versions since I'm using ZFS, and I don't have access to the older builds. If necessary I will rebuild zfs-linux from scratch when I have time.

loqs · 2024-02-05 09:05:44

johnramsden wrote:

I just came across this thread, I have a feeling this might be the cause of my issue https://bbs.archlinux.org/viewtopic.php?id=290707

That issue was introduced in 6.7.

johnramsden wrote:

I have attempted fallback. I'm not able to revert to old versions since I'm using ZFS, and I don't have access to the older builds.

zfs-dkms.

johnramsden · 2024-02-05 19:28:34

There are reports of it happening in 6.6 (which is where I noticed it) in that thread. Yes when I said compile it from scratch I meant dkms

loqs · 2024-02-05 21:15:16

$ git revert 5f38ac54e60562323ea4abb1bfb37d043ee23357
Auto-merging drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
Auto-merging drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
Auto-merging drivers/gpu/drm/amd/pm/swsmu/inc/amdgpu_smu.h
Auto-merging drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c
Auto-merging drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_0_ppt.c
Auto-merging drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0_7_ppt.c
HEAD detached at v6.6.15
nothing to commit, working tree clean

The causal commit that thread identified is not part of 6.6 which is why attempting to revert it produces an empty commit, which is also why the upstream fix for that issue will not be applied to 6.6 by upstream. The symptoms may be the same but the cause can not be.

johnramsden · 2024-02-05 21:32:35

Thanks.

Basically what I experienced was going from linux-lts 6.1.71 -> 6.6.13 caused my system to hang with no error message indefinitely at triggering uevents.

I will give compiling from dkms a shot on the weekend.

johnramsden · 2024-02-05 21:33:21

Also relevant. It does not appear to be my userland. I attempted keeping my userland the same and only upgrading the kernel, and that fails.

seth · 2024-02-05 22:31:07

Jan 28 16:28:23 fenix kernel: pci 0000:03:00.1: [1002:ab30] type 00 class 0x040300
Jan 28 16:28:23 fenix kernel: pci 0000:03:00.1: reg 0x10: [mem 0x83420000-0x83423fff]
Jan 28 16:28:23 fenix kernel: pci 0000:03:00.1: PME# supported from D1 D2 D3hot D3cold
Jan 28 16:28:23 fenix kernel: pci 0000:03:00.0: vgaarb: setting as boot VGA device
Jan 28 16:28:23 fenix kernel: pci 0000:03:00.0: vgaarb: bridge control possible
…
Jan 28 16:28:26 fenix (udev-worker)[1661]: Failed to insert module 'amdgpu': Invalid argument

modinfo amdgpu | curl -F 'file=@-' 0x0.st
modprobe -c | grep amdgpu | curl -F 'file=@-' 0x0.st

johnramsden · 2024-02-05 22:36:48

Would that be because nomodeset is on?

seth · 2024-02-05 22:41:10

maybe, but you can boot w/ nomodeset and get "stuck" w/o when udev is triggering and it's triggering here and complaining about the amdgpu module - so we'll take a look at that unless you can somehow show that this is not a problem when booting w/o nomodeset (eg. rebooting via https://wiki.archlinux.org/title/Keyboa … el_(SysRq) and REISUB)

johnramsden · 2024-02-05 22:50:14

No errors are mentioned at all when I do not use nomodeset. When I use nomodeset it boots just fine, but that message is displayed (I assume because you cannot use AMDGPU when you have nomodeset enabled https://wiki.archlinux.org/title/AMDGPU#Loading).

I don't think I understand how to use SysRq. Can I use it to query information about what is going on when I get stuck?

I should start by setting sysrq_always_enabled=1 to my kernel, and then when I get stuck I should do what?

seth · 2024-02-05 22:54:16

You press Alt+SysRq+r,e,i,s,u,b (wait some seconds between each key) - this will ideally sync and unmount all filesystem and reboot so you can

sudo journalctl -b -1

look at the journal of the bad boot.

johnramsden · 2024-02-06 00:17:28

Oh, nice. Thanks ill try that.

Succulent of your garden · 2024-02-29 13:40:19

Hi everyone!

I have been very busy to post here until now, I'm also having this issue using the hardened version of the kernel, so I guess the error appears no matter the linux kernel variation. Using nomodeset in my Xorg machines everything works fine, but not for my Wayland machine that I like to use to see how Wayland progress is going :C . I came to the conclusion that with downgrading the kernel up to version 6.6.8 everything works fine, in wayland and Xorg. After that it seems that the system breaks with the udev trigger issue, and using nomodeset with my Xorg machines everything works fine.

Last 2 weeks I tested with a new kernel update in my wayland machine and the error persists, so I plug a arch usb stick and do chroot to downgrade the kernel, everything works.. So I guess is still an issue until now.

Every help or information that help me to take more care of my Wayland machine I will appreciate so much, now that I have the time to fix this.

loqs · 2024-02-29 14:33:24

@Succulent of you garden on your system 6.6.8.hardened1-1, 6.6.9.arch1-1 do not produce the issue and while 6.6.9.arch1-1, 6.6.10.hardened1-1 while do produce the issue?

Last edited by loqs (2024-02-29 14:34:13)

b.ohnsorg · 2024-02-29 14:48:56

https://bugs.gentoo.org/925861 – ran into the same issue but with radeon driver + firmware (TAHITI GPU does not work with AMDGPU)

Succulent of your garden · 2024-03-07 22:27:16

loqs wrote:

@Succulent of you garden on your system 6.6.8.hardened1-1, 6.6.9.arch1-1 do not produce the issue and while 6.6.9.arch1-1, 6.6.10.hardened1-1 while do produce the issue?

Yes exactly. That's the issue with the Wayland base machine. Every next release of the Linux kernel after 6.6.8 will produce the issue, or at least two weeks ago, I never tried after two weeks. To busy to check and to the chroot with the usb stick to recover the machine just in case.

Last edited by Succulent of your garden (2024-03-07 22:52:33)

Succulent of your garden · 2024-03-07 22:52:02

b.ohnsorg wrote:

https://bugs.gentoo.org/925861 – ran into the same issue but with radeon driver + firmware (TAHITI GPU does not work with AMDGPU)

Thanks for the Gentoo thread. I will read the links in the thread tomorrow. But I checked all the posts in the gentoo forum. I'm not sure if my GPU is a TAHITI. But surely is Radeon. It seems this laptop came with an integrated gpu in the cpu, because nvtop doesn't show any dedicated graphics cards. Maybe it's a issue with only integrated graphics cards from amd. So It will be nice to know if someone with a dedicated graphics cards and an amd cpu could launch a tty with wayland fine.

This bug seems to be very special, I don't know if could be a wayland issue, a wlroots issue or a Linux Kernel issue. So any troubleshooting help will be appreciated, at least until I know if there is really a Kernel issue, I don't want to go and post to the Linux git just because after 6.6.8 my machine broke and getting back solves the issue. Maybe the bug comes with Wayland, I don't know. I'm not into wayland like a developer to know exactly where this issue begins.

Thanks for the info and have nice day!

Last edited by Succulent of your garden (2024-03-07 22:53:32)

seth · 2024-03-08 00:25:03

https://gitlab.freedesktop.org/drm/amd/ … te_2316675
Upstream seems to suspect https://github.com/torvalds/linux/commit/27655b9bb9f0 which would cover both drivers but exists since 6.5

Rolling w/ the original 6.1/6.5 split I'll also throw in "initcall_blacklist=simpledrm_platform_driver_init", https://wiki.archlinux.org/title/Kernel_parameters

As for radeon GCN chips specifically, see https://wiki.archlinux.org/title/AMDGPU … K)_support
(There were quite some reports itr)

If you don't know what GPU you have you can just try (amdgpu will not pick up any older cards) or correlate "lspci" and https://www.x.org/wiki/RadeonFeature/#d … etingnames

b0ss_ · 2024-03-08 00:42:37

Hey.

I've been having a similiar issue, but my system hangs at "Loading initial ramdisk". The main difference I think is that for me, the lts kernel doesn't make a difference. If I add "acpi=off" it boots. Nevertheless, neither my APU nor my GPU load the AMDGPU driver.

I'm using an RX580 with a Ryzen 2200G.

Hope the kernel parameter does it for you... and I hope it lets you actually load the module

seth · 2024-03-08 08:13:17

That's a different problem and disabling acpi will prevent a lot of things from working.
It's a debugging step, not a solution.

See whether there's a firmware update (UEFI/BIOS) availble for your system and if you're dual-booting, see the 3rd link below. Mandatory.
Disable it (it's NOT the BIOS setting!) and reboot windows and linux twice for voodo reasons.

Arch Linux

#1 2024-01-26 18:26:43

Stuck at triggering uevents after 6.6.13 update

#2 2024-01-26 19:17:24

Re: Stuck at triggering uevents after 6.6.13 update

#3 2024-01-29 00:59:17

Re: Stuck at triggering uevents after 6.6.13 update

#4 2024-01-29 20:01:50

Re: Stuck at triggering uevents after 6.6.13 update

#5 2024-01-29 21:33:28

Re: Stuck at triggering uevents after 6.6.13 update

#6 2024-02-04 20:08:33

Re: Stuck at triggering uevents after 6.6.13 update

#7 2024-02-05 09:05:44

Re: Stuck at triggering uevents after 6.6.13 update

#8 2024-02-05 19:28:34

Re: Stuck at triggering uevents after 6.6.13 update

#9 2024-02-05 21:15:16

Re: Stuck at triggering uevents after 6.6.13 update

#10 2024-02-05 21:32:35

Re: Stuck at triggering uevents after 6.6.13 update

#11 2024-02-05 21:33:21

Re: Stuck at triggering uevents after 6.6.13 update

#12 2024-02-05 22:31:07

Re: Stuck at triggering uevents after 6.6.13 update

#13 2024-02-05 22:36:48

Re: Stuck at triggering uevents after 6.6.13 update

#14 2024-02-05 22:41:10

Re: Stuck at triggering uevents after 6.6.13 update

#15 2024-02-05 22:50:14

Re: Stuck at triggering uevents after 6.6.13 update

#16 2024-02-05 22:54:16

Re: Stuck at triggering uevents after 6.6.13 update

#17 2024-02-06 00:17:28

Re: Stuck at triggering uevents after 6.6.13 update

#18 2024-02-29 13:40:19

Re: Stuck at triggering uevents after 6.6.13 update

#19 2024-02-29 14:33:24

Re: Stuck at triggering uevents after 6.6.13 update

#20 2024-02-29 14:48:56

Re: Stuck at triggering uevents after 6.6.13 update

#21 2024-03-07 22:27:16

Re: Stuck at triggering uevents after 6.6.13 update

#22 2024-03-07 22:52:02

Re: Stuck at triggering uevents after 6.6.13 update

#23 2024-03-08 00:25:03

Re: Stuck at triggering uevents after 6.6.13 update

#24 2024-03-08 00:42:37

Re: Stuck at triggering uevents after 6.6.13 update

#25 2024-03-08 08:13:17

Re: Stuck at triggering uevents after 6.6.13 update

Board footer