AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

frumble · 2023-03-09 17:31:46

Hello, a few week ago, I encountered system freezes when returning from standby mode: Black screen with just my mouse, frozen. AMD Radeon RX 5700 XT, KDE Plasma 5.26/27 on X.org, starting with Linux 6.1.4. Since it doesn’t happen every time, I had to test each kernel version since for several days. These are my findings:

6.1.3: Stable
6.1.4: Freeze after standby
6.1.5: Seemingly stable again, but now my monitor LG 27GP950-B lost several refresh rate options… But this is a secondary problem for now… From 144, 120, 96, 72, 60, 50, 48 Hz down to just 144, 95, 60 Hz, still through Linux 6.2.x.
6.1.6: Again freezes
6.1.7: Freezes
6.1.8: Stable again
6.1.9: Still stable
6.1.10: Seemingly not the old freeze after standby but now strange freezes when I initiate standby; another time freeze after standby with the lockscreen visible (with lockscreen UI etc. visible).
6.1.11: Again the older freeze with black screen and mouse after standby
6.1.12: Same Freeze
6.2.0: Same Freeze

This is exhausting, I shouldn’t need to debug AMD’s driver but here we are. I need to bisect the kernels. First question: Where should I start? 6.1.4 or 6.1.11? Or 6.1.10?
Second question: Which PKGBUILD should I use? My last kernel bisects needed a customized file and I’m still a novice in this area.

Last edited by frumble (2023-06-09 23:53:52)

mearkat7 · 2023-03-09 21:24:39

It sounds like a similar issue to what i'm getting (thread here: https://bbs.archlinux.org/viewtopic.php?id=284076). I've gone back a few kernels but not as far back as you, on your findings i've currently switched to 6.1.9 to see if stability improves for me.

I'm new to all of this but let me know if I can help in any way.

Last edited by mearkat7 (2023-03-09 21:26:58)

loqs · 2023-03-09 21:36:09

I would suggest bisecting 6.1 to 6.2 as that would be where the issue was introduced upstream or 6.1.3 to 6.1.4 as that is the first stable with the issue.
You can find some prebuilt bisection kernels for 6.1 to 6.2 in https://bugs.archlinux.org/task/77632#comment215509

seth · 2023-03-10 09:21:35

Do you get similar stack traces as https://bbs.archlinux.org/viewtopic.php … 8#p2089168 ?
@mearkat7, do you also encounter those freezes (only) after S3 cycles?

mearkat7 · 2023-03-12 20:46:15

seth wrote:

Do you get similar stack traces as https://bbs.archlinux.org/viewtopic.php … 8#p2089168 ?
@mearkat7, do you also encounter those freezes (only) after S3 cycles?

I have tried both 6.1.3, 6.1.8 and 6.1.9 and still get freezes on all of the above so it must be a different issue unfortunately. Forgive my ignorance but what is an S3 cycle?

seth · 2023-03-12 21:23:49

https://en.wikipedia.org/wiki/ACPI#Global_states

mearkat7 · 2023-03-14 00:51:27

seth wrote:

https://en.wikipedia.org/wiki/ACPI#Global_states

Not from suspend for me no.

The freezes will occur at any random time from boot but from what i've noticed my PC will only restart and fully crash when my display has gone to sleep.

seth · 2023-03-14 10:23:19

That would be dpms, https://bbs.archlinux.org/viewtopic.php … 7#p2089127

Last edited by seth (2023-03-14 10:25:15)

mearkat7 · 2023-03-15 03:38:59

seth wrote:

That would be dpms, https://bbs.archlinux.org/viewtopic.php … 7#p2089127

Sadly i've tried turning dpms off and the issue still occurs.

seth · 2023-03-15 07:35:21

So

i've noticed my PC will only restart and fully crash when my display has gone to sleep.

does no longer hold?
Does something else (screensaver) trigger DPMS?
Is it maybe not DPMS but some fancy screensaver animation that just co-incided w/ "display has gone to sleep"?

frumble · 2023-04-13 00:23:37

Sorry for my unanswering but I needed a stable kernel for some weeks and was frustrated anyway.

@seth: I don’t get any straces, since my system goes into a total freeze.

@loqs: I would very thankful if you could build me once again bisection kernels. Let’s start from 6.1.3, going to 6.1.4.

I have tried these kernels from your link:

linux-6.1.r7411.g1ca06f1c1ace-1-x86_64.pkg.tar.zst - PulseAudio crash, unsuitable to daily-drive

linux-6.1.r3656.g8715c6d3100f-1-x86_64.pkg.tar.zst - same

linux-6.1rc6.r1434.g66efff515a65-1-x86_64.pkg.tar.zst - no PulseAudio crash, but flickering in Firefox, unsuitable to daily-drive

linux-6.1.r6266.gc5589c436d46-1-x86_64.pkg.tar.zst - PulseAudio crash + flickering in Firefox, unsuitable to daily-drive

linux-6.1.r6831.g86a0b4255e84-1-x86_64.pkg.tar.zst - same

linux-6.1.r204.g0e2c9884cbba-1-x86_64.pkg.tar.zst - PA runs, no graphic glitches

linux-6.1.r7318.g4eb77fa102eb-1-x86_64.pkg.tar.zst - PA problem again + glitches

linux-6.1.r7370.ga70210f41566-1-x86_64.pkg.tar.zst - same

linux-6.1.r7019.g71946a25f357-1-x86_64.pkg.tar.zst - same

linux-6.1rc8.r45.ge8dfdf3162eb-1-x86_64.pkg.tar.zst - PA runs, no glitches, no immediate freezes after standby

linux-6.1.r7250.g717e6eb49bdd-1-x86_64.pkg.tar.zst - PA problem again + glitches

linux-6.1.r7298.g8b9ed79c2d58-1-x86_64.pkg.tar.zst - PA problem, no glitches, no immediate freezes after standby

linux-6.1.r204.g0e2c9884cbba-1-x86_64.pkg.tar.zst - PA problems, no glitches, but crash like described in my first post! But very rarely, maybe once a week. Not sure if this is the same bug or another bug with similar outcome?

Let’s just start with 6.1.3, I think that would be wiser than trying to understand what was going on with those 6.1.0 (?) development versions.

loqs · 2023-04-13 01:19:28

$ git bisect start
status: waiting for both good and bad commits
$ git bisect bad v6.1.4
status: waiting for good commit(s), bad commit known
$ git bisect good v6.1.3
Bisecting: 103 revisions left to test after this (roughly 7 steps)
[e9f7a3bbaa5c0bc1c9dab5bf3ea5f2802034e50b] cifs: fix confusing debug message

https://drive.google.com/file/d/1U_K3uf … share_link linux-6.1.3.r103.ge9f7a3bbaa5c-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1ZUDL6b … share_link linux-headers-6.1.3.r103.ge9f7a3bbaa5c-1-x86_64.pkg.tar.zst

frumble · 2023-04-13 16:22:10

Thank you! I will test this kernel now. But this whole process could take several weeks because the freezes after standby happened only for about every third day.

frumble · 2023-04-17 22:03:03

This kernel is good.

loqs · 2023-04-18 04:02:44

$ git bisect good
Bisecting: 51 revisions left to test after this (roughly 6 steps)
[3650c063f22d03795026bd6f3d473e5bbdabb442] drm/mgag200: Fix PLL setup for G200_SE_A rev >=4

https://drive.google.com/file/d/1iWquSN … share_link linux-6.1.3.r155.g3650c063f22d-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1YxuOSr … share_link linux-headers-6.1.3.r155.g3650c063f22d-1-x86_64.pkg.tar.zst

frumble · 2023-04-18 10:37:28

Haha, this went bad fast: This kernel is bad.

loqs · 2023-04-18 14:07:39

$ git bisect bad
Bisecting: 25 revisions left to test after this (roughly 5 steps)
[abbb887da77408892c0c8fb4cbbc2a5bb03b140e] riscv: Fixup compile error with !MMU

https://drive.google.com/file/d/1g2Zdvc … share_link linux-6.1.3.r129.gabbb887da774-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1D4YhR6 … share_link linux-headers-6.1.3.r129.gabbb887da774-1-x86_64.pkg.tar.zst

frumble · 2023-04-28 15:29:20

This kernel is good.

loqs · 2023-04-28 17:46:27

$ git bisect good
Bisecting: 12 revisions left to test after this (roughly 4 steps)
[17183187dc862a828f8e54380d0596eafa0b09f8] hugetlb: really allocate vma lock for all sharable vmas

https://drive.google.com/file/d/1cEyX1V … share_link linux-6.1.3.r142.g17183187dc86-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1nNwNLK … share_link linux-headers-6.1.3.r142.g17183187dc86-1-x86_64.pkg.tar.zst

MadCat_X · 2023-04-30 08:53:16

My understanding of the current situation is that recent work on the amdgpu driver made a mess of things at some point around 6.0 and 6.1. I've been hit by this too. Bisection is made complicated because some patches have been backported to late version of 6.1 and 6.2. This means that, say, 6.1.14 might be less broken than 6.2.0 because of backported patches.

My suggestion would be to report issues here (https://gitlab.freedesktop.org/drm/amd/-/issues) and test whatever patches the devs recommend.

frumble · 2023-05-05 13:14:02

This kernel is good.

frumble · 2023-05-09 11:00:44

Tried the last two days driving 6.3.1-arch1-1, still the same occurring issue with black screen and frozen mouse.

loqs · 2023-05-09 19:12:11

Was unavailable the past few days, sorry for the delay.

The toolchain update to gcc 13 breaks the build so I pinned the mirror to https://archive.archlinux.org/repos/202 … o/os/$arch

$ git bisect good
Bisecting: 6 revisions left to test after this (roughly 3 steps)
[553bc5890ed96a8d006224c3a4673c47fee0d12a] parisc: Fix locking in pdc_iodc_print() firmware call

https://drive.google.com/file/d/1X7DCll … share_link linux-6.1.3.r148.g553bc5890ed9-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1NwcfI4 … share_link linux-headers-6.1.3.r148.g553bc5890ed9-1-x86_64.pkg.tar.zst

frumble · 2023-05-12 16:21:55

No problem at all, life comes first. Gave me time to check the 6.3.x kernel.

Your last kernel seems to be good.

loqs · 2023-05-12 16:44:04

Will build a new good bisection kernel, to me the most likely cause based on the commits left is https://git.kernel.org/pub/scm/linux/ke … ec150bcc2a
Edit:

$ git bisect good
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[4def68cc15f37287a6b3bb8ccaaaba2aee6c5185] parisc: Drop PMD_SHIFT from calculation in pgtable.h

https://drive.google.com/file/d/1mtRxXs … share_link linux-6.1.3.r151.g4def68cc15f3-1-x86_64.pkg.tar.zst
https://drive.google.com/file/d/1BwfRkc … share_link linux-headers-6.1.3.r151.g4def68cc15f3-1-x86_64.pkg.tar.zst

Last edited by loqs (2023-05-12 17:06:25)

Arch Linux

#1 2023-03-09 17:31:46

AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#2 2023-03-09 21:24:39

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#3 2023-03-09 21:36:09

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#4 2023-03-10 09:21:35

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#5 2023-03-12 20:46:15

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#6 2023-03-12 21:23:49

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#7 2023-03-14 00:51:27

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#8 2023-03-14 10:23:19

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#9 2023-03-15 03:38:59

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#10 2023-03-15 07:35:21

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#11 2023-04-13 00:23:37

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#12 2023-04-13 01:19:28

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#13 2023-04-13 16:22:10

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#14 2023-04-17 22:03:03

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#15 2023-04-18 04:02:44

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#16 2023-04-18 10:37:28

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#17 2023-04-18 14:07:39

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#18 2023-04-28 15:29:20

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#19 2023-04-28 17:46:27

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#20 2023-04-30 08:53:16

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#21 2023-05-05 13:14:02

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#22 2023-05-09 11:00:44

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#23 2023-05-09 19:12:11

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#24 2023-05-12 16:21:55

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

#25 2023-05-12 16:44:04

Re: AMDGPU freeze problems with Linux 6.1.4+, need kernel bisection [SVD]

Board footer