You are not logged in.

#1 2021-04-14 15:07:48

Erotaku
Member
Registered: 2017-07-08
Posts: 66

GPU hangs while gaming

Greetings friends,

As the title suggests, I experience GPU hangs (AMD RX 5700 XT) after a few minutes of firing up a game. I tried seeking help from a number of discords, but no one was able to assist me. So, you are my last hope to tackle this beast of a problem.
Naturally, I have taken numerous steps to isolate the problem and make sure that it's not a program like lutris that is causing the issue. More relevant troubleshooting steps taken were:

  • Tried the up-to-date vanilla/standard, LTS, XanMod and TKG UPDS kernels.

  • Moved my games to a completely different SSD drive.

  • Switched to mesa-git.

  • Used two kernel parameters separately; iommu=pt and amdgpu.gpu_recovery=1.

  • Deleted all the relevant cache folders in case of corruption.

I was able to SSH into my system right after the crash and obtain a dmseg log with an error of "ring gfx timeout but soft recovered" here: PasteBin

Please end my living nightmare :(

Offline

#2 2021-04-14 15:21:16

V1del
Forum Moderator
Registered: 2012-10-16
Posts: 21,415

Re: GPU hangs while gaming

It's crashing somewhere in dc code, try amdgpu.dc=0 that will break HDMI audio and Freesync/VRR but should at least avoid this particular issue.

Offline

#3 2021-04-14 15:40:35

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

So, something interesting just happened. Pacman was compiling a package, and the GPU hang happened despite not gaming. I guess I was right that it has nothing to do with gaming then.

Last edited by Erotaku (2021-04-17 17:07:27)

Offline

#4 2021-04-14 15:49:47

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

V1del wrote:

It's crashing somewhere in dc code, try amdgpu.dc=0 that will break HDMI audio and Freesync/VRR but should at least avoid this particular issue.

That boots me to a glitched screen: https://cdn.discordapp.com/attachments/ … 174348.jpg

Offline

#5 2021-04-14 19:21:56

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

Did you find https://bugzilla.kernel.org/show_bug.cgi?id=207881 & https://bbs.archlinux.org/viewtopic.php?id=260550 (the latter has a potential wokaround that suggests PCI issues)

Online

#6 2021-04-14 22:35:29

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

Did you find https://bugzilla.kernel.org/show_bug.cgi?id=207881 & https://bbs.archlinux.org/viewtopic.php?id=260550 (the latter has a potential wokaround that suggests PCI issues)

There is not really much I can try from those links. The first link leads to a shockingly extremely long bug reports about similar issues. From what I have found from quickly sifting through them, I don't see any proposed solutions. Unless I missed something buried in that encyclopedia of a link.

Second link suggests using gen4 pcie in the bios. I don't have a gen4 motherboard to begin with, but I went ahead and tried setting it on gen 3 instead of auto. The GPU still hangs after testing it again.
It was also suggested to use another kernel. I have tested plenty already. I might have even tested the zen kernel as well.

Offline

#7 2021-04-15 07:04:24

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

https://bbs.archlinux.org/viewtopic.php … 2#p1967022 suggests amdgpu improvements (but for a different chip) w/ mesa-git and linux-firmware-git … maybe just don't use trizen to install them.

Online

#8 2021-04-15 14:55:57

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

https://bbs.archlinux.org/viewtopic.php … 2#p1967022 suggests amdgpu improvements (but for a different chip) w/ mesa-git and linux-firmware-git … maybe just don't use trizen to install them.

I used paru this time, don't worry. I thanked you more in the other topic, but thank you again!

Anyway, I just finished installing, rebooting and testing with the new git packages. The GPU still hangs within two minutes, sadly.

Would you actually recommend me to try the llvm-git branch instead, as another potential fix for my issue? A knowledgeable lad advised to me give it a try on discord two days ago.

Offline

#9 2021-04-16 15:40:43

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

Update: So, after reading and searching more and more about this curse... because that's all that I do with my free time for the past week. I discovered that more people did recommend the llvm-minimal-git package. So, I had to just go through with it as I am getting quite desperate.
.... The system crashed during the compiling process. I tried again, and it crashed again. So, I can't even install what could possibly fix this.

It's worth noting that the crash's behaviour is different from the GPU hang that I experience during gaming. The GPU (and screen as a result) doesn't turn off and on. Everything just freezes. It's also unresponsive compared to how the system retains its responsiveness during the gaming GPU hang.

Last edited by Erotaku (2021-04-17 17:08:48)

Offline

#10 2021-04-16 16:47:08

V1del
Forum Moderator
Registered: 2012-10-16
Posts: 21,415

Re: GPU hangs while gaming

How up to date is your UEFI? There have been freezing issues on older ryzen/agesa versions

Last edited by V1del (2021-04-16 18:18:17)

Offline

#11 2021-04-16 17:47:51

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

V1del wrote:

How up to date is your UEFI? There have been freezing issues on older ryzen/agesa versions which mo

I purchased my R7 2700 + motherboard well after the 3000 series chips came out. So I assume that my agesa doesn't need to be updated. Especially since I installed windows 10 the first thing after building my PC. Till this day, I don't have any issues on that operating system.

That being said, my AGESA version is 1.0.0.3, and my BIOS/UEFI is 1.80.

Offline

#12 2021-04-17 01:39:52

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

Another update: Disregard the last update. The llvm-minimal-git package was the problem. I only got discover that after trying to installing it manually and finding out the error message it was spitting out half way the compiling process. And it was responsible for the latest crash.

But yea, now that I can't try llvm-minimal. I have nothing to try to fix my original GPU hang. What a true nightmare...

Edit: The epiphany is real... So, that minimal llvm? It really needs a lot of memory. So much so, that 16GBs of ram plus a small swap partition/file are simply not sufficient. And that's where the error was stemming from. I recreated my swap file to be almost 9GBs, and I compiled it... BAM, it's done! Let the testing begin.

Last edited by Erotaku (2021-04-17 03:43:15)

Offline

#13 2021-04-17 22:36:19

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

Final update: I have installed llvm-minimal-git, mesa-git, lib32-llvm-minimal-git and lib32-mesa-git... And the GPU would still hangs after all what that I have i installed. There was only just one final step to take.
I installed amdgpu-pro... And guess what? I changed my lutris vulkan ICD loader to that, and it actually works! I have been in-game trying to push the GPU as much as I can, and it hasn't crashed in over an hour. Meanwhile, the moment I change back to radeon/opensource driver ICD, it would crash within minutes.

I have spent the better part of an entire week compiling a ton of packages back and forth, trying every kernel parameter/setting known to mankind and scouring every inch of the web... only to "fix" the issue with the pro drivers. My poor poor mental health.

But it's not over. What do I do from there? I can't keep using the pro drivers. It's cutting my performance in half, if not more. Can anyone advise me what the next step could be? Or do I just wait and hope for a magical fix to happen in mesa?

Offline

#14 2021-04-18 07:37:14

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

It's cutting my performance in half, if not more.

xorg log and glxinfo? I could imagine you're actually running on software emulation…

Online

#15 2021-04-18 13:58:49

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:
It's cutting my performance in half, if not more.

xorg log and glxinfo? I could imagine you're actually running on software emulation…

This is purely a guess on my part, but I think you are right. lshw still shows that I am using amggpu as my used driver. The only change I made was the vulkan ICD setting in lutris. Anyway, I only use wayland. So, I can only provide you the glxinfo log here: https://pastebin.com/G8TdkBqw

Offline

#16 2021-04-18 14:43:35

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

I only use wayland

Are you sure?
Because lutris looks like it using wine and that will most liekly still run on xwayland.
So on that revelation, I'd first and formost try the behavior on an Xorg session to cut out the xwayland layer.

"glxinfo" is GLX for sure so you'll get an xwayland server for that and that will also get you a log - thogh the glxinfo output doesn't look swrast-ish to me at all.

However: amdgpu does NOT provide a kernel module, but the glxinfo doesn't suggest you're somehow using it either, https://wiki.archlinux.org/index.php/AM … PRO_driver

Online

#17 2021-04-18 15:40:45

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

I only use wayland

Are you sure?
Because lutris looks like it using wine and that will most liekly still run on xwayland.

Oh no, my mistake. I did not mean a pure wayland environment without xwayland. Just without xorg instead.

So on that revelation, I'd first and formost try the behavior on an Xorg session to cut out the xwayland layer.

"glxinfo" is GLX for sure so you'll get an xwayland server for that and that will also get you a log - thogh the glxinfo output doesn't look swrast-ish to me at all.

However: amdgpu does NOT provide a kernel module, but the glxinfo doesn't suggest you're somehow using it either, https://wiki.archlinux.org/index.php/AM … PRO_driver

I am using SwayWM, not KDE/Gnome. So, I do not have any xorg sessions I could try out. And I would feel a bit silly installing an entire DE for that.
What doesn't add up in my head, is the fact that the GPU hang happens in the two games that push my GPU the highest, compared to the other two games that do not. Wayland could have been the culprit if the experienced problem affected each and every game I would play.

As for the wiki link. Yea, I have tried that command before. It always returns "AMD" as the result. So yea, the amdgpu-pro driver is not chosen on boot. Is there a way to enforce the system to use it without having to uninstall the mesa drivers (if that's even the way to do it)?

Offline

#18 2021-04-18 15:46:22

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

Openbox/fluxbox/icewm/i3/dwm/… are going to cut it. You don't need KDE or Gnome for this (as a matter of fact, "startx xterm" would do)

I don't think that "amdgpu-pro for wayland" is a thing, but that's moot because whatever has caused the change is apparently not the amdgpu-pro driver and I'm not even sure whether there's a change or whether this is temperature or power supply driven (ie. the CPU/GPU won't upscale as much on battery as on the wall plug…)

Online

#19 2021-04-18 17:01:00

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

Openbox/fluxbox/icewm/i3/dwm/… are going to cut it. You don't need KDE or Gnome for this (as a matter of fact, "startx xterm" would do)

I don't think that "amdgpu-pro for wayland" is a thing, but that's moot because whatever has caused the change is apparently not the amdgpu-pro driver and I'm not even sure whether there's a change or whether this is temperature or power supply driven (ie. the CPU/GPU won't upscale as much on battery as on the wall plug…)

I hope this is the Xorg log: https://pastebin.com/KFDxkYEM I feel like taking a shower after installing xorg on my system just now.

I am using a PC with no batteries, by the way. Also, my PC thermals are excellent during gaming. The GPU hang does not happen while using windows. So, I don't think thermals/wattage could be the reason. Unless linux changes things up there.

Offline

#20 2021-04-18 17:08:56

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

I feel like taking a shower after installing xorg on my system just now.

Take some extra ones. What do you think xwayland is? It's a nested xorg server.
You're running yor games on Xorg on a wayland host…

You're running on the modesetting driver (and DISPLAY :2, so there're two other Xorg servers running?) - there's no trace of either amdgpu nor the amdgpu-pro driver but for some reason you installed xf86-video-vesa and also ati is considered rather than amdgpu - which doesn't seem quite right.

The more interesting question though is which impact this has on game performance and stability.
Make sure there's not a parallel wayland server (so the two don't argue about who gets the GPU)

Online

#21 2021-04-18 17:34:52

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

I feel like taking a shower after installing xorg on my system just now.

Take some extra ones. What do you think xwayland is? It's a nested xorg server.
You're running yor games on Xorg on a wayland host…

Hey, not for long! Fingers crossed for wine-wayland. On a more serious note, wine is the only thing I can think of that does run under xwayland. I even removed dunst for mako because things running under xwayland exhibit weird behaviour in some of the times.

You're running on the modesetting driver (and DISPLAY :2, so there're two other Xorg servers running?) - there's no trace of either amdgpu nor the amdgpu-pro driver but for some reason you installed xf86-video-vesa and also ati is considered rather than amdgpu - which doesn't seem quite right.

The more interesting question though is which impact this has on game performance and stability.
Make sure there's not a parallel wayland server (so the two don't argue about who gets the GPU)

Right. I restarted and changed lightdm settings. Hopefully this should be a lot cleaner: https://pastebin.com/kCzfguts
I also have absolutely no idea what installed that xf86 driver... I swiftly removed that.

Offline

#22 2021-04-18 20:02:50

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

There hasn't much changed about the driver constellation, you're certainly not using the amdgpu-pro driver (nor xf86-video-amdgpu)

Sooo…

The more interesting question though is which impact this has on game performance and stability.

Online

#23 2021-04-18 20:25:53

Erotaku
Member
Registered: 2017-07-08
Posts: 66

Re: GPU hangs while gaming

seth wrote:

There hasn't much changed about the driver constellation, you're certainly not using the amdgpu-pro driver (nor xf86-video-amdgpu)

Yep, that's something we already knew from the glxinfo command's output from the wiki, right? I am sorry, but there seems to be a disconnect in communication here. I could be missing something, so excuse and correct me if that's the case.
Just to reiterate: Setting the vulkan ICD to the amdgpu-pro ICD changes the driver ONLY for the game I specify and target. So, playing around with that setting will not show in any glxinfo or log (unless it's lutris logs itself). I guess take it as if lutris is containerizing the game in that driver environment. That's how I interpret it with my limited understanding.
Whatever that may amount to on paper or logs, the GPU hang does not happen anymore when I do that.

Sooo…

The more interesting question though is which impact this has on game performance and stability.

I am not sure I understand this question clearly as well. Amdgpu-pro drivers perform differently from the opensource driver's counterpart. That's a given. I thought to myself "maybe because it's wayland" and you said amdgpu-pro is might not be a thing there. But I tried the game on xorg, and the poor performance is identical. So, that's out-ruled.

As for the stability, I think it's simple. The opensource drivers is the culprit here; because I tested everything else and it showed no signs of being responsible. So, there is some sort of bug in the drivers that is tied in with my GPU. That bug is not present in the pro drivers? Again, that's the only way I can justify and make sense out of my findings.

Edit: After testing more and more and more. The shader compiling started calming down during the gameplay, and the performance of the pro drivers actually went up quite a bit. While it's getting close to the opensource drivers' level, it's still not there 100%. I think I am losing around 10-15 fps. While that is very nice to see, I still don't think this is the fix I should be using. Pro drivers have some weird behaviour like game cursor's shape not being consistent. Ideally I should be using the opensource counterpart. Anyway, just wanted to drop this update for clarity's sake.

Last edited by Erotaku (2021-04-19 03:39:19)

Offline

#24 2021-04-19 07:01:11

seth
Member
Registered: 2012-09-03
Posts: 49,956

Re: GPU hangs while gaming

I tested everything else

Incl. the amdgpu driver on xorg (no wayland/xwayland)?

Did you install the  amdgpu-pro-libgl &  lib32-amdgpu-pro-libgl packages and what does vulkaninfo tell you (idk. whether the pro vulkan libs work w/o the GL ones or xf86-video-amdgpu but it's kinda suspicious that your performance dies and the stbility improves w/ that driver which makes me wonder whether you're actually running the sw emulation. Perhaps game™ allows you to query the used renderer?)

Online

#25 2021-04-19 11:09:54

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: GPU hangs while gaming

For clarity : are all games that hang using vulkan ?


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

Board footer

Powered by FluxBB