Getting random Lock ups / freezes

RononDex · 2024-05-26 19:22:01

Since a few weeks / months I am getting random lock ups / freezes on my system and I am really unsure on how to correctly diagnose the issue.
My system info:
- AMD Ryzen 5800X
- Crosshair VII Wifi (Motherbaord)
- AMD 7900 XTX GPU
- Wayland with Hyprland

What usually happens: After a few hours of usage, sometimes I can come back to my pc after a short break and see it frozen (only recoverable through a hard reset with the power button).
Also once it happened once, it appears to happen more often (within a few minutes sometimes after reboot). I am also not 100% sure, but it seems that playing games (even non-GPU intensive ones) increasees the risk of it happening.

My journalctl of one of those incidents:

May 26 21:04:39 ATLANTIS-LAB kernel: Command line: BOOT_IMAGE=/vmlinuz-linux-vivepro2 root=UUID=8c1a32fa-7495-48b9-ab68-3605b9c01246 rw loglevel=3 quiet
May 26 21:04:39 ATLANTIS-LAB kernel: Linux version 6.8.7-arch1-1-vivepro2 (linux-vivepro2@archlinux) (gcc (GCC) 13.2.1 20240417, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Tue, 30 Apr 2024 06:57:16 +0000
-- Boot 04a9246409d645238c49f822bed19322 --
May 26 21:02:02 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 21:02:02 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 21:02:01 ATLANTIS-LAB crond[688]: FILE /var/spool/cron/root USER root PID 6012 job sys-daily
May 26 21:01:19 ATLANTIS-LAB NetworkManager[674]: <info>  [1716750079.5830] device (wlp6s0): supplicant interface state: interface_disabled -> inactive
May 26 21:01:19 ATLANTIS-LAB NetworkManager[674]: <info>  [1716750079.5828] device (wlp6s0): supplicant interface state: disconnected -> interface_disabled
May 26 21:01:19 ATLANTIS-LAB NetworkManager[674]: <info>  [1716750079.0959] device (wlp6s0): set-hw-addr: set MAC address to 16:13:92:70:30:16 (scanning)
May 26 21:01:01 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 21:01:01 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 21:00:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 21:00:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:59:58 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:59:58 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:59:27 ATLANTIS-LAB csc_vpnagent[701]: Function: GetSettings File: ../../vpn/Agent/ServicePluginMgr.cpp Line: 292 m_pIServicePlugin is NULL
May 26 20:58:59 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:59 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:25 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:25 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:25 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:25 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:12 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:58:12 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:57:58 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:57:58 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:57:39 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 12 threads of 8 processes of 1 users.
May 26 20:57:39 ATLANTIS-LAB rtkit-daemon[1164]: Successfully made thread 5463 of process 2706 owned by '1000' RT at priority 10.
May 26 20:57:39 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:57:39 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:57:36 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:57:36 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:56:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:56:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:55:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:55:42 ATLANTIS-LAB rtkit-daemon[1164]: Supervising 11 threads of 7 processes of 1 users.
May 26 20:55:06 ATLANTIS-LAB systemd[1]: systemd-timedated.service: Deactivated successfully.
May 26 20:55:04 ATLANTIS-LAB systemd[1]: blueman-mechanism.service: Deactivated successfully.
May 26 20:55:01 ATLANTIS-LAB crond[688]: FILE /var/spool/cron/root USER root PID 3452 job sys-hourly
May 26 20:54:55 ATLANTIS-LAB systemd[1]: systemd-hostnamed.service: Deactivated successfully.
May 26 20:54:54 ATLANTIS-LAB systemd[1]: Mounted Arbitrary Executable File Formats File System.
May 26 20:54:54 ATLANTIS-LAB systemd[1]: Mounting Arbitrary Executable File Formats File System...

There is nothing that jumps at me here.

Does anyone have an idea how I could try to diagnose the issue or figure out what is causing these freezes / lock ups?

RononDex · 2024-05-26 19:35:15

Also to add, just before it happens I usually get some wierd artifacts on the screen, it looks like frame tearing for a few frames or like random noise on a few pixel-rows on the screen

RononDex · 2024-05-26 20:20:07

I also got a 4k screen running at 144hz at around the time issues started happening, idk if that could be related

seth · 2024-05-26 22:04:36

only recoverable through a hard reset with the power button

This is gonna drop the journal.
See whether https://wiki.archlinux.org/title/Keyboa … el_(SysRq) works (nb. that you've to explicitly enable the feature first!)

Also then please don't filter the journal (you can obfuscate stuff you consider "private" but nothing in there is supposed to be sensitive) and under no circumstances use the -x (spam) or -r (top-posting problem) flags.

RononDex · 2024-05-27 06:08:26

Hello Seth

Thank you very much for you answer and time

I am enabling it after boot using

sysctl kernel.sysrq=1

If I understand it correctly, it allows me to reboot the system in case system starts hanging using some key combination? Should that retain the necessary journalctl entries?

seth · 2024-05-27 06:51:28

Yes. Please notice that the sysctl change will NOT survive a reboot, so you'd have to repeat that.
Depending on how bad the "lock up" (do the keyboard LEDs start flashing?) is alt+sysrq+REISUB will cause a somewhat clean reboot, incl. a disk sync (ie. the journal is actually written to the disk) and ideally the journal tail will hint the cause then.

RononDex · 2024-05-27 07:36:05

Thanks, I will try this next time it happens.

So to be clear, I should do alt+sysrq+REISUB (once for every REISUB key) when it happens?

seth · 2024-05-27 07:40:54

You press alt+sysrq and keep it pressed and wait ~3s between the (lower case, don't press shift) REISUB keys.
It's NOT an emacs style combo where you first press alt+sysrq, release it and then type reisub as some magic cheatcode or something like this. Every sysrq key does something different, REISUB is just a common pattern and easy mnemonic and will work here.

RononDex · 2024-05-27 19:11:19

So I just tried the SysReq sequence in two lock ups, it never worked. I tested it while the system was running normally and it worked then. So whatever is causing the crash / lock up also prevents SysReq from registering the key presses.

seth · 2024-05-27 19:51:49

Assuming it's not a hardware issue the only way to debug a kernel panic is https://wiki.archlinux.org/title/Kdump - and somewhat of a PITA
(nb. that you don't have to build a kernel for this, the required features are enabled in the pre-built arch kernels)

RononDex · 2024-05-27 20:06:14

To be honest I am starting to strongly suspect that it is my PSU slowsly dying (it is 5-6 years old and had heavy use).

A few components in my pc have that age (cpu cooler, motherboard, ram and psu).
CPU runs fine, I can render stuff with 100% usage for hours without issues, but as soon as I start using gpu in games those freezes start happening and then afterward happen even if I don't use the gpu in the next boot.
My interpretation is, that the PSU probably gets hot / unstable due to the higher power draw from the gpu and afterwards needs time to recover.

Usually the next day it is stable again for like 8-10 hours when I leave the pc off over night.

Also when it crashes I can still hear the audio buffer in infinite loop

seth · 2024-05-27 21:03:50

My interpretation is, that the PSU probably gets hot / unstable due to the higher power draw from the gpu and afterwards needs time to recover.

Do you have a ventilator you can point at it?
(Into the open case, don't blow against the PSU fan )

Arch Linux

#1 2024-05-26 19:22:01

Getting random Lock ups / freezes

#2 2024-05-26 19:35:15

Re: Getting random Lock ups / freezes

#3 2024-05-26 20:20:07

Re: Getting random Lock ups / freezes

#4 2024-05-26 22:04:36

Re: Getting random Lock ups / freezes

#5 2024-05-27 06:08:26

Re: Getting random Lock ups / freezes

#6 2024-05-27 06:51:28

Re: Getting random Lock ups / freezes

#7 2024-05-27 07:36:05

Re: Getting random Lock ups / freezes

#8 2024-05-27 07:40:54

Re: Getting random Lock ups / freezes

#9 2024-05-27 19:11:19

Re: Getting random Lock ups / freezes

#10 2024-05-27 19:51:49

Re: Getting random Lock ups / freezes

#11 2024-05-27 20:06:14

Re: Getting random Lock ups / freezes

#12 2024-05-27 21:03:50

Re: Getting random Lock ups / freezes

Board footer