You are not logged in.

#1 2025-06-07 12:53:59

sphere101
Member
Registered: 2017-08-04
Posts: 42

amdgpu: GPU reset begin

Hello everyone,
I have been running into these lock up scenario's for a few weeks now, seems like a bug. Here are the logs in journal.

Jun 07 07:29:36 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:29:41 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:41 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:45 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:45 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Dumping IP State Completed
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: ring sdma1 timeout, signaled seq=26744, emitted seq=26745
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: GPU reset begin!
Jun 07 07:30:13 figaro kernel: watchdog: Watchdog detected hard LOCKUP on cpu 3
Jun 07 07:30:13 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:13 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:13 figaro kernel: CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Tainted: P           OE      6.12.31-1-lts #1 e7c67df8a8ca221e76be609de09b67>
Jun 07 07:30:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Jun 07 07:30:13 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:13 figaro kernel: RIP: 0010:delay_halt_mwaitx+0x20/0x50
Jun 07 07:30:13 figaro kernel: Code: 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 53 65 48 8b 05 a6 49 fb 4e 31 d2 48 89 d1 48 05 00>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e802ccc78 EFLAGS: 00000086
Jun 07 07:30:13 figaro kernel: RAX: ffff8e1d6eb86000 RBX: 0000000000000ed2 RCX: 0000000000000000
Jun 07 07:30:13 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000ed2 RDI: 0001ac9368c9ddd6
Jun 07 07:30:13 figaro kernel: RBP: 0001ac9368c9ddd6 R08: 0000000000002000 R09: 0000000000001e40
Jun 07 07:30:13 figaro kernel: R10: ffffd12ea10c6d00 R11: ffff8e0e8ae90c00 R12: 00000000000186a0
Jun 07 07:30:13 figaro kernel: R13: ffffd12e802ccd10 R14: 0000000000000000 R15: 0000000000000001
Jun 07 07:30:13 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6eb80000(0000) knlGS:0000000000000000
Jun 07 07:30:13 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:13 figaro kernel: CR2: 00007685e7c55000 CR3: 0000000134dd0000 CR4: 0000000000f50ef0
Jun 07 07:30:13 figaro kernel: PKRU: 55555554
Jun 07 07:30:13 figaro kernel: Call Trace:
Jun 07 07:30:13 figaro kernel:  <IRQ>
Jun 07 07:30:13 figaro kernel:  delay_halt+0x3f/0x70
Jun 07 07:30:13 figaro kernel:  dmub_srv_wait_for_idle+0x55/0x90 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_dmub_srv_cmd_run_list+0x7d/0x170 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_dmub_srv_drr_update_cmd+0x99/0x110 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  optc1_set_drr+0x59/0x1b0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:30:13 figaro kernel:  ? __pfx_optc1_set_drr+0x10/0x10 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  set_drr_and_clear_adjust_pending+0x3c/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dcn10_set_drr+0x82/0xe0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_stream_adjust_vmin_vmax+0x147/0x250 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dm_crtc_high_irq+0x2b4/0x320 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_dm_irq_handler+0x85/0x1f0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_irq_dispatch+0xd2/0x230 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_ih_process+0x80/0x100 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_irq_handler+0x23/0x60 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  __handle_irq_event_percpu+0x4a/0x190
Jun 07 07:30:13 figaro kernel:  handle_irq_event+0x38/0x80
Jun 07 07:30:13 figaro kernel:  handle_edge_irq+0x8b/0x220
Jun 07 07:30:13 figaro kernel:  __common_interrupt+0x41/0xa0
Jun 07 07:30:13 figaro kernel:  common_interrupt+0x80/0xa0
Jun 07 07:30:13 figaro kernel:  </IRQ>
Jun 07 07:30:13 figaro kernel:  <TASK>
Jun 07 07:30:13 figaro kernel:  asm_common_interrupt+0x26/0x40
Jun 07 07:30:13 figaro kernel: RIP: 0010:cpuidle_enter_state+0xbb/0x410
Jun 07 07:30:13 figaro kernel: Code: 00 00 e8 98 62 26 ff e8 83 f1 ff ff 48 89 c5 0f 1f 44 00 00 31 ff e8 94 f2 24 ff 45 84 ff 0f 85 a6 01 00 00>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e801c7e78 EFLAGS: 00000246
Jun 07 07:30:13 figaro kernel: RAX: ffff8e1d6eb80000 RBX: 0000000000000002 RCX: 0000000000000000
Jun 07 07:30:13 figaro kernel: RDX: 000070f5c31ba958 RSI: fffffffd0804d3c6 RDI: 0000000000000000
Jun 07 07:30:13 figaro kernel: RBP: 000070f5c31ba958 R08: 0000000000000002 R09: ffff8e1d6ebb51c4
Jun 07 07:30:13 figaro kernel: R10: 000000000000485d R11: ffff8e1d6ebb51a4 R12: ffff8e0e80e6dc00
Jun 07 07:30:13 figaro kernel: R13: ffffffffb25c7e20 R14: 0000000000000002 R15: 0000000000000000
Jun 07 07:30:13 figaro kernel:  ? cpuidle_enter_state+0xac/0x410
Jun 07 07:30:13 figaro kernel:  cpuidle_enter+0x31/0x50
Jun 07 07:30:13 figaro kernel:  do_idle+0x1b6/0x210
Jun 07 07:30:13 figaro kernel:  cpu_startup_entry+0x29/0x30
Jun 07 07:30:13 figaro kernel:  start_secondary+0x11e/0x140
Jun 07 07:30:13 figaro kernel:  common_startup_64+0x13e/0x141
Jun 07 07:30:13 figaro kernel:  </TASK>
Jun 07 07:30:13 figaro kernel: INFO: NMI handler (perf_event_nmi_handler) took too long to run: 2.544 msecs
Jun 07 07:30:13 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [kworker/u64:4:237372]
Jun 07 07:30:13 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:30:13 figaro kernel:         #1: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:13 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:13 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OE      6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:30:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Jun 07 07:30:13 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:13 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:30:13 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:30:13 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:30:13 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:30:13 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:30:13 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:30:13 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:30:13 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:30:13 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:30:13 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:13 figaro kernel: CR2: 000076872e267000 CR3: 000000012e3f0000 CR4: 0000000000f50ef0
Jun 07 07:30:13 figaro kernel: PKRU: 55555554
Jun 07 07:30:13 figaro kernel: Call Trace:
Jun 07 07:30:13 figaro kernel:  <TASK>
Jun 07 07:30:13 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:30:13 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:30:13 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:30:13 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:30:13 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:30:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:30:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:30:13 figaro kernel:  </TASK>
 

Any suggestions or additional information would be helpful.
Thank you.

Offline

#2 2025-06-07 13:10:51

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 13,804

Re: amdgpu: GPU reset begin

These look a bit weird for a gpu lock up.

Jun 07 07:30:13 figaro kernel: watchdog: Watchdog detected hard LOCKUP on cpu 3

Jun 07 07:30:13 figaro kernel: CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Tainted: P           OE      6.12.31-1-lts #1 e7c67df8a8ca221e76be609de09b67>
Jun 07 07:30:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE

Please post the full output of lspci -k and the whole journal from the last lockup .


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Online

#3 2025-06-07 13:16:16

sphere101
Member
Registered: 2017-08-04
Posts: 42

Re: amdgpu: GPU reset begin

Sure, here is are the logs

figaro ~]$ lspci -k
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex
	Subsystem: ASUSTeK Computer Inc. Device 8808
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU
	Subsystem: ASUSTeK Computer Inc. Device 8808
00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: pcieport
00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: pcieport
00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
	DeviceName:  Onboard IGD
00:03.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:03.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: pcieport
00:04.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:05.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:07.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:07.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: pcieport
00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge
00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B]
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: pcieport
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
	Subsystem: ASUSTeK Computer Inc. Device 87c0
	Kernel driver in use: piix4_smbus
	Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
	Subsystem: ASUSTeK Computer Inc. Device 87c0
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 3
	Kernel driver in use: k10temp
	Kernel modules: k10temp
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 5
00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 6
00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Matisse/Vermeer Data Fabric: Device 18h; Function 7
01:00.0 Non-Volatile memory controller: Sandisk Corp WD Black SN850X NVMe SSD (rev 01)
	Subsystem: Sandisk Corp WD Black SN850X NVMe SSD
	Kernel driver in use: nvme
	Kernel modules: nvme
02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset USB 3.1 XHCI Controller
	Subsystem: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller
	Kernel driver in use: xhci_hcd
02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset SATA Controller
	Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller
	Kernel driver in use: ahci
02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] 500 Series Chipset Switch Upstream Port
	Subsystem: ASMedia Technology Inc. Device 0201
	Kernel driver in use: pcieport
03:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea
	Subsystem: ASMedia Technology Inc. Device 3308
	Kernel driver in use: pcieport
03:08.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea
	Subsystem: ASMedia Technology Inc. Device 3308
	Kernel driver in use: pcieport
03:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43ea
	Subsystem: ASMedia Technology Inc. Device 3308
	Kernel driver in use: pcieport
05:00.0 Network controller: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter
	Subsystem: Foxconn International, Inc. Device e0cd
	Kernel driver in use: mt7921e
	Kernel modules: mt7921e
06:00.0 Ethernet controller: Intel Corporation Ethernet Controller I226-V (rev 06)
	Subsystem: ASUSTeK Computer Inc. Device 8867
	Kernel driver in use: igc
	Kernel modules: igc
07:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch (rev c7)
	Kernel driver in use: pcieport
08:00.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch
	Kernel driver in use: pcieport
09:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 23 [Radeon RX 6600/6600 XT/6600M] (rev c7)
	Subsystem: ASRock Incorporation Device 5217
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu
09:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller
	Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21/23 HDMI/DP Audio Controller
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function
	Subsystem: ASUSTeK Computer Inc. Device 8808
0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
	Subsystem: ASUSTeK Computer Inc. Device 8808
0b:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
	Subsystem: ASUSTeK Computer Inc. Device 8808
	Kernel driver in use: ccp
	Kernel modules: ccp
0b:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller
	Subsystem: ASUSTeK Computer Inc. Device 87c0
	Kernel driver in use: xhci_hcd
0b:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller
	Subsystem: ASUSTeK Computer Inc. Device 87c5
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

Journal logs

Jun 07 07:28:41 figaro rtkit-daemon[1475]: Successfully made thread 239574 of process 239270 owned by '1000' RT at priority 10.
Jun 07 07:28:41 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:28:42 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:28:42 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:29:20 figaro kernel: amdgpu 0000:09:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
Jun 07 07:29:20 figaro kernel: amdgpu 0000:09:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
Jun 07 07:29:20 figaro kernel: amdgpu 0000:09:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
Jun 07 07:29:22 figaro unix_chkpwd[239673]: password check failed for user (root)
Jun 07 07:29:22 figaro sshd-session[239671]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=188.138.122>
Jun 07 07:29:24 figaro sshd-session[239671]: Failed password for root from 188.138.122.80 port 37330 ssh2
Jun 07 07:29:24 figaro sshd-session[239671]: Received disconnect from 188.138.122.80 port 37330:11: Bye Bye [preauth]
Jun 07 07:29:24 figaro sshd-session[239671]: Disconnected from authenticating user root 188.138.122.80 port 37330 [preauth]
Jun 07 07:29:25 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:25 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:30 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:30 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:30 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Dumping IP State
Jun 07 07:29:36 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:36 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:36 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:29:36 figaro rtkit-daemon[1475]: Supervising 11 threads of 8 processes of 1 users.
Jun 07 07:29:41 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:41 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:45 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:45 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82>
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Failed to disable gfxoff!
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: Dumping IP State Completed
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: ring sdma1 timeout, signaled seq=26744, emitted seq=26745
Jun 07 07:29:50 figaro kernel: amdgpu 0000:09:00.0: amdgpu: GPU reset begin!
Jun 07 07:30:13 figaro kernel: watchdog: Watchdog detected hard LOCKUP on cpu 3
Jun 07 07:30:13 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:13 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:13 figaro kernel: CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Tainted: P           OE      6.12.31-1-lts #1 e7c67df8a8ca221e76be609de09b67>
Jun 07 07:30:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Jun 07 07:30:13 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:13 figaro kernel: RIP: 0010:delay_halt_mwaitx+0x20/0x50
Jun 07 07:30:13 figaro kernel: Code: 90 90 90 90 90 90 90 90 90 90 f3 0f 1e fa 0f 1f 44 00 00 53 65 48 8b 05 a6 49 fb 4e 31 d2 48 89 d1 48 05 00>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e802ccc78 EFLAGS: 00000086
Jun 07 07:30:13 figaro kernel: RAX: ffff8e1d6eb86000 RBX: 0000000000000ed2 RCX: 0000000000000000
Jun 07 07:30:13 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000ed2 RDI: 0001ac9368c9ddd6
Jun 07 07:30:13 figaro kernel: RBP: 0001ac9368c9ddd6 R08: 0000000000002000 R09: 0000000000001e40
Jun 07 07:30:13 figaro kernel: R10: ffffd12ea10c6d00 R11: ffff8e0e8ae90c00 R12: 00000000000186a0
Jun 07 07:30:13 figaro kernel: R13: ffffd12e802ccd10 R14: 0000000000000000 R15: 0000000000000001
Jun 07 07:30:13 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6eb80000(0000) knlGS:0000000000000000
Jun 07 07:30:13 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:13 figaro kernel: CR2: 00007685e7c55000 CR3: 0000000134dd0000 CR4: 0000000000f50ef0
Jun 07 07:30:13 figaro kernel: PKRU: 55555554
Jun 07 07:30:13 figaro kernel: Call Trace:
Jun 07 07:30:13 figaro kernel:  <IRQ>
Jun 07 07:30:13 figaro kernel:  delay_halt+0x3f/0x70
Jun 07 07:30:13 figaro kernel:  dmub_srv_wait_for_idle+0x55/0x90 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_dmub_srv_cmd_run_list+0x7d/0x170 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_dmub_srv_drr_update_cmd+0x99/0x110 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  optc1_set_drr+0x59/0x1b0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:30:13 figaro kernel:  ? __pfx_optc1_set_drr+0x10/0x10 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  set_drr_and_clear_adjust_pending+0x3c/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dcn10_set_drr+0x82/0xe0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dc_stream_adjust_vmin_vmax+0x147/0x250 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  dm_crtc_high_irq+0x2b4/0x320 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_dm_irq_handler+0x85/0x1f0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_irq_dispatch+0xd2/0x230 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_ih_process+0x80/0x100 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_irq_handler+0x23/0x60 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  __handle_irq_event_percpu+0x4a/0x190
Jun 07 07:30:13 figaro kernel:  handle_irq_event+0x38/0x80
Jun 07 07:30:13 figaro kernel:  handle_edge_irq+0x8b/0x220
Jun 07 07:30:13 figaro kernel:  __common_interrupt+0x41/0xa0
Jun 07 07:30:13 figaro kernel:  common_interrupt+0x80/0xa0
Jun 07 07:30:13 figaro kernel:  </IRQ>
Jun 07 07:30:13 figaro kernel:  <TASK>
Jun 07 07:30:13 figaro kernel:  asm_common_interrupt+0x26/0x40
Jun 07 07:30:13 figaro kernel: RIP: 0010:cpuidle_enter_state+0xbb/0x410
Jun 07 07:30:13 figaro kernel: Code: 00 00 e8 98 62 26 ff e8 83 f1 ff ff 48 89 c5 0f 1f 44 00 00 31 ff e8 94 f2 24 ff 45 84 ff 0f 85 a6 01 00 00>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e801c7e78 EFLAGS: 00000246
Jun 07 07:30:13 figaro kernel: RAX: ffff8e1d6eb80000 RBX: 0000000000000002 RCX: 0000000000000000
Jun 07 07:30:13 figaro kernel: RDX: 000070f5c31ba958 RSI: fffffffd0804d3c6 RDI: 0000000000000000
Jun 07 07:30:13 figaro kernel: RBP: 000070f5c31ba958 R08: 0000000000000002 R09: ffff8e1d6ebb51c4
Jun 07 07:30:13 figaro kernel: R10: 000000000000485d R11: ffff8e1d6ebb51a4 R12: ffff8e0e80e6dc00
Jun 07 07:30:13 figaro kernel: R13: ffffffffb25c7e20 R14: 0000000000000002 R15: 0000000000000000
Jun 07 07:30:13 figaro kernel:  ? cpuidle_enter_state+0xac/0x410
Jun 07 07:30:13 figaro kernel:  cpuidle_enter+0x31/0x50
Jun 07 07:30:13 figaro kernel:  do_idle+0x1b6/0x210
Jun 07 07:30:13 figaro kernel:  cpu_startup_entry+0x29/0x30
Jun 07 07:30:13 figaro kernel:  start_secondary+0x11e/0x140
Jun 07 07:30:13 figaro kernel:  common_startup_64+0x13e/0x141
Jun 07 07:30:13 figaro kernel:  </TASK>
Jun 07 07:30:13 figaro kernel: INFO: NMI handler (perf_event_nmi_handler) took too long to run: 2.544 msecs
Jun 07 07:30:13 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [kworker/u64:4:237372]
Jun 07 07:30:13 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:30:13 figaro kernel:         #1: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:13 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:13 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:13 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OE      6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:30:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
Jun 07 07:30:13 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:13 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:30:13 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:30:13 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:30:13 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:30:13 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:30:13 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:30:13 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:30:13 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:30:13 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:30:13 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:30:13 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:13 figaro kernel: CR2: 000076872e267000 CR3: 000000012e3f0000 CR4: 0000000000f50ef0
Jun 07 07:30:13 figaro kernel: PKRU: 55555554
Jun 07 07:30:13 figaro kernel: Call Trace:
Jun 07 07:30:13 figaro kernel:  <TASK>
Jun 07 07:30:13 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:30:13 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:13 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:30:13 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:30:13 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:30:13 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:30:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:30:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:13 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:30:13 figaro kernel:  </TASK>
Jun 07 07:30:25 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [rcu_exp_gp_kthr:22]
Jun 07 07:30:25 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:30:25 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:25 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:25 figaro kernel:         #3: 100% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:30:25 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:25 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:25 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:25 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:25 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:30:25 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:30:25 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:25 figaro kernel: RIP: 0010:smp_call_function_single+0xe9/0x140
Jun 07 07:30:25 figaro kernel: Code: 38 65 48 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08>
Jun 07 07:30:25 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:30:25 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:30:25 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:30:25 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:30:25 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:30:25 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:30:25 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:30:25 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:25 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:30:25 figaro kernel: PKRU: 55555554
Jun 07 07:30:25 figaro kernel: Call Trace:
Jun 07 07:30:25 figaro kernel:  <TASK>
Jun 07 07:30:25 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:30:25 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:30:25 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:30:25 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:30:25 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:30:25 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:30:25 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:30:25 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:25 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:30:25 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:25 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:30:25 figaro kernel:  </TASK>
Jun 07 07:30:38 figaro kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Jun 07 07:30:38 figaro kernel: rcu:         3-...0: (8 GPs behind) idle=e88c/1/0x4000000000000002 softirq=2895029/2895029 fqs=5531
Jun 07 07:30:38 figaro kernel: rcu:         (detected by 4, t=18002 jiffies, g=6414797, q=13749 ncpus=16)
Jun 07 07:30:38 figaro kernel: Sending NMI from CPU 4 to CPUs 3:
Jun 07 07:30:38 figaro kernel: NMI backtrace for cpu 3
Jun 07 07:30:38 figaro kernel: CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be609de09b67>
Jun 07 07:30:38 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:30:38 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:38 figaro kernel: RIP: 0010:amdgpu_device_rreg.part.0+0x38/0xf0 [amdgpu]
Jun 07 07:30:38 figaro kernel: Code: 00 00 00 53 48 89 fb 48 83 ec 08 48 3b 97 f8 08 00 00 73 18 a8 02 75 09 f6 87 98 a2 04 00 10 75 78 48 03 93>
Jun 07 07:30:38 figaro kernel: RSP: 0018:ffffd12e802ccc60 EFLAGS: 00000086
Jun 07 07:30:38 figaro kernel: RAX: 0000000000000000 RBX: ffff8e0e93600000 RCX: 0000000000001e80
Jun 07 07:30:38 figaro kernel: RDX: ffffd12e8470da5c RSI: 0000000000003697 RDI: ffff8e0e93600000
Jun 07 07:30:38 figaro kernel: RBP: 0000000000003697 R08: 0000000000002000 R09: 0000000000001e40
Jun 07 07:30:38 figaro kernel: R10: ffffd12ea10c6d00 R11: ffff8e0e8ae90c00 R12: 00000000000186a0
Jun 07 07:30:38 figaro kernel: R13: ffffd12e802ccd10 R14: 0000000000000000 R15: 0000000000000001
Jun 07 07:30:38 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6eb80000(0000) knlGS:0000000000000000
Jun 07 07:30:38 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:38 figaro kernel: CR2: 00007685e7c55000 CR3: 0000000134dd0000 CR4: 0000000000f50ef0
Jun 07 07:30:38 figaro kernel: PKRU: 55555554
Jun 07 07:30:38 figaro kernel: Call Trace:
Jun 07 07:30:38 figaro kernel:  <IRQ>
Jun 07 07:30:38 figaro kernel:  dm_read_reg_func+0x5c/0xe0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  ? delay_halt+0x3f/0x70
Jun 07 07:30:38 figaro kernel:  dmub_srv_wait_for_idle+0x32/0x90 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  dc_dmub_srv_cmd_run_list+0x7d/0x170 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  dc_dmub_srv_drr_update_cmd+0x99/0x110 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  optc1_set_drr+0x59/0x1b0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:30:38 figaro kernel:  ? __pfx_optc1_set_drr+0x10/0x10 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  set_drr_and_clear_adjust_pending+0x3c/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  dcn10_set_drr+0x82/0xe0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  dc_stream_adjust_vmin_vmax+0x147/0x250 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  dm_crtc_high_irq+0x2b4/0x320 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  amdgpu_dm_irq_handler+0x85/0x1f0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  amdgpu_irq_dispatch+0xd2/0x230 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  amdgpu_ih_process+0x80/0x100 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  amdgpu_irq_handler+0x23/0x60 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:38 figaro kernel:  __handle_irq_event_percpu+0x4a/0x190
Jun 07 07:30:38 figaro kernel:  handle_irq_event+0x38/0x80
Jun 07 07:30:38 figaro kernel:  handle_edge_irq+0x8b/0x220
Jun 07 07:30:38 figaro kernel:  __common_interrupt+0x41/0xa0
Jun 07 07:30:38 figaro kernel:  common_interrupt+0x80/0xa0
Jun 07 07:30:38 figaro kernel:  </IRQ>
Jun 07 07:30:38 figaro kernel:  <TASK>
Jun 07 07:30:38 figaro kernel:  asm_common_interrupt+0x26/0x40
Jun 07 07:30:38 figaro kernel: RIP: 0010:cpuidle_enter_state+0xbb/0x410
Jun 07 07:30:38 figaro kernel: Code: 00 00 e8 98 62 26 ff e8 83 f1 ff ff 48 89 c5 0f 1f 44 00 00 31 ff e8 94 f2 24 ff 45 84 ff 0f 85 a6 01 00 00>
Jun 07 07:30:38 figaro kernel: RSP: 0018:ffffd12e801c7e78 EFLAGS: 00000246
Jun 07 07:30:38 figaro kernel: RAX: ffff8e1d6eb80000 RBX: 0000000000000002 RCX: 0000000000000000
Jun 07 07:30:38 figaro kernel: RDX: 000070f5c31ba958 RSI: fffffffd0804d3c6 RDI: 0000000000000000
Jun 07 07:30:38 figaro kernel: RBP: 000070f5c31ba958 R08: 0000000000000002 R09: ffff8e1d6ebb51c4
Jun 07 07:30:38 figaro kernel: R10: 000000000000485d R11: ffff8e1d6ebb51a4 R12: ffff8e0e80e6dc00
Jun 07 07:30:38 figaro kernel: R13: ffffffffb25c7e20 R14: 0000000000000002 R15: 0000000000000000
Jun 07 07:30:38 figaro kernel:  ? cpuidle_enter_state+0xac/0x410
Jun 07 07:30:38 figaro kernel:  cpuidle_enter+0x31/0x50
Jun 07 07:30:38 figaro kernel:  do_idle+0x1b6/0x210
Jun 07 07:30:38 figaro kernel:  cpu_startup_entry+0x29/0x30
Jun 07 07:30:38 figaro kernel:  start_secondary+0x11e/0x140
Jun 07 07:30:38 figaro kernel:  common_startup_64+0x13e/0x141
Jun 07 07:30:38 figaro kernel:  </TASK>
Jun 07 07:30:38 figaro kernel: INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 2.819 msecs
Jun 07 07:30:41 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 47s! [kworker/u64:4:237372]
Jun 07 07:30:41 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:30:41 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:41 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:41 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:41 figaro kernel:         #4: 100% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:30:41 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:41 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:41 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:41 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:30:41 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:30:41 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:41 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:30:41 figaro kernel: RIP: 0010:__synchronize_hardirq+0x35/0xc0
Jun 07 07:30:41 figaro kernel: Code: 55 48 8d af a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24>
Jun 07 07:30:41 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:30:41 figaro kernel: RAX: 0000000019458200 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:30:41 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:30:41 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:30:41 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:30:41 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:30:41 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:30:41 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:41 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:30:41 figaro kernel: PKRU: 55555554
Jun 07 07:30:41 figaro kernel: Call Trace:
Jun 07 07:30:41 figaro kernel:  <TASK>
Jun 07 07:30:41 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:30:41 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:41 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:41 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:41 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:30:41 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:30:41 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:30:41 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:30:41 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:30:41 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:30:41 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:41 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:30:41 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:41 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:30:41 figaro kernel:  </TASK>
Jun 07 07:30:53 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 49s! [rcu_exp_gp_kthr:22]
Jun 07 07:30:53 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:30:53 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:53 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:53 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:53 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:53 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:30:53 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:30:53 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:30:53 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:30:53 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:30:53 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:30:53 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:30:53 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:30:53 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:30:53 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:30:53 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:30:53 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:30:53 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:30:53 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:30:53 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:30:53 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:30:53 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:30:53 figaro kernel: PKRU: 55555554
Jun 07 07:30:53 figaro kernel: Call Trace:
Jun 07 07:30:53 figaro kernel:  <TASK>
Jun 07 07:30:53 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:30:53 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:30:53 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:30:53 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:30:53 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:30:53 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:30:53 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:30:53 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:53 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:30:53 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:30:53 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:30:53 figaro kernel:  </TASK>
Jun 07 07:31:09 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 73s! [kworker/u64:4:237372]
Jun 07 07:31:09 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:31:09 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:09 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:09 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:09 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:09 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:09 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:31:09 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:31:09 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:31:09 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:31:09 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:31:09 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:31:09 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:31:09 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:31:09 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:31:09 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:31:09 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:31:09 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:31:09 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:31:09 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:31:09 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:31:09 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:31:09 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:31:09 figaro kernel: PKRU: 55555554
Jun 07 07:31:09 figaro kernel: Call Trace:
Jun 07 07:31:09 figaro kernel:  <TASK>
Jun 07 07:31:09 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:31:09 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:09 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:09 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:09 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:09 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:31:09 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:31:09 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:31:09 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:31:09 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:31:09 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:09 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:31:09 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:09 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:31:09 figaro kernel:  </TASK>
Jun 07 07:31:21 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 75s! [rcu_exp_gp_kthr:22]
Jun 07 07:31:21 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:31:21 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:21 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:21 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:21 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:21 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:21 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:31:21 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:31:21 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:31:21 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:31:21 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:31:21 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:31:21 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:31:21 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:31:21 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:31:21 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:31:21 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:31:21 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:31:21 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:31:21 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:31:21 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:31:21 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:31:21 figaro kernel: PKRU: 55555554
Jun 07 07:31:21 figaro kernel: Call Trace:
Jun 07 07:31:21 figaro kernel:  <TASK>
Jun 07 07:31:21 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:31:21 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:31:21 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:31:21 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:31:21 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:31:21 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:31:21 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:31:21 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:21 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:31:21 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:21 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:31:21 figaro kernel:  </TASK>
Jun 07 07:31:37 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 100s! [kworker/u64:4:237372]
Jun 07 07:31:37 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:31:37 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:37 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:37 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:37 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:37 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:37 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:31:37 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:31:37 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:31:37 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:31:37 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:31:37 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:31:37 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:31:37 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:31:37 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:31:37 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:31:37 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:31:37 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:31:37 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:31:37 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:31:37 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:31:37 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:31:37 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:31:37 figaro kernel: PKRU: 55555554
Jun 07 07:31:37 figaro kernel: Call Trace:
Jun 07 07:31:37 figaro kernel:  <TASK>
Jun 07 07:31:37 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:31:37 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:37 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:37 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:37 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:31:37 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:31:37 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:31:37 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:31:37 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:31:37 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:31:37 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:37 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:31:37 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:37 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:31:37 figaro kernel:  </TASK>
Jun 07 07:31:49 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 101s! [rcu_exp_gp_kthr:22]
Jun 07 07:31:49 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:31:49 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:49 figaro kernel:         #2: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:31:49 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:49 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:49 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:31:49 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:31:49 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:31:49 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:31:49 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:31:49 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:31:49 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:31:49 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:31:49 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:31:49 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:31:49 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:31:49 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:31:49 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:31:49 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:31:49 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:31:49 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:31:49 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:31:49 figaro kernel: PKRU: 55555554
Jun 07 07:31:49 figaro kernel: Call Trace:
Jun 07 07:31:49 figaro kernel:  <TASK>
Jun 07 07:31:49 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:31:49 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:31:49 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:31:49 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:31:49 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:31:49 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:31:49 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:31:49 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:49 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:31:49 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:31:49 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:31:49 figaro kernel:  </TASK>
Jun 07 07:32:05 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 126s! [kworker/u64:4:237372]
Jun 07 07:32:05 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:32:05 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:05 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:05 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:05 figaro kernel:         #4: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:32:05 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:05 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:32:05 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:32:05 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:32:05 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:32:05 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:32:05 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:32:05 figaro kernel: RIP: 0010:__synchronize_hardirq+0x35/0xc0
Jun 07 07:32:05 figaro kernel: Code: 55 48 8d af a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24>
Jun 07 07:32:05 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:32:05 figaro kernel: RAX: 0000000019458200 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:32:05 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:32:05 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:32:05 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:32:05 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:32:05 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:32:05 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:32:05 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:32:05 figaro kernel: PKRU: 55555554
Jun 07 07:32:05 figaro kernel: Call Trace:
Jun 07 07:32:05 figaro kernel:  <TASK>
Jun 07 07:32:05 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:32:05 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:05 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:05 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:05 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:05 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:32:05 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:05 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:05 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:05 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:05 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:05 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:05 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:05 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:05 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task khugepaged:125 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:khugepaged      state:D stack:0     pid:125   tgid:125   ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? sched_clock_cpu+0xf/0x200
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? psi_group_change+0x129/0x300
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  wait_for_completion+0x85/0x170
Jun 07 07:32:11 figaro kernel:  __flush_work+0x26b/0x360
Jun 07 07:32:11 figaro kernel:  ? __pfx_wq_barrier_func+0x10/0x10
Jun 07 07:32:11 figaro kernel:  __lru_add_drain_all+0x146/0x1f0
Jun 07 07:32:11 figaro kernel:  khugepaged+0x66/0x930
Jun 07 07:32:11 figaro kernel:  ? __pfx_autoremove_wake_function+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ? __pfx_khugepaged+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/u65:1:162623 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/u65:1   state:D stack:0     pid:162623 tgid:162623 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: ttm ttm_bo_delayed_delete [ttm]
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? get_nohz_timer_target+0x2f/0x140
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? timerqueue_add+0x73/0xd0
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  dma_fence_default_wait+0x184/0x230
Jun 07 07:32:11 figaro kernel:  ? __pfx_dma_fence_default_wait_cb+0x10/0x10
Jun 07 07:32:11 figaro kernel:  dma_fence_wait_timeout+0x109/0x140
Jun 07 07:32:11 figaro kernel:  dma_resv_wait_timeout+0xc8/0x1c0
Jun 07 07:32:11 figaro kernel:  ttm_bo_delayed_delete+0x2a/0x80 [ttm 87922f8d934d495e42e89953731509bafa445a1c]
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/u65:2:162624 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/u65:2   state:D stack:0     pid:162624 tgid:162624 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: ttm ttm_bo_delayed_delete [ttm]
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? get_nohz_timer_target+0x2f/0x140
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? timerqueue_add+0x73/0xd0
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  dma_fence_default_wait+0x184/0x230
Jun 07 07:32:11 figaro kernel:  ? __pfx_dma_fence_default_wait_cb+0x10/0x10
Jun 07 07:32:11 figaro kernel:  dma_fence_wait_timeout+0x109/0x140
Jun 07 07:32:11 figaro kernel:  dma_resv_wait_timeout+0xc8/0x1c0
Jun 07 07:32:11 figaro kernel:  ttm_bo_delayed_delete+0x2a/0x80 [ttm 87922f8d934d495e42e89953731509bafa445a1c]
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/u64:0:235688 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/u64:0   state:D stack:0     pid:235688 tgid:235688 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: events_unbound cfg80211_wiphy_work [cfg80211]
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  wait_for_completion_state+0x134/0x200
Jun 07 07:32:11 figaro kernel:  __wait_rcu_gp+0x18a/0x190
Jun 07 07:32:11 figaro kernel:  synchronize_rcu_normal.part.0+0x45/0x70
Jun 07 07:32:11 figaro kernel:  ? __pfx_call_rcu_hurry+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ? __pfx_wakeme_after_rcu+0x10/0x10
Jun 07 07:32:11 figaro kernel:  synchronize_rcu_normal+0xaa/0xc0
Jun 07 07:32:11 figaro kernel:  __ieee80211_scan_completed+0xae/0x330 [mac80211 ea7cb372f8529ba85a93ed0072144042d2835207]
Jun 07 07:32:11 figaro kernel:  ieee80211_scan_work+0x13b/0x620 [mac80211 ea7cb372f8529ba85a93ed0072144042d2835207]
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? kmem_cache_free+0x411/0x4c0
Jun 07 07:32:11 figaro kernel:  ? skb_release_data+0x185/0x1e0
Jun 07 07:32:11 figaro kernel:  ? mt7921_scan_work+0x30/0x120 [mt7921_common 8520294177306b6d8a89264a30817b3179caa0f5]
Jun 07 07:32:11 figaro kernel:  cfg80211_wiphy_work+0x106/0x180 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/6:1:238032 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/6:1     state:D stack:0     pid:238032 tgid:238032 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: events free_ipc
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  wait_for_completion_state+0x134/0x200
Jun 07 07:32:11 figaro kernel:  __wait_rcu_gp+0x18a/0x190
Jun 07 07:32:11 figaro kernel:  synchronize_rcu_normal.part.0+0x45/0x70
Jun 07 07:32:11 figaro kernel:  ? __pfx_call_rcu_hurry+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ? __pfx_wakeme_after_rcu+0x10/0x10
Jun 07 07:32:11 figaro kernel:  synchronize_rcu_normal+0xaa/0xc0
Jun 07 07:32:11 figaro kernel:  free_ipc+0x56/0x130
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/u65:3:239166 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/u65:3   state:D stack:0     pid:239166 tgid:239166 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: ttm ttm_bo_delayed_delete [ttm]
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? get_nohz_timer_target+0x2f/0x140
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? timerqueue_add+0x73/0xd0
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:32:11 figaro kernel:  dma_fence_default_wait+0x184/0x230
Jun 07 07:32:11 figaro kernel:  ? __pfx_dma_fence_default_wait_cb+0x10/0x10
Jun 07 07:32:11 figaro kernel:  dma_fence_wait_timeout+0x109/0x140
Jun 07 07:32:11 figaro kernel:  dma_resv_wait_timeout+0xc8/0x1c0
Jun 07 07:32:11 figaro kernel:  ttm_bo_delayed_delete+0x2a/0x80 [ttm 87922f8d934d495e42e89953731509bafa445a1c]
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:11 figaro kernel: INFO: task kworker/u64:7:239773 blocked for more than 122 seconds.
Jun 07 07:32:11 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:32:11 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:32:11 figaro kernel: task:kworker/u64:7   state:D stack:0     pid:239773 tgid:239773 ppid:2      flags:0x00004000
Jun 07 07:32:11 figaro kernel: Workqueue: netns cleanup_net
Jun 07 07:32:11 figaro kernel: Call Trace:
Jun 07 07:32:11 figaro kernel:  <TASK>
Jun 07 07:32:11 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:32:11 figaro kernel:  ? ttwu_queue_wakelist+0xf9/0x110
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  ? try_to_wake_up+0x2fc/0x710
Jun 07 07:32:11 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:32:11 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:32:11 figaro kernel:  synchronize_rcu_expedited+0x1b8/0x210
Jun 07 07:32:11 figaro kernel:  ? __pfx_autoremove_wake_function+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:32:11 figaro kernel:  cleanup_net+0x223/0x490
Jun 07 07:32:11 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:11 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:11 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:11 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:11 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:11 figaro kernel:  </TASK>
Jun 07 07:32:17 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 127s! [rcu_exp_gp_kthr:22]
Jun 07 07:32:17 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:32:17 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:17 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:17 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:17 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:17 figaro kernel:         #5: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:32:17 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:32:17 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:32:17 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:32:17 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:32:17 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:32:17 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:32:17 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:32:17 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:32:17 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:32:17 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:32:17 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:32:17 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:32:17 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:32:17 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:32:17 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:32:17 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:32:17 figaro kernel: PKRU: 55555554
Jun 07 07:32:17 figaro kernel: Call Trace:
Jun 07 07:32:17 figaro kernel:  <TASK>
Jun 07 07:32:17 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:32:17 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:32:17 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:32:17 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:32:17 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:32:17 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:32:17 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:17 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:17 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:17 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:17 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:17 figaro kernel:  </TASK>
Jun 07 07:32:33 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 152s! [kworker/u64:4:237372]
Jun 07 07:32:33 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:32:33 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:33 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:33 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:33 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:33 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:33 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:32:33 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:32:33 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:32:33 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:32:33 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:32:33 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:32:33 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:32:33 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:32:33 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:32:33 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:32:33 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:32:33 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:32:33 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:32:33 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:32:33 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:32:33 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:32:33 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:32:33 figaro kernel: PKRU: 55555554
Jun 07 07:32:33 figaro kernel: Call Trace:
Jun 07 07:32:33 figaro kernel:  <TASK>
Jun 07 07:32:33 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:32:33 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:33 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:33 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:33 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:32:33 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:32:33 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:32:33 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:32:33 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:32:33 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:33 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:33 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:33 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:33 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:33 figaro kernel:  </TASK>
Jun 07 07:32:45 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 153s! [rcu_exp_gp_kthr:22]
Jun 07 07:32:45 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:32:45 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:45 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:45 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:45 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:45 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:32:45 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:32:45 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:32:45 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:32:45 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:32:45 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:32:45 figaro kernel: RIP: 0010:smp_call_function_single+0xe9/0x140
Jun 07 07:32:45 figaro kernel: Code: 38 65 48 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08>
Jun 07 07:32:45 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:32:45 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:32:45 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:32:45 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:32:45 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:32:45 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:32:45 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:32:45 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:32:45 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:32:45 figaro kernel: PKRU: 55555554
Jun 07 07:32:45 figaro kernel: Call Trace:
Jun 07 07:32:45 figaro kernel:  <TASK>
Jun 07 07:32:45 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:32:45 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:32:45 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:32:45 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:32:45 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:32:45 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:32:45 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:32:45 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:45 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:32:45 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:32:45 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:32:45 figaro kernel:  </TASK>
Jun 07 07:33:01 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 178s! [kworker/u64:4:237372]
Jun 07 07:33:01 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:33:01 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:01 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:01 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:01 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:01 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:01 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:33:01 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:33:01 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:33:01 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:01 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:01 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:33:01 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:33:01 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:33:01 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:33:01 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:33:01 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:33:01 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:33:01 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:33:01 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:33:01 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:33:01 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:01 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:33:01 figaro kernel: PKRU: 55555554
Jun 07 07:33:01 figaro kernel: Call Trace:
Jun 07 07:33:01 figaro kernel:  <TASK>
Jun 07 07:33:01 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:33:01 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:01 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:01 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:01 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:01 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:33:01 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:33:01 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:33:01 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:33:01 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:33:01 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:01 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:33:01 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:01 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:33:01 figaro kernel:  </TASK>
Jun 07 07:33:13 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 179s! [rcu_exp_gp_kthr:22]
Jun 07 07:33:13 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:33:13 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:13 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:13 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:13 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:13 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:13 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:33:13 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:33:13 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:33:13 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:13 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:13 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:33:13 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:33:13 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:33:13 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:33:13 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:33:13 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:33:13 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:33:13 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:33:13 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:33:13 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:13 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:33:13 figaro kernel: PKRU: 55555554
Jun 07 07:33:13 figaro kernel: Call Trace:
Jun 07 07:33:13 figaro kernel:  <TASK>
Jun 07 07:33:13 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:33:13 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:33:13 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:33:13 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:33:13 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:33:13 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:33:13 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:33:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:13 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:33:13 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:13 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:33:13 figaro kernel:  </TASK>
Jun 07 07:33:29 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 204s! [kworker/u64:4:237372]
Jun 07 07:33:29 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:33:29 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:29 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:29 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:29 figaro kernel:         #4: 100% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:33:29 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:29 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:33:29 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:33:29 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:33:29 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:29 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:29 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:33:29 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:33:29 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:33:29 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:33:29 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:33:29 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:33:29 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:33:29 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:33:29 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:33:29 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:33:29 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:29 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:33:29 figaro kernel: PKRU: 55555554
Jun 07 07:33:29 figaro kernel: Call Trace:
Jun 07 07:33:29 figaro kernel:  <TASK>
Jun 07 07:33:29 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:33:29 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:29 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:29 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:29 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:29 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:33:29 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:33:29 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:33:29 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:33:29 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:33:29 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:29 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:33:29 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:29 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:33:29 figaro kernel:  </TASK>
Jun 07 07:33:38 figaro kernel: rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
Jun 07 07:33:38 figaro kernel: rcu:         3-...0: (8 GPs behind) idle=e88c/1/0x4000000000000002 softirq=2895029/2895029 fqs=21842
Jun 07 07:33:38 figaro kernel: rcu:         (detected by 4, t=72007 jiffies, g=6414797, q=29180 ncpus=16)
Jun 07 07:33:38 figaro kernel: Sending NMI from CPU 4 to CPUs 3:
Jun 07 07:33:38 figaro kernel: NMI backtrace for cpu 3
Jun 07 07:33:38 figaro kernel: CPU: 3 UID: 0 PID: 0 Comm: swapper/3 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be609de09b67>
Jun 07 07:33:38 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:38 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:38 figaro kernel: RIP: 0010:delay_halt_mwaitx+0x3c/0x50
Jun 07 07:33:38 figaro kernel: Code: 31 d2 48 89 d1 48 05 00 60 00 00 0f 01 fa b8 ff ff ff ff b9 02 00 00 00 48 39 c6 48 0f 47 f0 b8 f0 00 00 00>
Jun 07 07:33:38 figaro kernel: RSP: 0018:ffffd12e802ccc78 EFLAGS: 00000093
Jun 07 07:33:38 figaro kernel: RAX: 00000000000000f0 RBX: 0000000000000ed2 RCX: 0000000000000002
Jun 07 07:33:38 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000ed2 RDI: 0001ad50be5d8714
Jun 07 07:33:38 figaro kernel: RBP: 0001ad50be5d8714 R08: 0000000000002000 R09: 0000000000001e40
Jun 07 07:33:38 figaro kernel: R10: ffffd12ea10c6d00 R11: ffff8e0e8ae90c00 R12: 00000000000186a0
Jun 07 07:33:38 figaro kernel: R13: ffffd12e802ccd10 R14: 0000000000000000 R15: 0000000000000001
Jun 07 07:33:38 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6eb80000(0000) knlGS:0000000000000000
Jun 07 07:33:38 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:38 figaro kernel: CR2: 00007685e7c55000 CR3: 0000000134dd0000 CR4: 0000000000f50ef0
Jun 07 07:33:38 figaro kernel: PKRU: 55555554
Jun 07 07:33:38 figaro kernel: Call Trace:
Jun 07 07:33:38 figaro kernel:  <IRQ>
Jun 07 07:33:38 figaro kernel:  delay_halt+0x3f/0x70
Jun 07 07:33:38 figaro kernel:  dmub_srv_wait_for_idle+0x55/0x90 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  dc_dmub_srv_cmd_run_list+0x7d/0x170 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  dc_dmub_srv_drr_update_cmd+0x99/0x110 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  optc1_set_drr+0x59/0x1b0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:33:38 figaro kernel:  ? __pfx_optc1_set_drr+0x10/0x10 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  set_drr_and_clear_adjust_pending+0x3c/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  dcn10_set_drr+0x82/0xe0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  dc_stream_adjust_vmin_vmax+0x147/0x250 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  dm_crtc_high_irq+0x2b4/0x320 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  amdgpu_dm_irq_handler+0x85/0x1f0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  amdgpu_irq_dispatch+0xd2/0x230 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  amdgpu_ih_process+0x80/0x100 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  amdgpu_irq_handler+0x23/0x60 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:38 figaro kernel:  __handle_irq_event_percpu+0x4a/0x190
Jun 07 07:33:38 figaro kernel:  handle_irq_event+0x38/0x80
Jun 07 07:33:38 figaro kernel:  handle_edge_irq+0x8b/0x220
Jun 07 07:33:38 figaro kernel:  __common_interrupt+0x41/0xa0
Jun 07 07:33:38 figaro kernel:  common_interrupt+0x80/0xa0
Jun 07 07:33:38 figaro kernel:  </IRQ>
Jun 07 07:33:38 figaro kernel:  <TASK>
Jun 07 07:33:38 figaro kernel:  asm_common_interrupt+0x26/0x40
Jun 07 07:33:38 figaro kernel: RIP: 0010:cpuidle_enter_state+0xbb/0x410
Jun 07 07:33:38 figaro kernel: Code: 00 00 e8 98 62 26 ff e8 83 f1 ff ff 48 89 c5 0f 1f 44 00 00 31 ff e8 94 f2 24 ff 45 84 ff 0f 85 a6 01 00 00>
Jun 07 07:33:38 figaro kernel: RSP: 0018:ffffd12e801c7e78 EFLAGS: 00000246
Jun 07 07:33:38 figaro kernel: RAX: ffff8e1d6eb80000 RBX: 0000000000000002 RCX: 0000000000000000
Jun 07 07:33:38 figaro kernel: RDX: 000070f5c31ba958 RSI: fffffffd0804d3c6 RDI: 0000000000000000
Jun 07 07:33:38 figaro kernel: RBP: 000070f5c31ba958 R08: 0000000000000002 R09: ffff8e1d6ebb51c4
Jun 07 07:33:38 figaro kernel: R10: 000000000000485d R11: ffff8e1d6ebb51a4 R12: ffff8e0e80e6dc00
Jun 07 07:33:38 figaro kernel: R13: ffffffffb25c7e20 R14: 0000000000000002 R15: 0000000000000000
Jun 07 07:33:38 figaro kernel:  ? cpuidle_enter_state+0xac/0x410
Jun 07 07:33:38 figaro kernel:  cpuidle_enter+0x31/0x50
Jun 07 07:33:38 figaro kernel:  do_idle+0x1b6/0x210
Jun 07 07:33:38 figaro kernel:  cpu_startup_entry+0x29/0x30
Jun 07 07:33:38 figaro kernel:  start_secondary+0x11e/0x140
Jun 07 07:33:38 figaro kernel:  common_startup_64+0x13e/0x141
Jun 07 07:33:38 figaro kernel:  </TASK>
Jun 07 07:33:41 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 205s! [rcu_exp_gp_kthr:22]
Jun 07 07:33:41 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:33:41 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:41 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:41 figaro kernel:         #3: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:33:41 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:41 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:41 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:33:41 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:33:41 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:33:41 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:41 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:41 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:33:41 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:33:41 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:33:41 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:33:41 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:33:41 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:33:41 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:33:41 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:33:41 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:33:41 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:41 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:33:41 figaro kernel: PKRU: 55555554
Jun 07 07:33:41 figaro kernel: Call Trace:
Jun 07 07:33:41 figaro kernel:  <TASK>
Jun 07 07:33:41 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:33:41 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:33:41 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:33:41 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:33:41 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:33:41 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:33:41 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:33:41 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:41 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:33:41 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:41 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:33:41 figaro kernel:  </TASK>
Jun 07 07:33:57 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 230s! [kworker/u64:4:237372]
Jun 07 07:33:57 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:33:57 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:57 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:57 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:57 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:57 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:33:57 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:33:57 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:33:57 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:33:57 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:33:57 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:33:57 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:33:57 figaro kernel: RIP: 0010:__synchronize_hardirq+0x35/0xc0
Jun 07 07:33:57 figaro kernel: Code: 55 48 8d af a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24>
Jun 07 07:33:57 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:33:57 figaro kernel: RAX: 0000000019458200 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:33:57 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:33:57 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:33:57 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:33:57 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:33:57 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:33:57 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:33:57 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:33:57 figaro kernel: PKRU: 55555554
Jun 07 07:33:57 figaro kernel: Call Trace:
Jun 07 07:33:57 figaro kernel:  <TASK>
Jun 07 07:33:57 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:33:57 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:57 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:57 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:57 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:33:57 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:33:57 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:33:57 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:33:57 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:33:57 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:33:57 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:57 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:33:57 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:33:57 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:33:57 figaro kernel:  </TASK>
Jun 07 07:34:09 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 231s! [rcu_exp_gp_kthr:22]
Jun 07 07:34:09 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:34:09 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:09 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:09 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:09 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:09 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:09 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:34:09 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:34:09 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:34:09 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:34:09 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:34:09 figaro kernel: RIP: 0010:smp_call_function_single+0xe9/0x140
Jun 07 07:34:09 figaro kernel: Code: 38 65 48 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08>
Jun 07 07:34:09 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:34:09 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:34:09 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:34:09 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:34:09 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:34:09 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:34:09 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:34:09 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:34:09 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:34:09 figaro kernel: PKRU: 55555554
Jun 07 07:34:09 figaro kernel: Call Trace:
Jun 07 07:34:09 figaro kernel:  <TASK>
Jun 07 07:34:09 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:34:09 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:34:09 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:34:09 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:34:09 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:34:09 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:34:09 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:09 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:09 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:09 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:09 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:09 figaro kernel:  </TASK>
Jun 07 07:34:14 figaro kernel: INFO: task khugepaged:125 blocked for more than 245 seconds.
Jun 07 07:34:14 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:34:14 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:34:14 figaro kernel: task:khugepaged      state:D stack:0     pid:125   tgid:125   ppid:2      flags:0x00004000
Jun 07 07:34:14 figaro kernel: Call Trace:
Jun 07 07:34:14 figaro kernel:  <TASK>
Jun 07 07:34:14 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? sched_clock_cpu+0xf/0x200
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? psi_group_change+0x129/0x300
Jun 07 07:34:14 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:34:14 figaro kernel:  schedule_timeout+0x133/0x170
Jun 07 07:34:14 figaro kernel:  wait_for_completion+0x85/0x170
Jun 07 07:34:14 figaro kernel:  __flush_work+0x26b/0x360
Jun 07 07:34:14 figaro kernel:  ? __pfx_wq_barrier_func+0x10/0x10
Jun 07 07:34:14 figaro kernel:  __lru_add_drain_all+0x146/0x1f0
Jun 07 07:34:14 figaro kernel:  khugepaged+0x66/0x930
Jun 07 07:34:14 figaro kernel:  ? __pfx_autoremove_wake_function+0x10/0x10
Jun 07 07:34:14 figaro kernel:  ? __pfx_khugepaged+0x10/0x10
Jun 07 07:34:14 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:14 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:14 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:14 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:14 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:14 figaro kernel:  </TASK>
Jun 07 07:34:14 figaro kernel: INFO: task NetworkManager:1234 blocked for more than 122 seconds.
Jun 07 07:34:14 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:34:14 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:34:14 figaro kernel: task:NetworkManager  state:D stack:0     pid:1234  tgid:1234  ppid:1      flags:0x00000002
Jun 07 07:34:14 figaro kernel: Call Trace:
Jun 07 07:34:14 figaro kernel:  <TASK>
Jun 07 07:34:14 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:34:14 figaro kernel:  ? poll_freewait+0x80/0xa0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_sys_poll+0x4d7/0x600
Jun 07 07:34:14 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:34:14 figaro kernel:  schedule_preempt_disabled+0x15/0x30
Jun 07 07:34:14 figaro kernel:  __mutex_lock.constprop.0+0x31c/0x620
Jun 07 07:34:14 figaro kernel:  ? security_capable+0x7d/0x1a0
Jun 07 07:34:14 figaro kernel:  rtnetlink_rcv_msg+0xf8/0x3f0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? __ep_eventpoll_poll.isra.0+0x14c/0x180
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? __skb_try_recv_datagram+0xa4/0x190
Jun 07 07:34:14 figaro kernel:  ? __pfx_rtnetlink_rcv_msg+0x10/0x10
Jun 07 07:34:14 figaro kernel:  netlink_rcv_skb+0x5d/0x110
Jun 07 07:34:14 figaro kernel:  netlink_unicast+0x23c/0x370
Jun 07 07:34:14 figaro kernel:  netlink_sendmsg+0x1fc/0x430
Jun 07 07:34:14 figaro kernel:  ____sys_sendmsg+0x3d4/0x410
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ___sys_sendmsg+0x9a/0xe0
Jun 07 07:34:14 figaro kernel:  __sys_sendmsg+0x80/0xd0
Jun 07 07:34:14 figaro kernel:  do_syscall_64+0x7b/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? syscall_exit_to_user_mode+0x37/0x1c0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? __x64_sys_poll+0xc6/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? syscall_exit_to_user_mode+0x37/0x1c0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? syscall_exit_to_user_mode+0x37/0x1c0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? __x64_sys_poll+0xc6/0x190
Jun 07 07:34:14 figaro kernel:  ? __x64_sys_epoll_wait+0x71/0x120
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? syscall_exit_to_user_mode+0x37/0x1c0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? irqentry_exit_to_user_mode+0x2c/0x1b0
Jun 07 07:34:14 figaro kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Jun 07 07:34:14 figaro kernel: RIP: 0033:0x7b1f42242e22
Jun 07 07:34:14 figaro kernel: RSP: 002b:00007fff8081cb68 EFLAGS: 00000246 ORIG_RAX: 000000000000002e
Jun 07 07:34:14 figaro kernel: RAX: ffffffffffffffda RBX: 0000000000000001 RCX: 00007b1f42242e22
Jun 07 07:34:14 figaro kernel: RDX: 0000000000000000 RSI: 00007fff8081cc00 RDI: 000000000000000d
Jun 07 07:34:14 figaro kernel: RBP: 00007fff8081cba0 R08: 0000000000000000 R09: 0000000000000000
Jun 07 07:34:14 figaro kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000621a017d67a0
Jun 07 07:34:14 figaro kernel: R13: 0000000000000253 R14: 00007fff8081cd94 R15: 0000000000000000
Jun 07 07:34:14 figaro kernel:  </TASK>
Jun 07 07:34:14 figaro kernel: INFO: task wpa_supplicant:1334 blocked for more than 122 seconds.
Jun 07 07:34:14 figaro kernel:       Tainted: P           OEL     6.12.31-1-lts #1
Jun 07 07:34:14 figaro kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun 07 07:34:14 figaro kernel: task:wpa_supplicant  state:D stack:0     pid:1334  tgid:1334  ppid:1      flags:0x00000002
Jun 07 07:34:14 figaro kernel: Call Trace:
Jun 07 07:34:14 figaro kernel:  <TASK>
Jun 07 07:34:14 figaro kernel:  __schedule+0x3c7/0x12f0
Jun 07 07:34:14 figaro kernel:  schedule+0x27/0xf0
Jun 07 07:34:14 figaro kernel:  schedule_preempt_disabled+0x15/0x30
Jun 07 07:34:14 figaro kernel:  __mutex_lock.constprop.0+0x31c/0x620
Jun 07 07:34:14 figaro kernel:  ? __cfg80211_wdev_from_attrs+0x16e/0x360 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:34:14 figaro kernel:  ? __nla_parse+0x22/0x30
Jun 07 07:34:14 figaro kernel:  nl80211_pre_doit+0x242/0x270 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:34:14 figaro kernel:  genl_family_rcv_msg_doit+0xdc/0x140
Jun 07 07:34:14 figaro kernel:  genl_rcv_msg+0x1ac/0x2b0
Jun 07 07:34:14 figaro kernel:  ? __pfx_nl80211_pre_doit+0x10/0x10 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:34:14 figaro kernel:  ? __pfx_nl80211_abort_scan+0x10/0x10 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:34:14 figaro kernel:  ? __pfx_nl80211_post_doit+0x10/0x10 [cfg80211 624ed8292861a3e899d1410f86f8e3ec0e79f79d]
Jun 07 07:34:14 figaro kernel:  ? __pfx_genl_rcv_msg+0x10/0x10
Jun 07 07:34:14 figaro kernel:  netlink_rcv_skb+0x5d/0x110
Jun 07 07:34:14 figaro kernel:  genl_rcv+0x28/0x40
Jun 07 07:34:14 figaro kernel:  netlink_unicast+0x23c/0x370
Jun 07 07:34:14 figaro kernel:  netlink_sendmsg+0x1fc/0x430
Jun 07 07:34:14 figaro kernel:  ____sys_sendmsg+0x3d4/0x410
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ___sys_sendmsg+0x9a/0xe0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  __sys_sendmsg+0x80/0xd0
Jun 07 07:34:14 figaro kernel:  do_syscall_64+0x7b/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? syscall_exit_to_user_mode+0x37/0x1c0
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? do_syscall_64+0x87/0x190
Jun 07 07:34:14 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:14 figaro kernel:  ? irqentry_exit_to_user_mode+0x2c/0x1b0
Jun 07 07:34:14 figaro kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Jun 07 07:34:14 figaro kernel: RIP: 0033:0x71993c2a2006
Jun 07 07:34:14 figaro kernel: RSP: 002b:00007ffd295fd610 EFLAGS: 00000202 ORIG_RAX: 000000000000002e
Jun 07 07:34:14 figaro kernel: RAX: ffffffffffffffda RBX: 00005e6d2779b520 RCX: 000071993c2a2006
Jun 07 07:34:14 figaro kernel: RDX: 0000000000000000 RSI: 00007ffd295fd6b0 RDI: 0000000000000006
Jun 07 07:34:14 figaro kernel: RBP: 00007ffd295fd630 R08: 0000000000000000 R09: 0000000000000000
Jun 07 07:34:14 figaro kernel: R10: 0000000000000000 R11: 0000000000000202 R12: 00005e6d2787ee50
Jun 07 07:34:14 figaro kernel: R13: 00005e6d2779c7b0 R14: 00007ffd295fd6b0 R15: 0000000000000000
Jun 07 07:34:14 figaro kernel:  </TASK>
Jun 07 07:34:14 figaro kernel: Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings
Jun 07 07:34:25 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 256s! [kworker/u64:4:237372]
Jun 07 07:34:25 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:34:25 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:25 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:25 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:25 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:25 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:25 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:34:25 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:34:25 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:34:25 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:34:25 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:34:25 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:34:25 figaro kernel: RIP: 0010:__synchronize_hardirq+0x35/0xc0
Jun 07 07:34:25 figaro kernel: Code: 55 48 8d af a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24>
Jun 07 07:34:25 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:34:25 figaro kernel: RAX: 0000000019458200 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:34:25 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:34:25 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:34:25 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:34:25 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:34:25 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:34:25 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:34:25 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:34:25 figaro kernel: PKRU: 55555554
Jun 07 07:34:25 figaro kernel: Call Trace:
Jun 07 07:34:25 figaro kernel:  <TASK>
Jun 07 07:34:25 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:34:25 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:25 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:25 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:25 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:25 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:34:25 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:34:25 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:34:25 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:34:25 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:25 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:25 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:25 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:25 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:25 figaro kernel:  </TASK>
Jun 07 07:34:37 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 257s! [rcu_exp_gp_kthr:22]
Jun 07 07:34:37 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:34:37 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:37 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:37 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:37 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:37 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:37 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:34:37 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:34:37 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:34:37 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:34:37 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:34:37 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:34:37 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:34:37 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:34:37 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:34:37 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:34:37 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:34:37 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:34:37 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:34:37 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:34:37 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:34:37 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:34:37 figaro kernel: PKRU: 55555554
Jun 07 07:34:37 figaro kernel: Call Trace:
Jun 07 07:34:37 figaro kernel:  <TASK>
Jun 07 07:34:37 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:34:37 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:34:37 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:34:37 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:34:37 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:34:37 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:34:37 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:37 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:37 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:37 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:37 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:37 figaro kernel:  </TASK>
Jun 07 07:34:53 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 282s! [kworker/u64:4:237372]
Jun 07 07:34:53 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:34:53 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:53 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:53 figaro kernel:         #3: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:53 figaro kernel:         #4: 100% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:34:53 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:34:53 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:34:53 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:34:53 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:34:53 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:34:53 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:34:53 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:34:53 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:34:53 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:34:53 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:34:53 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:34:53 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:34:53 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:34:53 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:34:53 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:34:53 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:34:53 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:34:53 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:34:53 figaro kernel: PKRU: 55555554
Jun 07 07:34:53 figaro kernel: Call Trace:
Jun 07 07:34:53 figaro kernel:  <TASK>
Jun 07 07:34:53 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:34:53 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:53 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:53 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:53 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:34:53 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:34:53 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:34:53 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:34:53 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:34:53 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:53 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:53 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:53 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:53 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:53 figaro kernel:  </TASK>
Jun 07 07:34:58 figaro kernel: ------------[ cut here ]------------
Jun 07 07:34:58 figaro kernel: WARNING: CPU: 1 PID: 239046 at io_uring/io_uring.c:2928 io_ring_exit_work+0x143/0x4ab
Jun 07 07:34:58 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:34:58 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:34:58 figaro kernel: CPU: 1 UID: 0 PID: 239046 Comm: kworker/u64:3 Tainted: P           OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:34:58 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:34:58 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:34:58 figaro kernel: Workqueue: iou_exit io_ring_exit_work
Jun 07 07:34:58 figaro kernel: RIP: 0010:io_ring_exit_work+0x143/0x4ab
Jun 07 07:34:58 figaro kernel: Code: e8 42 a1 83 ff 4c 89 ff e8 5a 23 83 ff 4c 89 e7 e8 02 e6 ff ff 31 f6 48 89 74 24 10 48 8b 05 a4 07 37 01 48>
Jun 07 07:34:58 figaro kernel: RSP: 0018:ffffd12ef766bdd0 EFLAGS: 00010293
Jun 07 07:34:58 figaro kernel: RAX: 000000010238a386 RBX: ffff8e0fe345d440 RCX: 0000000000000000
Jun 07 07:34:58 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8e0fe345d040
Jun 07 07:34:58 figaro kernel: RBP: ffffd12ef766be58 R08: ffff8e0fe345d4a8 R09: 0000000000000014
Jun 07 07:34:58 figaro kernel: R10: ffff8e1011bf8000 R11: 0000000000000001 R12: ffff8e0fe345d000
Jun 07 07:34:58 figaro kernel: R13: 000000000000000f R14: ffff8e0fe345d040 R15: 0000000000000000
Jun 07 07:34:58 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ea80000(0000) knlGS:0000000000000000
Jun 07 07:34:58 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:34:58 figaro kernel: CR2: 000077f606cc5000 CR3: 000000018282a000 CR4: 0000000000f50ef0
Jun 07 07:34:58 figaro kernel: PKRU: 55555554
Jun 07 07:34:58 figaro kernel: Call Trace:
Jun 07 07:34:58 figaro kernel:  <TASK>
Jun 07 07:34:58 figaro kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jun 07 07:34:58 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:34:58 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:34:58 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:34:58 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:34:58 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:58 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:34:58 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:34:58 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:34:58 figaro kernel:  </TASK>
Jun 07 07:34:58 figaro kernel: ---[ end trace 0000000000000000 ]---
Jun 07 07:35:05 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 283s! [rcu_exp_gp_kthr:22]
Jun 07 07:35:05 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:35:05 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:05 figaro kernel:         #2: 100% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:35:05 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:05 figaro kernel:         #4: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:05 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:05 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:35:05 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:35:05 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P        W  OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:35:05 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:35:05 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:35:05 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:35:05 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:35:05 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:35:05 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:35:05 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:35:05 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:35:05 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:35:05 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:35:05 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:35:05 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:35:05 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:35:05 figaro kernel: PKRU: 55555554
Jun 07 07:35:05 figaro kernel: Call Trace:
Jun 07 07:35:05 figaro kernel:  <TASK>
Jun 07 07:35:05 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:35:05 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:35:05 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:35:05 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:35:05 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:35:05 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:35:05 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:35:05 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:05 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:35:05 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:05 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:35:05 figaro kernel:  </TASK>
Jun 07 07:35:21 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 308s! [kworker/u64:4:237372]
Jun 07 07:35:21 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:35:21 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:21 figaro kernel:         #2: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:21 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:21 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:21 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:21 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:35:21 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:35:21 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P        W  OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:35:21 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:35:21 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:35:21 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:35:21 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:35:21 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:35:21 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:35:21 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:35:21 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:35:21 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:35:21 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:35:21 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:35:21 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:35:21 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:35:21 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:35:21 figaro kernel: PKRU: 55555554
Jun 07 07:35:21 figaro kernel: Call Trace:
Jun 07 07:35:21 figaro kernel:  <TASK>
Jun 07 07:35:21 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:35:21 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:21 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:21 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:21 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:21 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:35:21 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:35:21 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:35:21 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:35:21 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:35:21 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:21 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:35:21 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:21 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:35:21 figaro kernel:  </TASK>
Jun 07 07:35:33 figaro kernel: watchdog: BUG: soft lockup - CPU#13 stuck for 309s! [rcu_exp_gp_kthr:22]
Jun 07 07:35:33 figaro kernel: CPU#13 Utilization every 4s during lockup:
Jun 07 07:35:33 figaro kernel:         #1: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:33 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:33 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:33 figaro kernel:         #4: 101% system,          0% softirq,          1% hardirq,          0% idle
Jun 07 07:35:33 figaro kernel:         #5: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:33 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:35:33 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:35:33 figaro kernel: CPU: 13 UID: 0 PID: 22 Comm: rcu_exp_gp_kthr Tainted: P        W  OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be60>
Jun 07 07:35:33 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:35:33 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:35:33 figaro kernel: RIP: 0010:smp_call_function_single+0xec/0x140
Jun 07 07:35:33 figaro kernel: Code: 2b 14 25 28 00 00 00 75 66 c9 e9 2a d5 f2 00 65 48 8b 05 2f 2e c4 4f 48 8d b0 40 81 03 00 8b 46 08 a8 01 74>
Jun 07 07:35:33 figaro kernel: RSP: 0018:ffffd12e8019fde0 EFLAGS: 00000202
Jun 07 07:35:33 figaro kernel: RAX: 0000000000000001 RBX: 0000000000000005 RCX: 0000000000000000
Jun 07 07:35:33 figaro kernel: RDX: 0000000000000000 RSI: ffff8e1d6f0b8140 RDI: 0000000000000005
Jun 07 07:35:33 figaro kernel: RBP: ffffd12e8019fe28 R08: ffffffffb0390060 R09: 0000000000000292
Jun 07 07:35:33 figaro kernel: R10: ffff8e0e80b0b600 R11: 0000000000008000 R12: 0000000000031560
Jun 07 07:35:33 figaro kernel: R13: ffff8e1d6ecb7d80 R14: 000000000000ab17 R15: 0000000000000020
Jun 07 07:35:33 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6f080000(0000) knlGS:0000000000000000
Jun 07 07:35:33 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:35:33 figaro kernel: CR2: 00007685afb29000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:35:33 figaro kernel: PKRU: 55555554
Jun 07 07:35:33 figaro kernel: Call Trace:
Jun 07 07:35:33 figaro kernel:  <TASK>
Jun 07 07:35:33 figaro kernel:  __sync_rcu_exp_select_node_cpus+0x21a/0x370
Jun 07 07:35:33 figaro kernel:  ? __pfx_wait_rcu_exp_gp+0x10/0x10
Jun 07 07:35:33 figaro kernel:  sync_rcu_exp_select_cpus+0x160/0x2d0
Jun 07 07:35:33 figaro kernel:  wait_rcu_exp_gp+0x13/0x20
Jun 07 07:35:33 figaro kernel:  kthread_worker_fn+0xa0/0x230
Jun 07 07:35:33 figaro kernel:  ? __pfx_kthread_worker_fn+0x10/0x10
Jun 07 07:35:33 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:35:33 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:33 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:35:33 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:33 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:35:33 figaro kernel:  </TASK>
Jun 07 07:35:49 figaro kernel: watchdog: BUG: soft lockup - CPU#7 stuck for 334s! [kworker/u64:4:237372]
Jun 07 07:35:49 figaro kernel: CPU#7 Utilization every 4s during lockup:
Jun 07 07:35:49 figaro kernel:         #1: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:49 figaro kernel:         #2: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:49 figaro kernel:         #3: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:49 figaro kernel:         #4: 100% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:49 figaro kernel:         #5: 101% system,          0% softirq,          0% hardirq,          0% idle
Jun 07 07:35:49 figaro kernel: Modules linked in: rpcrdma rdma_cm iw_cm ib_cm ib_core snd_seq_dummy rfcomm snd_hrtimer snd_seq vfat fat cmac alg>
Jun 07 07:35:49 figaro kernel:  gpio_generic mac_hid vmmon(OE) vmw_vmci pkcs8_key_parser sg crypto_user loop nfnetlink bpf_preload ip_tables x_t>
Jun 07 07:35:49 figaro kernel: CPU: 7 UID: 0 PID: 237372 Comm: kworker/u64:4 Tainted: P        W  OEL     6.12.31-1-lts #1 e7c67df8a8ca221e76be6>
Jun 07 07:35:49 figaro kernel: Tainted: [P]=PROPRIETARY_MODULE, [W]=WARN, [O]=OOT_MODULE, [E]=UNSIGNED_MODULE, [L]=SOFTLOCKUP
Jun 07 07:35:49 figaro kernel: Hardware name: ASUS System Product Name/ROG STRIX B550-F GAMING WIFI II, BIOS 3611 09/29/2024
Jun 07 07:35:49 figaro kernel: Workqueue: amdgpu-reset-dev drm_sched_job_timedout [gpu_sched]
Jun 07 07:35:49 figaro kernel: RIP: 0010:__synchronize_hardirq+0x39/0xc0
Jun 07 07:35:49 figaro kernel: Code: a4 00 00 00 53 48 89 fb 48 83 ec 10 65 4c 8b 24 25 28 00 00 00 4c 89 64 24 08 41 89 f4 c6 44 24 07 00 eb 02>
Jun 07 07:35:49 figaro kernel: RSP: 0018:ffffd12e9d1bfc18 EFLAGS: 00000206
Jun 07 07:35:49 figaro kernel: RAX: ffff8e0e894a5000 RBX: ffff8e0e894a5000 RCX: ffff8e0e8dc1e208
Jun 07 07:35:49 figaro kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8e0e894a5000
Jun 07 07:35:49 figaro kernel: RBP: ffff8e0e894a50a4 R08: ffff8e0e8dc1e200 R09: 0000000000000000
Jun 07 07:35:49 figaro kernel: R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000001
Jun 07 07:35:49 figaro kernel: R13: 0000000000000001 R14: ffff8e1133274c00 R15: ffff8e0e93610220
Jun 07 07:35:49 figaro kernel: FS:  0000000000000000(0000) GS:ffff8e1d6ed80000(0000) knlGS:0000000000000000
Jun 07 07:35:49 figaro kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 07 07:35:49 figaro kernel: CR2: 000076872e267000 CR3: 0000000fae622000 CR4: 0000000000f50ef0
Jun 07 07:35:49 figaro kernel: PKRU: 55555554
Jun 07 07:35:49 figaro kernel: Call Trace:
Jun 07 07:35:49 figaro kernel:  <TASK>
Jun 07 07:35:49 figaro kernel:  __synchronize_irq+0x26/0xd0
Jun 07 07:35:49 figaro kernel:  amdgpu_fence_driver_isr_toggle+0x2a/0x70 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:49 figaro kernel:  amdgpu_device_pre_asic_reset+0x5e/0x2c0 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:49 figaro kernel:  amdgpu_device_gpu_recover.cold+0x5d4/0xc61 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:49 figaro kernel:  amdgpu_job_timedout.cold+0x2f4/0x334 [amdgpu 9edc94761b16da80c9d2bf54afc16f6f8e93988c]
Jun 07 07:35:49 figaro kernel:  drm_sched_job_timedout+0x88/0x120 [gpu_sched 1733f170d73299e354510db84fbd36a2486f4255]
Jun 07 07:35:49 figaro kernel:  process_one_work+0x17f/0x340
Jun 07 07:35:49 figaro kernel:  worker_thread+0x2d2/0x400
Jun 07 07:35:49 figaro kernel:  ? __pfx_worker_thread+0x10/0x10
Jun 07 07:35:49 figaro kernel:  kthread+0xd2/0x100
Jun 07 07:35:49 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:49 figaro kernel:  ret_from_fork+0x34/0x50
Jun 07 07:35:49 figaro kernel:  ? __pfx_kthread+0x10/0x10
Jun 07 07:35:49 figaro kernel:  ret_from_fork_asm+0x1a/0x30
Jun 07 07:35:49 figaro kernel:  </TASK>
-- Boot 88213b17bfa044a4b08b42353d26a10b --
Jun 07 07:36:22 figaro kernel: Linux version 6.12.31-1-lts (linux-lts@archlinux) (gcc (GCC) 15.1.1 20250425, GNU ld (GNU Binutils) 2.44.0) #1 SM>
Jun 07 07:36:22 figaro kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-linux-lts root=/dev/mapper/archvol-root rw loglevel=3 quiet

Offline

#4 2025-06-07 13:26:16

Lone_Wolf
Administrator
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 13,804

Re: amdgpu: GPU reset begin

The log shows a failed connection attempt coming from a hosting provider .

Is this some kind of server you administer remotely ?


ADDED
Is this an amd radeon RX 9000 card ?

If so, https://gitlab.freedesktop.org/mesa/mesa/-/issues/13312 looks relevant .

Last edited by Lone_Wolf (2025-06-07 15:26:47)


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

Online

#5 2025-06-07 16:40:29

LinuxLover471
Member
Registered: 2025-02-23
Posts: 128

Re: amdgpu: GPU reset begin

Also, always provide info about your hardware, either manually or by installing inxi and executing inxi -Fxxz and posting it.

Offline

#6 2025-06-07 22:03:39

sphere101
Member
Registered: 2017-08-04
Posts: 42

Re: amdgpu: GPU reset begin

Hey,
This is not being remotely manager, local via usb wireless keyboard and mouse.
Graphics card is a ASRock Radeon RX 6600 Challenger D 8GB GDDR6 PCI Express 4.0 Graphics Card RX6600 CLD 8G.
I appreciate the help on this matter...
Below is the rest of the hardware details..

figaro ~]# inxi -Fxxz
System:
  Kernel: 6.12.31-1-lts arch: x86_64 bits: 64 compiler: gcc v: 15.1.1
  Desktop: GNOME v: 48.2 tk: GTK v: 3.24.49 wm: gnome-shell dm: GDM
    Distro: Arch Linux
Machine:
  Type: Desktop System: ASUS product: N/A v: N/A serial: N/A
  Mobo: ASUSTeK model: ROG STRIX B550-F GAMING WIFI II v: Rev X.0x
    serial: <filter> part-nu: SKU UEFI: American Megatrends v: 3611
    date: 09/29/2024
Battery:
  Device-1: hidpp_battery_0 model: Logitech K850 Performance Wireless Keyboard
    serial: <filter> charge: 100% (should be ignored) status: discharging
  Device-2: hidpp_battery_1 model: Logitech M720 Triathlon Multi-Device
    Mouse serial: <filter> charge: 55% (should be ignored) status: discharging
CPU:
  Info: 8-core model: AMD Ryzen 7 5800XT bits: 64 type: MT MCP arch: Zen 3+
    rev: 2 cache: L1: 512 KiB L2: 4 MiB L3: 32 MiB
  Speed (MHz): avg: 550 min/max: 550/4966 boost: enabled cores: 1: 550
    2: 550 3: 550 4: 550 5: 550 6: 550 7: 550 8: 550 9: 550 10: 550 11: 550
    12: 550 13: 550 14: 550 15: 550 16: 550 bogomips: 121415
  Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3
Graphics:
  Device-1: Advanced Micro Devices [AMD/ATI] Navi 23 [Radeon RX 6600/6600
    XT/6600M] vendor: ASRock driver: amdgpu v: kernel arch: RDNA-2 pcie:
    speed: 16 GT/s lanes: 16 ports: active: HDMI-A-1 empty: DP-1, DP-2, DP-3,
    Writeback-1 bus-ID: 09:00.0 chip-ID: 1002:73ff
  Display: unspecified server: X.Org v: 24.1.6 with: Xwayland v: 24.1.6
    compositor: gnome-shell driver: dri: radeonsi gpu: amdgpu display-ID: :0
    screens: 1
  Screen-1: 0 s-res: 2560x1440 s-dpi: 96
  Monitor-1: HDMI-A-1 mapped: HDMI-1 model: Dell S2721D res: 2560x1440
    hz: 60 dpi: 110 diag: 676mm (26.6")
  API: OpenGL Message: Unable to show GL data. glxinfo is missing.
  Info: Tools: x11: xdriinfo, xdpyinfo, xprop, xrandr
Audio:
  Device-1: Advanced Micro Devices [AMD/ATI] Navi 21/23 HDMI/DP Audio
    driver: snd_hda_intel v: kernel pcie: speed: 16 GT/s lanes: 16
    bus-ID: 09:00.1 chip-ID: 1002:ab28
  Device-2: Advanced Micro Devices [AMD] Starship/Matisse HD Audio
    vendor: ASUSTeK driver: snd_hda_intel v: kernel pcie: speed: 16 GT/s
    lanes: 16 bus-ID: 0b:00.4 chip-ID: 1022:1487
  Device-3: LOUD Audio LLC CR2-X Cube
    driver: hid-generic,snd-usb-audio,usbhid type: USB rev: 1.1 speed: 12 Mb/s
    lanes: 1 bus-ID: 1-1:2 chip-ID: 0a73:0030
  API: ALSA v: k6.12.31-1-lts status: kernel-api
  Server-1: JACK v: 1.9.22 status: off
  Server-2: PipeWire v: 1.4.5 status: n/a (root, process) with:
    1: pipewire-pulse status: active 2: wireplumber status: active
Network:
  Device-1: MEDIATEK MT7922 802.11ax PCI Express Wireless Network Adapter
    vendor: Foxconn driver: mt7921e v: kernel pcie: speed: 5 GT/s lanes: 1
    bus-ID: 05:00.0 chip-ID: 14c3:0616
  IF: wlan0 state: down mac: <filter>
  Device-2: Intel Ethernet I226-V vendor: ASUSTeK driver: igc v: kernel
    pcie: speed: 5 GT/s lanes: 1 port: N/A bus-ID: 06:00.0 chip-ID: 8086:125c
  IF: enp6s0 state: up speed: 1000 Mbps duplex: full mac: <filter>
  IF-ID-1: vmnet1 state: unknown speed: N/A duplex: N/A mac: <filter>
  IF-ID-2: vmnet8 state: unknown speed: N/A duplex: N/A mac: <filter>
Bluetooth:
  Device-1: MediaTek Wireless_Device driver: btusb v: 0.8 type: USB rev: 2.1
    speed: 480 Mb/s lanes: 1 bus-ID: 1-5:3 chip-ID: 0489:e0e2
  Report: btmgmt ID: hci0 rfk-id: 0 state: up address: N/A
RAID:
  Device-1: zfshomenas type: zfs status: ONLINE level: raidz1-0 raw:
    size: 29.1 TiB free: 16.4 TiB zfs-fs: size: 21.02 TiB free: 11.79 TiB
  Components: Online: 1: sda1 2: sdb1 3: sdc1 4: sdd1
Drives:
  Local Storage: total: raw: 30.02 TiB usable: 21.93 TiB used: 9.3 TiB (42.4%)
  ID-1: /dev/nvme0n1 vendor: Western Digital model: WD BLACK SN850X 1000GB
    size: 931.51 GiB speed: 63.2 Gb/s lanes: 4 serial: <filter> temp: 39.9 C
  ID-2: /dev/sda vendor: Western Digital model: WD80EFAX-68LHPN0
    size: 7.28 TiB speed: 6.0 Gb/s serial: <filter>
  ID-3: /dev/sdb vendor: Western Digital model: WD80EMAZ-00WJTA0
    size: 7.28 TiB speed: 6.0 Gb/s serial: <filter>
  ID-4: /dev/sdc vendor: Seagate model: ST8000VN0022-2EL112 size: 7.28 TiB
    speed: 6.0 Gb/s serial: <filter>
  ID-5: /dev/sdd vendor: Western Digital model: WD80EFAX-68LHPN0
    size: 7.28 TiB speed: 6.0 Gb/s serial: <filter>
Partition:
  ID-1: / size: 97.87 GiB used: 15.54 GiB (15.9%) fs: ext4 dev: /dev/dm-0
    mapped: archvol-root
  ID-2: /home size: 294.23 GiB used: 12.34 GiB (4.2%) fs: ext4
    dev: /dev/dm-1 mapped: archvol-home
  ID-3: /usr size: 195.8 GiB used: 9.23 GiB (4.7%) fs: ext4 dev: /dev/dm-2
    mapped: archvol-usr
  ID-4: /var size: 261.38 GiB used: 25.81 GiB (9.9%) fs: ext4 dev: /dev/dm-4
    mapped: archvol-var
Swap:
  ID-1: swap-1 type: partition size: 64 GiB used: 0 KiB (0.0%) priority: -2
    dev: /dev/dm-3 mapped: archvol-swap
Sensors:
  System Temperatures: cpu: 47.8 C mobo: 36.0 C gpu: amdgpu temp: 37.0 C
    mem: 38.0 C
  Fan Speeds (rpm): N/A gpu: amdgpu fan: 0
Info:
  Memory: total: 64 GiB available: 62.71 GiB used: 4.4 GiB (7.0%)
  Processes: 519 Power: uptime: 8h 21m wakeups: 0 Init: systemd v: 257
    default: graphical
  Packages: pm: pacman pkgs: 981 Compilers: clang: 19.1.7 gcc: 15.1.1
    Shell: Bash v: 5.2.37 running-in: kgx inxi: 3.3.38

Offline

#7 2025-06-08 19:13:27

sklorpion
Member
From: Cracovia
Registered: 2013-02-03
Posts: 10

Re: amdgpu: GPU reset begin

i have same problem and no idea what to do.

[   74.032193] wlan0: Limiting TX power to 14 (17 - 3) dBm as advertised by 60:38:xxxxxxx
[   82.187600] wlan0: deauthenticating from 60:38:xxxxxxxxxxxx by local choice (Reason: 3=DEAUTH_LEAVING)
[  179.157590] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  179.429226] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  190.103135] amdgpu 0000:29:00.0: amdgpu: Dumping IP State
[  195.998016] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  195.998020] amdgpu 0000:29:00.0: amdgpu: Failed to disable gfxoff!
[  196.001476] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  196.273665] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  196.545237] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  196.816768] amdgpu 0000:29:00.0: [drm] *ERROR* dc_dmub_srv_log_diagnostic_data: DMCUB error - collecting diagnostic data
[  201.630009] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  201.630015] amdgpu 0000:29:00.0: amdgpu: Failed to disable gfxoff!
[  206.986103] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  206.986108] amdgpu 0000:29:00.0: amdgpu: Failed to disable gfxoff!
[  212.333040] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  212.333046] amdgpu 0000:29:00.0: amdgpu: Failed to disable gfxoff!
[  212.333222] amdgpu 0000:29:00.0: amdgpu: Dumping IP State Completed
[  212.617076] amdgpu 0000:29:00.0: amdgpu: ring gfx_0.0.0 timeout, signaled seq=20646, emitted seq=20648
[  212.617080] amdgpu 0000:29:00.0: amdgpu: Process information: process chrome pid 4566 thread chrome:cs0 pid 4628
[  212.899980] amdgpu 0000:29:00.0: amdgpu: GPU reset begin!
[  217.969646] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  217.969651] amdgpu 0000:29:00.0: amdgpu: Failed to disable gfxoff!
[  236.349875] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  236.349880] amdgpu 0000:29:00.0: amdgpu: Failed to disable smu features.
[  237.457842] amdgpu 0000:29:00.0: amdgpu: psp gfx command DESTROY_TMR(0x7) failed and response status is (0x80000306)
[  237.479001] amdgpu 0000:29:00.0: amdgpu: MODE1 reset
[  237.479005] amdgpu 0000:29:00.0: amdgpu: GPU mode1 reset
[  237.479098] amdgpu 0000:29:00.0: amdgpu: GPU smu mode1 reset
[  242.785289] amdgpu 0000:29:00.0: amdgpu: SMU: I'm not done with your previous command: SMN_C2PMSG_66:0x00000029 SMN_C2PMSG_82:0x00000000
[  242.785295] amdgpu 0000:29:00.0: amdgpu: GPU mode1 reset failed
[  242.785297] amdgpu 0000:29:00.0: amdgpu: ASIC reset failed with error, -62 for drm dev, 0000:29:00.0
[  242.785312] amdgpu 0000:29:00.0: amdgpu: GPU reset(2) failed
[  242.901702] snd_hda_intel 0000:29:00.1: CORB reset timeout#2, CORBRP = 65535
[  242.901713] amdgpu 0000:29:00.0: amdgpu: GPU reset end with ret = -62
[  242.901716] amdgpu 0000:29:00.0: amdgpu: GPU Recovery Failed: -62
[  252.823808] amdgpu 0000:29:00.0: amdgpu: Dumping IP State
[  252.826088] amdgpu 0000:29:00.0: amdgpu: Dumping IP State Completed
[  252.826097] amdgpu 0000:29:00.0: amdgpu: ring sdma1 timeout, signaled seq=4200, emitted seq=4203

Offline

Board footer

Powered by FluxBB