NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

Tharbad · 2024-12-22 07:43:50

seth wrote:

Had to kill plasma as it was stuck and didn't return at all.
The process responded to a standard SIGTERM though (kill default value)?

Had to use -9.

seth wrote:

https://wiki.archlinux.org/title/NVIDIA … er_suspend
Do you set a special NVreg_TemporaryFilePath ?
Do you have enough (free) RAM to use a tmpfs?

No and tmpfs is disabled. root has ~200GB free.

seth wrote:

What happens if you disable the suspend/resume services and add "nvidia.NVreg_PreserveVideoMemoryAllocations=0" to the kernel parameters?

Will try.

seth wrote:

In case of
Tried removing options nvidia NVreg_PreserveVideoMemoryAllocations=1 but things just got worse: I can login but I can't use the GUI.
please elaborate on whyt you can't use the GUI and in doubt post the xorg log reflecting such attempt.

GUI was stuck. Didn't return after ~2 min and I didn't wait any longer. Had to use systemctl reboot from TTY.

Log:
https://pastebin.com/U32jZyy4

Last edited by Tharbad (2024-12-22 09:49:08)

Tharbad · 2024-12-22 08:26:00

seth wrote:

What happens if you disable the suspend/resume services and add "nvidia.NVreg_PreserveVideoMemoryAllocations=0" to the kernel parameters?

I tried. Got the dreaded black screen of nothing. reverted settings.

seth · 2024-12-22 09:53:45

tmpfs is disabled

??

findmnt -T /tmp

Maybe actually try to save the VRAM in RAM, not on disk.
Speaking of which, there's a least one nvme and a metric shit-ton of SATA devices (sda - sdn)?
What /is/ the VRAM currently stored on (drive, raid, filesystem)?

Let's radically simply this.
Boot the multi-user.target (2nd link below) and run "startx xterm" (you can also run an openbox session for a no-nonnse WM)

Run glxgears.
Susupend and resume the system.
Does glxgears lose its GL context?

Run some GL ridden browser (chromium or firefox will do)
Susupend and resume the system.
Does the browser lose its GL context?

Also, sanity check: is this nividia or nvidia-open?

Tharbad · 2024-12-22 10:35:43

seth wrote:

tmpfs is disabled
??
findmnt -T /tmp

TARGET SOURCE         FSTYPE OPTIONS
/      /dev/nvme0n1p2 ext4   rw,relatime

seth wrote:

Maybe actually try to save the VRAM in RAM, not on disk.

Could be risky. Not enough RAM.

seth wrote:

Speaking of which, there's a least one nvme and a metric shit-ton of SATA devices (sda - sdn)?
What /is/ the VRAM currently stored on (drive, raid, filesystem)?

Most of them are external (JBOD). Manually mounted.
ext4 nvme.

seth wrote:

Let's radically simply this.
Boot the multi-user.target (2nd link below) and run "startx xterm" (you can also run an openbox session for a no-nonnse WM)
Run glxgears.
Susupend and resume the system.
Does glxgears lose its GL context?
Run some GL ridden browser (chromium or firefox will do)
Susupend and resume the system.
Does the browser lose its GL context?

Not familiar with these. LXDE is enough? It's my backup DE.

seth wrote:

Also, sanity check: is this nividia or nvidia-open?

nvidia-dkms with zen. I should've said that before.

seth · 2024-12-22 14:16:54

LXDE should do for a basic test, we just want to get the compositor and all the other GL contexts out of the way.

https://wiki.archlinux.org/title/Solid_ … leshooting - probably try to disable APST but if the nvme powers down before the VRAM has been archived and written, you're screwed.
Go for the "risk" of a tmpfs, you should have sufficent RAM on naked lxde and it's probably also not getting much worse than now…

Tharbad · 2024-12-23 15:48:48

seth wrote:

LXDE should do for a basic test, we just want to get the compositor and all the other GL contexts out of the way.
https://wiki.archlinux.org/title/Solid_ … leshooting - probably try to disable APST but if the nvme powers down before the VRAM has been archived and written, you're screwed.
Go for the "risk" of a tmpfs, you should have sufficent RAM on naked lxde and it's probably also not getting much worse than now…

Checked the last 1,000,000 lines in the logs for said nvme failures, got nothing. Will try lxde when I have time.

Tharbad · 2024-12-24 19:38:48

LXDE log:
https://pastebin.com/hrdLxgX3

Same results. discord and kitty (terminal) didn't return.
LXDE itself did return.

seth · 2024-12-24 20:12:03

It'll only affect GL contexts (which LXDE for the most part is not)

seth wrote:

Go for the "risk" of a tmpfs, you should have sufficent RAM on naked lxde and it's probably also not getting much worse than now…

Tharbad · 2024-12-25 11:31:01

seth wrote:

It'll only affect GL contexts (which LXDE for the most part is not)
seth wrote:
Go for the "risk" of a tmpfs, you should have sufficent RAM on naked lxde and it's probably also not getting much worse than now…

Will try when I've time.
Meanwhile, I'm trying to understand why it would help. Suspend was working fine til driver v555. So why would tmpfs will be the solution? Bad syncing in the new driver suspend code?

seth · 2024-12-25 14:04:58

The theory would be that the VRAM doesn't make it to the permanent nvme storage and might get lost in APST, did you use the 555xx drivers along the more recent 6.12 kernels or only w/ the 6.11 kernels?
Did you test the 560/565xx drivers w/ the LTS kernel?

Tharbad · 2024-12-25 17:40:22

seth wrote:

The theory would be that the VRAM doesn't make it to the permanent nvme storage and might get lost in APST, did you use the 555xx drivers along the more recent 6.12 kernels or only w/ the 6.11 kernels?
Did you test the 560/565xx drivers w/ the LTS kernel?

Didn't test LTS. Will try that too later this week.
only 6.11
6.12 and 565 are incompatible. mkinitcpio fails.

seth · 2024-12-26 21:52:20

https://gitlab.archlinux.org/archlinux/ … de4d1971c7 pairs 565.57.01-6 and linux 6.11.9.arch1-1
There was no 565.77 build for any 6.11 kernel, but you'll have to downgrade the nvidia OOT packages and linux in sync or use nvidia-dkms (and the linux-headers package)

Tharbad · 2024-12-29 10:57:56

seth wrote:

https://gitlab.archlinux.org/archlinux/ … de4d1971c7 pairs 565.57.01-6 and linux 6.11.9.arch1-1
There was no 565.77 build for any 6.11 kernel, but you'll have to downgrade the nvidia OOT packages and linux in sync or use nvidia-dkms (and the linux-headers package)

That's what I'm trying to avoid.

LTS test: same result. tmpfs will be done later.

Tharbad · 2025-01-02 22:10:11

tmpfs: Same result.

7000k · 2025-01-09 15:17:26

Does anyone know what combination of kernel/driver/kernel options works?

I just had a ridiculous power bill because I've been unable to suspend my pc for months. I'm just about ready to start considering windows after using Linux on the desktop exclusively since 2018.

BTW, I'm on driver 535.129.03 (supposedly lts) and kernel 6.8.4 which I somewhere read is supposed to work.

Nope, if X was used (or is being used) it will freeze will kernel panic instead of suspend. If X was never run it suspend just fine.

Last edited by 7000k (2025-01-09 15:19:33)

seth · 2025-01-09 16:56:52

The VRAM decay will cause bogus textures and maybe your GUI session to wet itself (notably if it heavily relies on opengl) but not cause "kernel panics".
There's an incredibly high chance you're facing an entirely different problem *especially* if this happens after stopping the display server and suspending from the console.

I just had a ridiculous power bill because I've been unable to suspend my pc for months.

Is this supposed to be a joke? Or are you indeed unaware of the systems ability to shut down and reboot the next morning?
And your PC is driving your power bill? In winter?

7000k · 2025-01-12 02:30:56

seth wrote:

The VRAM decay will cause bogus textures and maybe your GUI session to wet itself (notably if it heavily relies on opengl) but not cause "kernel panics".
There's an incredibly high chance you're facing an entirely different problem *especially* if this happens after stopping the display server and suspending from the console.

I tried everything in relation to suspend. It freezes (and logs a kernel crash). Not even sysrq works. Regardless if X is running or not (same messages logged. It logs a register dump and freezes. If that is not a kernel panic then fine.

The main problem is that it doesn't work if the GPU was used in any way whatsoever beyond basic console with default fonts/resolution. It behaves the same as described by countless people online. It fails to standby at last stage and freezes where only a hard reset will do anything. It does that from within Xorg too. On other versions of the driver/kernel it may succeed to standby but it never resumes. Sometimes it resumes, but fails to initialise graphics (no switching VTs only worked on some ancient kernel and driver).

For me, now, It does the same when X is running normally, on a different VT, or not running at all(but was started before). And it does the same even if gnome is not running and X was run with xterm. If I run standby after never touching X it works.

On nVidia forums no one cares.

However , you haven't replied to my question . So I have to assume either you don't know the answer or there is no combination of kernel and nvidia driver that works. If anyone knows one that would be great. Regardless if it happens to crash on resume for the op or on suspend for me.

I think Linux people who could fix it, don't do it because of "f*** nVidia" attitude and nvidia doesn't fix it because it suits them just fine for high end GPUs to be unusable on Linux desktops (they prefer people who want cuda pay for cloud compute).

So I doubt this will get solved ever. I even considered going back to windows briefly. Despite my huge dislike for it I may have to go down that path and do all my stuff in WSL under windows.

seth wrote:

I just had a ridiculous power bill because I've been unable to suspend my pc for months.
Is this supposed to be a joke? Or are you indeed unaware of the systems ability to shut down and reboot the next morning?
And your PC is driving your power bill? In winter?

One can be aware of the ability to shut the system down and find it inconvenient enough not to do it. And yes, the PC is indeed driving my bill in winter. I have a very efficient energy use in my house as well as solar/battery backup and electricity is not used for heating beyond running a 30W circulation pump.

I'll tell you more. One may even find paying a large bill still more convenient than having to shut the pc down every day and wasting an hour or more getting back into ones work. And at the same time one can hope someone finally resolves this problem (I don't mean one specific error, but the ability to reliably suspend and resume on modern Linux with a modern nVidia Card without sacrificing all the reasons why one buys these cards in the first place - cuda etc )

But I'm not very hopeful to be honest.

Do you have any more questions ?

seth · 2025-01-12 08:35:26

I tried everything in relation to suspend. It freezes (and logs a kernel crash). Not even sysrq works. Regardless if X is running or not (same messages logged. It logs a register dump and freezes. If that is not a kernel panic then fine.

I didn't say you're not experiecing a kernel panic, but that the cause for that is unlikely the VRAM decay.
Maybe actually post the kernel backtrace …

7000k · 2025-01-12 12:51:00

seth wrote:

I tried everything in relation to suspend. It freezes (and logs a kernel crash). Not even sysrq works. Regardless if X is running or not (same messages logged. It logs a register dump and freezes. If that is not a kernel panic then fine.
I didn't say you're not experiecing a kernel panic, but that the cause for that is unlikely the VRAM decay.
Maybe actually post the kernel backtrace …

I was down this rabbit hole few times before, but perhaps someone will have some idea...

Here is the log when suspend is run on fresh boot, just X was started with xterm, no window manager etc:

Jan 12 13:36:25 megaczop systemd-logind[1505]: The system will suspend now!
Jan 12 13:36:25 megaczop ModemManager[1948]: <info>  [sleep-monitor-systemd] system is about to suspend
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.2744] manager: sleep: sleep requested (sleeping: no  enabled: yes)
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.2745] device (p2p-dev-wlp9s0): state change: disconnected -> unmanaged (reason 'sleeping', sys-iface-state: 'manag>
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.2747] device (wlp9s0): state change: activated -> deactivating (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 13:36:25 megaczop polkitd[1503]: Unregistered Authentication Agent for unix-process:5444:9127 (system bus name :1.34, object path /org/freedesktop/PolicyKit1/AuthenticationAg>
Jan 12 13:36:25 megaczop dbus-daemon[1497]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested>
Jan 12 13:36:25 megaczop systemd[1]: Starting NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service...
Jan 12 13:36:25 megaczop dbus-daemon[1497]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Jan 12 13:36:25 megaczop systemd[1]: Started NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service.
Jan 12 13:36:25 megaczop kernel: wlp9s0: deauthenticating from 5c:49:79:59:ca:70 by local choice (Reason: 3=DEAUTH_LEAVING)
Jan 12 13:36:25 megaczop wpa_supplicant[1944]: wlp9s0: CTRL-EVENT-DISCONNECTED bssid=5c:49:79:59:ca:70 reason=3 locally_generated=1
Jan 12 13:36:25 megaczop wpa_supplicant[1944]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5606] device (wlp9s0): state change: deactivating -> disconnected (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 13:36:25 megaczop wpa_supplicant[1944]: wlp9s0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Withdrawing address record for fe80::2cbf:a8:2f97:2665 on wlp9s0.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Leaving mDNS multicast group on interface wlp9s0.IPv6 with address fe80::2cbf:a8:2f97:2665.
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5761] dhcp4 (wlp9s0): canceled DHCP transaction
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5761] dhcp4 (wlp9s0): activation: beginning transaction (timeout in 45 seconds)
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5761] dhcp4 (wlp9s0): state changed no lease
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5762] dhcp6 (wlp9s0): canceled DHCP transaction
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5762] dhcp6 (wlp9s0): activation: beginning transaction (timeout in 45 seconds)
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.5763] dhcp6 (wlp9s0): state changed no lease
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Interface wlp9s0.IPv6 no longer relevant for mDNS.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Interface wlp9s0.IPv4 no longer relevant for mDNS.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Leaving mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Withdrawing address record for 192.168.1.132 on wlp9s0.
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.6777] device (wlp9s0): set-hw-addr: set MAC address to 16:FD:1C:4C:3D:D6 (scanning)
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Joining mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: New relevant interface wlp9s0.IPv4 for mDNS.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Registering new address record for 192.168.1.132 on wlp9s0.IPv4.
Jan 12 13:36:25 megaczop dnsmasq[4583]: no servers found in /etc/resolv.conf, will retry
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Withdrawing address record for 192.168.1.132 on wlp9s0.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Leaving mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 13:36:25 megaczop avahi-daemon[1495]: Interface wlp9s0.IPv4 no longer relevant for mDNS.
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.8048] device (wlp9s0): supplicant interface state: completed -> disconnected
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.8050] device (wlp9s0): state change: disconnected -> unmanaged (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 13:36:25 megaczop NetworkManager[1941]: <info>  [1736685385.8802] device (wlp9s0): set-hw-addr: reset MAC address to F0:A6:54:D1:8D:2B (unmanage)
Jan 12 13:36:25 megaczop wpa_supplicant[1944]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 13:36:25 megaczop systemd[1]: Reached target sleep.target - Sleep.
Jan 12 13:36:25 megaczop systemd[1]: Starting gnome-shell-suspend.service - Suspend gnome-shell...
Jan 12 13:36:25 megaczop suspend-gnome-shell.sh[5472]: gnome-shell: no process found
Jan 12 13:36:25 megaczop systemd[1]: gnome-shell-suspend.service: Main process exited, code=exited, status=1/FAILURE
Jan 12 13:36:25 megaczop systemd[1]: gnome-shell-suspend.service: Failed with result 'exit-code'.
Jan 12 13:36:25 megaczop systemd[1]: Failed to start gnome-shell-suspend.service - Suspend gnome-shell.
Jan 12 13:36:25 megaczop systemd[1]: Starting nvidia-suspend.service - NVIDIA system suspend actions...
Jan 12 13:36:25 megaczop suspend[5473]: nvidia-suspend.service
Jan 12 13:36:25 megaczop logger[5473]: <13>Jan 12 13:36:25 suspend: nvidia-suspend.service
Jan 12 13:36:26 megaczop wpa_supplicant[1944]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 13:36:26 megaczop wpa_supplicant[1944]: nl80211: deinit ifname=wlp9s0 disabled_11b_rates=0
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/ldac
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/aptx_hd
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx_hd
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/aptx
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/sbc
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/sbc
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/sbc_xq
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/sbc_xq
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx_ll_1
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx_ll_0
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_1
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_0
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/faststream
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/faststream_duplex
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/opus_05
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/opus_05
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSink/opus_05_duplex
Jan 12 13:36:26 megaczop bluetoothd[1496]: Endpoint unregistered: sender=:1.31 path=/MediaEndpoint/A2DPSource/opus_05_duplex
Jan 12 13:36:26 megaczop bluetoothd[1496]: Failed to remove UUID: Failed (0x03)
Jan 12 13:36:26 megaczop bluetoothd[1496]: Failed to remove UUID: Failed (0x03)
Jan 12 13:36:26 megaczop bluetoothd[1496]: Failed to remove UUID: Failed (0x03)
Jan 12 13:36:26 megaczop bluetoothd[1496]: Failed to remove UUID: Failed (0x03)
Jan 12 13:36:26 megaczop kernel: BUG: unable to handle page fault for address: 00000000000036c8
Jan 12 13:36:26 megaczop kernel: #PF: supervisor write access in kernel mode
Jan 12 13:36:26 megaczop kernel: #PF: error_code(0x0002) - not-present page
Jan 12 13:36:26 megaczop kernel: PGD 0 P4D 0
Jan 12 13:36:26 megaczop kernel: Oops: 0002 [#1] PREEMPT SMP NOPTI
Jan 12 13:36:26 megaczop kernel: CPU: 17 PID: 5474 Comm: nvidia-sleep.sh Tainted: P           OE      6.8.4-custom #4
Jan 12 13:36:26 megaczop kernel: Hardware name: ASUS System Product Name/ProArt X670E-CREATOR WIFI, BIOS 2124 07/17/2024
Jan 12 13:36:26 megaczop kernel: RIP: 0010:_nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel: Code: 89 de 4c 89 ff 41 89 c5 e8 05 cc ff ff ba 01 00 00 00 48 89 de 4c 89 ff 41 09 c5 e8 f2 cb ff ff 44 08 e8 74 05 e8 28 ef ff ff <c7> 83 c8 36 00>
Jan 12 13:36:26 megaczop kernel: RSP: 0018:ffffb5c8c9fb39b0 EFLAGS: 00010246
Jan 12 13:36:26 megaczop kernel: RAX: ffffffffc1ff03c8 RBX: 0000000000000000 RCX: 0000000000000000
Jan 12 13:36:26 megaczop kernel: RDX: ffffb5c8c0e03008 RSI: 0000000000000000 RDI: 0000000000000001
Jan 12 13:36:26 megaczop kernel: RBP: ffffb5c8c9fb3a00 R08: ffffffffc1ff03c0 R09: 0000000000000001
Jan 12 13:36:26 megaczop kernel: R10: 0000000000000000 R11: ffff95874d9a2008 R12: 0000000000000000
Jan 12 13:36:26 megaczop kernel: R13: 0000000000000000 R14: ffff95876117b368 R15: ffffb5c8c0cc3008
Jan 12 13:36:26 megaczop kernel: FS:  00007f8aad20d740(0000) GS:ffff959e3e440000(0000) knlGS:0000000000000000
Jan 12 13:36:26 megaczop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 12 13:36:26 megaczop kernel: CR2: 00000000000036c8 CR3: 000000017df0a000 CR4: 0000000000750ef0
Jan 12 13:36:26 megaczop kernel: PKRU: 55555554
Jan 12 13:36:26 megaczop kernel: Call Trace:
Jan 12 13:36:26 megaczop kernel:  <TASK>
Jan 12 13:36:26 megaczop kernel:  ? __die+0x1f/0x70
Jan 12 13:36:26 megaczop kernel:  ? page_fault_oops+0x17d/0x4c0
Jan 12 13:36:26 megaczop kernel:  ? _nv000468kms+0x270/0x600 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? exc_page_fault+0x73/0x170
Jan 12 13:36:26 megaczop kernel:  ? asm_exc_page_fault+0x22/0x30
Jan 12 13:36:26 megaczop kernel:  ? _nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? _nv002437kms+0xf0/0x180 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? _nv002760kms+0x3b80/0x4c40 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? down+0x1a/0x60
Jan 12 13:36:26 megaczop kernel:  ? _nv002775kms+0x18b/0x1f0 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? nvKmsSuspend+0x3a/0x90 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? down+0x1a/0x60
Jan 12 13:36:26 megaczop kernel:  ? nvkms_suspend+0x1f/0x40 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel:  ? nv_set_system_power_state+0x174/0x440 [nvidia]
Jan 12 13:36:26 megaczop kernel:  ? nv_procfs_write_suspend+0xe4/0x150 [nvidia]
Jan 12 13:36:26 megaczop kernel:  ? proc_reg_write+0x56/0xa0
Jan 12 13:36:26 megaczop kernel:  ? preempt_count_add+0x47/0xa0
Jan 12 13:36:26 megaczop kernel:  ? vfs_write+0xe7/0x480
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? __handle_mm_fault+0xb39/0xe50
Jan 12 13:36:26 megaczop kernel:  ? __do_sys_newfstatat+0x4e/0x80
Jan 12 13:36:26 megaczop kernel:  ? ksys_write+0x6b/0xf0
Jan 12 13:36:26 megaczop kernel:  ? do_syscall_64+0x86/0x1b0
Jan 12 13:36:26 megaczop kernel:  ? count_memcg_events.constprop.0+0x1a/0x30
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? handle_mm_fault+0x9e/0x360
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? preempt_count_add+0x47/0xa0
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? up_read+0x37/0x70
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? do_user_addr_fault+0x304/0x670
Jan 12 13:36:26 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 13:36:26 megaczop kernel:  ? fpregs_assert_state_consistent+0x22/0x50
Jan 12 13:36:26 megaczop kernel:  ? entry_SYSCALL_64_after_hwframe+0x73/0x7b
Jan 12 13:36:26 megaczop kernel:  </TASK>
Jan 12 13:36:26 megaczop kernel: Modules linked in: rfcomm snd_seq_dummy snd_hrtimer snd_seq xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_conntrack nft_chain_nat xt_MASQUERADE>
Jan 12 13:36:26 megaczop kernel:  snd_hda_codec aesni_intel snd_usbmidi_lib asus_nb_wmi drbg eeepc_wmi snd_hda_core snd_rawmidi asus_wmi ansi_cprng libarc4 ecdh_generic snd_seq_devi>
Jan 12 13:36:26 megaczop kernel:  crc32_pclmul usbcore crc64_rocksoft scsi_mod crc32c_intel crc64 crc_t10dif igc i2c_piix4 crct10dif_generic macsec usb_common scsi_common crct10dif_>
Jan 12 13:36:26 megaczop kernel: CR2: 00000000000036c8
Jan 12 13:36:26 megaczop kernel: ---[ end trace 0000000000000000 ]---
Jan 12 13:36:26 megaczop kernel: RIP: 0010:_nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 13:36:26 megaczop kernel: Code: 89 de 4c 89 ff 41 89 c5 e8 05 cc ff ff ba 01 00 00 00 48 89 de 4c 89 ff 41 09 c5 e8 f2 cb ff ff 44 08 e8 74 05 e8 28 ef ff ff <c7> 83 c8 36 00>
Jan 12 13:36:26 megaczop kernel: RSP: 0018:ffffb5c8c9fb39b0 EFLAGS: 00010246
Jan 12 13:36:26 megaczop kernel: RAX: ffffffffc1ff03c8 RBX: 0000000000000000 RCX: 0000000000000000
Jan 12 13:36:26 megaczop kernel: RDX: ffffb5c8c0e03008 RSI: 0000000000000000 RDI: 0000000000000001
Jan 12 13:36:26 megaczop kernel: RBP: ffffb5c8c9fb3a00 R08: ffffffffc1ff03c0 R09: 0000000000000001
Jan 12 13:36:26 megaczop kernel: R10: 0000000000000000 R11: ffff95874d9a2008 R12: 0000000000000000
Jan 12 13:36:26 megaczop kernel: R13: 0000000000000000 R14: ffff95876117b368 R15: ffffb5c8c0cc3008
Jan 12 13:36:26 megaczop kernel: FS:  00007f8aad20d740(0000) GS:ffff959e3e440000(0000) knlGS:0000000000000000
Jan 12 13:36:26 megaczop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 12 13:36:26 megaczop kernel: CR2: 00000000000036c8 CR3: 000000017df0a000 CR4: 0000000000750ef0
Jan 12 13:36:26 megaczop kernel: PKRU: 55555554
Jan 12 13:36:26 megaczop kernel: note: nvidia-sleep.sh[5474] exited with irqs disabled
Jan 12 13:36:26 megaczop systemd[1]: nvidia-suspend.service: Main process exited, code=killed, status=9/KILL
Jan 12 13:36:26 megaczop systemd[1]: nvidia-suspend.service: Failed with result 'signal'.
Jan 12 13:36:26 megaczop systemd[1]: Failed to start nvidia-suspend.service - NVIDIA system suspend actions.
Jan 12 13:36:26 megaczop systemd[1]: Starting systemd-suspend.service - System Suspend...
Jan 12 13:36:26 megaczop systemd-sleep[5485]: Entering sleep state 'suspend'...

Here on the other hand is a crash (very similar)_ when suspend is run from GNOME within and Xorg session:

Jan 12 10:53:55 megaczop systemd-logind[1515]: The system will suspend now!
Jan 12 10:53:55 megaczop ModemManager[1997]: <info>  [sleep-monitor-systemd] system is about to suspend
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.4560] manager: sleep: sleep requested (sleeping: no  enabled: yes)
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.4561] device (p2p-dev-wlp9s0): state change: disconnected -> unmanaged (reason 'sleeping', sys-iface-state: 'manag>
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.4563] device (wlp9s0): state change: activated -> deactivating (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 10:53:55 megaczop dbus-daemon[1505]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested>
Jan 12 10:53:55 megaczop systemd[1]: Starting NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service...
Jan 12 10:53:55 megaczop dbus-daemon[1505]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Jan 12 10:53:55 megaczop systemd[1]: Started NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service.
Jan 12 10:53:55 megaczop kernel: wlp9s0: deauthenticating from 5c:49:79:59:ca:70 by local choice (Reason: 3=DEAUTH_LEAVING)
Jan 12 10:53:55 megaczop wpa_supplicant[1991]: wlp9s0: CTRL-EVENT-DISCONNECTED bssid=5c:49:79:59:ca:70 reason=3 locally_generated=1
Jan 12 10:53:55 megaczop wpa_supplicant[1991]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 10:53:55 megaczop wpa_supplicant[1991]: wlp9s0: CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7333] device (wlp9s0): supplicant interface state: completed -> disconnected
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7334] device (wlp9s0): state change: deactivating -> disconnected (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Withdrawing address record for fe80::2cbf:a8:2f97:2665 on wlp9s0.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Leaving mDNS multicast group on interface wlp9s0.IPv6 with address fe80::2cbf:a8:2f97:2665.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Interface wlp9s0.IPv6 no longer relevant for mDNS.
Jan 12 10:53:55 megaczop dbus-daemon[1505]: [system] Activating via systemd: service name='org.freedesktop.PackageKit' unit='packagekit.service' requested by ':1.72' (uid=1000 pid=5>
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7768] dhcp4 (wlp9s0): canceled DHCP transaction
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7768] dhcp4 (wlp9s0): activation: beginning transaction (timeout in 45 seconds)
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7768] dhcp4 (wlp9s0): state changed no lease
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7770] dhcp6 (wlp9s0): canceled DHCP transaction
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7770] dhcp6 (wlp9s0): activation: beginning transaction (timeout in 45 seconds)
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.7770] dhcp6 (wlp9s0): state changed no lease
Jan 12 10:53:55 megaczop systemd[1]: Starting packagekit.service - PackageKit Daemon...
Jan 12 10:53:55 megaczop PackageKit[12270]: daemon start
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Interface wlp9s0.IPv4 no longer relevant for mDNS.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Leaving mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Withdrawing address record for 192.168.1.132 on wlp9s0.
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.8107] device (wlp9s0): set-hw-addr: set MAC address to BE:65:00:D6:87:F0 (scanning)
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Joining mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: New relevant interface wlp9s0.IPv4 for mDNS.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Registering new address record for 192.168.1.132 on wlp9s0.IPv4.
Jan 12 10:53:55 megaczop dnsmasq[4632]: no servers found in /etc/resolv.conf, will retry
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Withdrawing address record for 192.168.1.132 on wlp9s0.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Leaving mDNS multicast group on interface wlp9s0.IPv4 with address 192.168.1.132.
Jan 12 10:53:55 megaczop avahi-daemon[1500]: Interface wlp9s0.IPv4 no longer relevant for mDNS.
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.9414] device (wlp9s0): supplicant interface state: disconnected -> interface_disabled
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.9414] device (wlp9s0): supplicant interface state: interface_disabled -> disconnected
Jan 12 10:53:55 megaczop NetworkManager[1595]: <info>  [1736675635.9416] device (wlp9s0): state change: disconnected -> unmanaged (reason 'sleeping', sys-iface-state: 'managed')
Jan 12 10:53:55 megaczop dbus-daemon[1505]: [system] Successfully activated service 'org.freedesktop.PackageKit'
Jan 12 10:53:55 megaczop systemd[1]: Started packagekit.service - PackageKit Daemon.
Jan 12 10:53:56 megaczop NetworkManager[1595]: <info>  [1736675636.0083] device (wlp9s0): set-hw-addr: reset MAC address to F0:A6:54:D1:8D:2B (unmanage)
Jan 12 10:53:56 megaczop systemd[1]: Reached target sleep.target - Sleep.
Jan 12 10:53:56 megaczop systemd[1]: Starting gnome-shell-suspend.service - Suspend gnome-shell...
Jan 12 10:53:56 megaczop wpa_supplicant[1991]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 10:53:56 megaczop systemd[1]: gnome-shell-suspend.service: Deactivated successfully.
Jan 12 10:53:56 megaczop systemd[1]: Finished gnome-shell-suspend.service - Suspend gnome-shell.
Jan 12 10:53:56 megaczop systemd[1]: Starting nvidia-suspend.service - NVIDIA system suspend actions...
Jan 12 10:53:56 megaczop suspend[12291]: nvidia-suspend.service
Jan 12 10:53:56 megaczop logger[12291]: <13>Jan 12 10:53:56 suspend: nvidia-suspend.service
Jan 12 10:53:56 megaczop wpa_supplicant[1991]: wlp9s0: CTRL-EVENT-DSCP-POLICY clear_all
Jan 12 10:53:56 megaczop wpa_supplicant[1991]: nl80211: deinit ifname=wlp9s0 disabled_11b_rates=0
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop packagekitd[12270]: pk_package_set_update_severity: assertion 'update_severity == PK_INFO_ENUM_UNKNOWN || update_severity == PK_INFO_ENUM_LOW || update_seve>
Jan 12 10:53:56 megaczop PackageKit[12270]: get-updates transaction /43239_eecbccdb from uid 1000 finished with success after 294ms
Jan 12 10:53:56 megaczop PackageKit[12270]: get-updates transaction /43240_eecaecca from uid 1000 finished with cancelled-priority after 284ms
Jan 12 10:53:56 megaczop gnome-software[5740]: internal error: failed, but no error code: cancelled-priority
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/ldac
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/aptx_hd
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx_hd
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/aptx
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/sbc
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/sbc
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/sbc_xq
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/sbc_xq
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx_ll_1
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx_ll_0
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_1
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/aptx_ll_duplex_0
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/faststream
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/faststream_duplex
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/opus_05
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/opus_05
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSink/opus_05_duplex
Jan 12 10:53:56 megaczop bluetoothd[1504]: Endpoint unregistered: sender=:1.33 path=/MediaEndpoint/A2DPSource/opus_05_duplex
Jan 12 10:53:56 megaczop bluetoothd[1504]: Failed to remove UUID: Failed (0x03)
Jan 12 10:53:56 megaczop bluetoothd[1504]: Failed to remove UUID: Failed (0x03)
Jan 12 10:53:56 megaczop bluetoothd[1504]: Failed to remove UUID: Failed (0x03)
Jan 12 10:53:56 megaczop bluetoothd[1504]: Failed to remove UUID: Failed (0x03)
Jan 12 10:53:56 megaczop kernel: BUG: unable to handle page fault for address: 00000000000036c8
Jan 12 10:53:56 megaczop kernel: #PF: supervisor write access in kernel mode
Jan 12 10:53:56 megaczop kernel: #PF: error_code(0x0002) - not-present page
Jan 12 10:53:56 megaczop kernel: PGD 0 P4D 0
Jan 12 10:53:56 megaczop kernel: Oops: 0002 [#1] PREEMPT SMP NOPTI
Jan 12 10:53:56 megaczop kernel: CPU: 31 PID: 12292 Comm: nvidia-sleep.sh Tainted: P           OE      6.8.4-custom #4
Jan 12 10:53:56 megaczop kernel: Hardware name: ASUS System Product Name/ProArt X670E-CREATOR WIFI, BIOS 2124 07/17/2024
Jan 12 10:53:56 megaczop kernel: RIP: 0010:_nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel: Code: 89 de 4c 89 ff 41 89 c5 e8 05 cc ff ff ba 01 00 00 00 48 89 de 4c 89 ff 41 09 c5 e8 f2 cb ff ff 44 08 e8 74 05 e8 28 ef ff ff <c7> 83 c8 36 00>
Jan 12 10:53:56 megaczop kernel: RSP: 0018:ffffaa9c6b0cf9f0 EFLAGS: 00010246
Jan 12 10:53:56 megaczop kernel: RAX: ffffffffc5af93c8 RBX: 0000000000000000 RCX: 0000000000000000
Jan 12 10:53:56 megaczop kernel: RDX: ffffaa9c400c3008 RSI: 0000000000000000 RDI: 0000000000000001
Jan 12 10:53:56 megaczop kernel: RBP: ffffaa9c6b0cfa40 R08: ffffffffc5af93c0 R09: 0000000000000001
Jan 12 10:53:56 megaczop kernel: R10: 0000000000000000 R11: ffff8d750df46008 R12: 0000000000000000
Jan 12 10:53:56 megaczop kernel: R13: 0000000000000000 R14: ffff8d75207d0128 R15: ffffaa9c40cdc008
Jan 12 10:53:56 megaczop kernel: FS:  00007f2c38c01740(0000) GS:ffff8d8bfe7c0000(0000) knlGS:0000000000000000
Jan 12 10:53:56 megaczop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 12 10:53:56 megaczop kernel: CR2: 00000000000036c8 CR3: 000000020914c000 CR4: 0000000000750ef0
Jan 12 10:53:56 megaczop kernel: PKRU: 55555554
Jan 12 10:53:56 megaczop kernel: Call Trace:
Jan 12 10:53:56 megaczop kernel:  <TASK>
Jan 12 10:53:56 megaczop kernel:  ? __die+0x1f/0x70
Jan 12 10:53:56 megaczop kernel:  ? page_fault_oops+0x17d/0x4c0
Jan 12 10:53:56 megaczop kernel:  ? _nv000468kms+0x270/0x600 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? exc_page_fault+0x73/0x170
Jan 12 10:53:56 megaczop kernel:  ? asm_exc_page_fault+0x22/0x30
Jan 12 10:53:56 megaczop kernel:  ? _nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? _nv002437kms+0xf0/0x180 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? _nv002760kms+0x3b80/0x4c40 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? down+0x1a/0x60
Jan 12 10:53:56 megaczop kernel:  ? _nv002775kms+0x18b/0x1f0 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 10:53:56 megaczop kernel:  ? nvKmsSuspend+0x3a/0x90 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? down+0x1a/0x60
Jan 12 10:53:56 megaczop kernel:  ? nvkms_suspend+0x1f/0x40 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel:  ? nv_set_system_power_state+0x174/0x440 [nvidia]
Jan 12 10:53:56 megaczop kernel:  ? nv_procfs_write_suspend+0xe4/0x150 [nvidia]
Jan 12 10:53:56 megaczop kernel:  ? proc_reg_write+0x56/0xa0
Jan 12 10:53:56 megaczop kernel:  ? preempt_count_add+0x47/0xa0
Jan 12 10:53:56 megaczop kernel:  ? vfs_write+0xe7/0x480
Jan 12 10:53:56 megaczop kernel:  ? ksys_write+0x6b/0xf0
Jan 12 10:53:56 megaczop kernel:  ? do_syscall_64+0x86/0x1b0
Jan 12 10:53:56 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 10:53:56 megaczop kernel:  ? do_user_addr_fault+0x304/0x670
Jan 12 10:53:56 megaczop kernel:  ? srso_alias_return_thunk+0x5/0xfbef5
Jan 12 10:53:56 megaczop kernel:  ? fpregs_assert_state_consistent+0x22/0x50
Jan 12 10:53:56 megaczop kernel:  ? entry_SYSCALL_64_after_hwframe+0x73/0x7b
Jan 12 10:53:56 megaczop kernel:  </TASK>
Jan 12 10:53:56 megaczop kernel: Modules linked in: rfcomm snd_seq_dummy snd_hrtimer snd_seq xt_CHECKSUM ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_conntrack nft_chain_nat xt_MASQUERADE>
Jan 12 10:53:56 megaczop kernel:  snd_hda_codec crc16 joydev apple_mfi_fastcharge ghash_clmulni_intel mac80211 sha512_ssse3 snd_hda_core sha512_generic sha256_ssse3 snd_hwdep asus_n>
Jan 12 10:53:56 megaczop kernel:  crc32_pclmul atlantic thunderbolt usbcore scsi_mod crc32c_intel crc64 igc crc_t10dif i2c_piix4 macsec crct10dif_generic scsi_common usb_common crct>
Jan 12 10:53:56 megaczop kernel: CR2: 00000000000036c8
Jan 12 10:53:56 megaczop kernel: ---[ end trace 0000000000000000 ]---
Jan 12 10:53:56 megaczop kernel: RIP: 0010:_nv000740kms+0xc8/0x160 [nvidia_modeset]
Jan 12 10:53:56 megaczop kernel: Code: 89 de 4c 89 ff 41 89 c5 e8 05 cc ff ff ba 01 00 00 00 48 89 de 4c 89 ff 41 09 c5 e8 f2 cb ff ff 44 08 e8 74 05 e8 28 ef ff ff <c7> 83 c8 36 00>
Jan 12 10:53:56 megaczop kernel: RSP: 0018:ffffaa9c6b0cf9f0 EFLAGS: 00010246
Jan 12 10:53:56 megaczop kernel: RAX: ffffffffc5af93c8 RBX: 0000000000000000 RCX: 0000000000000000
Jan 12 10:53:56 megaczop kernel: RDX: ffffaa9c400c3008 RSI: 0000000000000000 RDI: 0000000000000001
Jan 12 10:53:56 megaczop kernel: RBP: ffffaa9c6b0cfa40 R08: ffffffffc5af93c0 R09: 0000000000000001
Jan 12 10:53:56 megaczop kernel: R10: 0000000000000000 R11: ffff8d750df46008 R12: 0000000000000000
Jan 12 10:53:56 megaczop kernel: R13: 0000000000000000 R14: ffff8d75207d0128 R15: ffffaa9c40cdc008
Jan 12 10:53:56 megaczop kernel: FS:  00007f2c38c01740(0000) GS:ffff8d8bfe7c0000(0000) knlGS:0000000000000000
Jan 12 10:53:56 megaczop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 12 10:53:56 megaczop kernel: CR2: 00000000000036c8 CR3: 000000020914c000 CR4: 0000000000750ef0
Jan 12 10:53:56 megaczop kernel: PKRU: 55555554
Jan 12 10:53:56 megaczop kernel: note: nvidia-sleep.sh[12292] exited with irqs disabled
Jan 12 10:53:56 megaczop systemd[1]: nvidia-suspend.service: Main process exited, code=killed, status=9/KILL
Jan 12 10:53:56 megaczop systemd[1]: nvidia-suspend.service: Failed with result 'signal'.
Jan 12 10:53:56 megaczop systemd[1]: Failed to start nvidia-suspend.service - NVIDIA system suspend actions.
Jan 12 10:53:56 megaczop systemd[1]: Starting systemd-suspend.service - System Suspend...
Jan 12 10:53:56 megaczop systemd-sleep[12318]: Entering sleep state 'suspend'...

If you or anyone else can suggest anything that would be great.

BTW, I have nvidia services running (I tried with stopped, no difference). I currently have this KERNEL command line:

GRUB_CMDLINE_LINUX_DEFAULT="quiet iomem=relaxed sysrq_always_enabled=1 nvidia_drm.modeset=1"

And I tried this modules config:

options nvidia-drm modeset=1
#options nvidia-drm fbdev=1
options nvidia NVreg_PreserveVideoMemoryAllocations=1
options nvidia NVreg_TemporaryFilePath=/var/tmp

(commenting/uncommenting various lines, I found no difference so I left it commented out for now).

My nvidia-smi info is:

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3090        Off | 00000000:01:00.0  On |                  N/A |
|  0%   30C    P8              31W / 370W |   1080MiB / 24576MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090        Off | 00000000:03:00.0 Off |                  N/A |
|  0%   29C    P8              15W / 370W |     12MiB / 24576MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      5048      G   /usr/lib/xorg/Xorg                          521MiB |
|    0   N/A  N/A      5249      G   /usr/bin/gnome-shell                        243MiB |
|    0   N/A  N/A      6035      G   alacritty                                    13MiB |
|    0   N/A  N/A      7252      G   ...eLinuxGL --disable-features=AllowQt      212MiB |
|    0   N/A  N/A      9127      G   /usr/bin/gnome-text-editor                   72MiB |
|    1   N/A  N/A      5048      G   /usr/lib/xorg/Xorg                            4MiB |
+---------------------------------------------------------------------------------------+

Nvidia-smi topo -m is:

        GPU0    GPU1    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      NV4     0-31    0               N/A
GPU1    NV4      X      0-31    0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

(I had very similar issues with just an RTX2080 so please don't get drawn to the fact I have a SLI/NVLink setup with two rtx3090)

and the kernel is: 6.8.4

Edit: I would prefer not to go to the beta driver (I read not a single report suspend actually works there). I'm looking for some LTS or main release Kernel/driver combination that works. I was thinking about trying 6.12 LTS.

Last edited by 7000k (2025-01-12 12:53:13)

seth · 2025-01-12 14:32:32

Why do you set "iomem=relaxed"?
What is the "6.8.4-custom" kernel? (Notably the "custom" part of that)

How much (free) space do you have in /var/run (which most likely is a tmpfs)?

df -h

What happens if you simply disable NVreg_PreserveVideoMemoryAllocations and the userspace services?
(Yes, you might run into the decay issue on longer sleeps but it's clearly what triggers the kernel module crash - the page fault heavily implying that you run out of allocatable space, ie. /var/run is underdimensioned to sponge the VRAM)

7000k · 2025-01-13 08:17:49

Anyone has any idea how to check which module (if this is indeed a module crash) this is happening?

Does this line mean what I think it means? (that the crash happened when nvidia_modeset was called)

RIP: 0010:_nv000740kms+0xc8/0x160 [nvidia_modeset]

seth · 2025-01-13 09:17:19

Does this line mean what I think it means?

Yes, did you actually read my last post?

bertieb · 2025-01-13 11:44:34

7000k wrote:

Does anyone know what combination of kernel/driver/kernel options works?

To answer this specific question:

pacman & uname & nvidia-debugdump * wrote:

$ pacman -Q nvidia
nvidia-535xx-dkms 535.183.01-2
$ uname -r
6.1.91-1-lts61
$ nvidia-debugdump --list
Found 1 NVIDIA devices
Device ID: 0
Device name: NVIDIA GeForce GTX 970 (*PrimaryCard)
GPU internal ID: GPU-bf73e6ad-0567-643c-5a9d-369b88e45323

modprobe.d/nvidia.conf wrote:

options nvidia_drm modeset=1 fbdev=1
options nvidia NVreg_PreserveVideoMemoryAllocations=1 NVreg_TemporaryFilePath=/var/tmp

- driver: 535
- kernel: 6.1
- options: preserve VM allocs & tempfile path in /var (I could probably use RAM tmpfs as seth suggested to Tharbad)
- card: GTX 970

There may be more recent versions that work, but this combination works for me.

Last edited by bertieb (2025-01-13 11:50:36)

7000k · 2025-01-13 14:46:48

The problem is resolved with Kernel 6.12.9 and driver 550.142 !

But It may help someone who can't migrate to this driver/kernel to describe some additional troubleshooting steps I went through as well as answer the questions below.

seth wrote:

Does this line mean what I think it means?
Yes, did you actually read my last post?

It wasn't there when I posted. Perhaps I didn't hit refresh before posting.

seth wrote:

Why do you set "iomem=relaxed"?

Because the driver I use to read RAM stick temps requires it.

seth wrote:

What is the "6.8.4-custom" kernel? (Notably the "custom" part of that)

It is not relevant to this issue. It contains some settings (that Ubuntu config has by default but Arch and many other distributions have the opposite) about RAM compression and such. It makes certain CPU executed AI model inference a lot faster when almost all of RAM is in use. I believe I described it on this forum, possibly a year or two ago for an older kernel. I just kept these settings as they work for me.I think it is not relevant because I tried the stock kernel with no difference.

seth wrote:

How much (free) space do you have in /var/run (which most likely is a tmpfs)?
df -h

24GB, /var/run is on /

seth wrote:

What happens if you simply disable NVreg_PreserveVideoMemoryAllocations and the userspace services?
(Yes, you might run into the decay issue on longer sleeps but it's clearly what triggers the kernel module crash - the page fault heavily implying that you run out of allocatable space, ie. /var/run is underdimensioned to sponge the VRAM)

That is an interesting take. I wish I read it sooner and I had a chance to test increasing the amount of space there, before trying the new kernel/driver combo. My total amount of VRAM is 48GB. About 2X the amount of space in /var/run. So perhaps you are right. Is it possible the new driver does not require it (or uses swap etc.)

Regarding other troubleshooting I updated the Kernel to 6.12.9. I also had top make a number of modifications to the Nvidia driver 535 to make it build on this new kernel (I post the diff below if someone wants it). But I ended up getting exact same error message and crash.

Only after I upgraded the driver to 550.142 it started working and continues to work (from Xorg as well as outside, there is not even a change of VT needed). I still have all the options commented out in modules.conf, my kernel command line is still "quiet iomem=relaxed sysrq_always_enabled=1 nvidia_drm.modeset=1" and all nvidia services are running.

Regarding these changes to make driver 535 work it is:

in nvidia/os-mlock.c replace existing nv_follow_pfn function with:

static inline int nv_follow_pfn(struct vm_area_struct *vma,
                                unsigned long address,
                                unsigned long *pfn)
{
    struct follow_pfnmap_args args;
    int ret;

    /*
     * Just like with the old follow_pfn() semantics,
     * we only allow IO mappings or raw PFN mappings.
     */
    if (!(vma->vm_flags & (VM_IO | VM_PFNMAP)))
        return -EINVAL;

    memset(&args, 0, sizeof(args));
    args.vma = vma;
    args.address = address;

    ret = follow_pfnmap_start(&args);
    if (ret)
        return ret;

    /* On success, copy out the PFN found. */
    *pfn = args.pfn;

    /*
     * Must be called after we’re done using the fields in ‘args’.
     * This will release any locks/resources acquired in follow_pfnmap_start().
     */
    follow_pfnmap_end(&args);

    return 0;
}

Then in nvidia-drm/nvidia-drm-drv.c replace these structs and functions:

static const struct drm_mode_config_funcs nv_mode_config_funcs = {
    .fb_create = nv_drm_framebuffer_create,

    .atomic_state_alloc = nv_drm_atomic_state_alloc,
    .atomic_state_clear = nv_drm_atomic_state_clear,
    .atomic_state_free  = nv_drm_atomic_state_free,
    .atomic_check  = nv_drm_atomic_check,
    .atomic_commit = nv_drm_atomic_commit,

    // Remove output_poll_changed as it is no longer present in drm_mode_config_funcs
    // .output_poll_changed = nv_drm_output_poll_changed, // Removed
};

/*
 * Helper function to initialize drm_device::mode_config from
 * NvKmsKapiDevice's resource information.
 */
static void
nv_drm_init_mode_config(struct nv_drm_device *nv_dev,
                        const struct NvKmsKapiDeviceResourcesInfo *pResInfo)
{
    struct drm_device *dev = nv_dev->dev;

    drm_mode_config_init(dev);
    drm_mode_create_dvi_i_properties(dev);

    dev->mode_config.funcs = &nv_mode_config_funcs;

    dev->mode_config.min_width  = pResInfo->caps.minWidthInPixels;
    dev->mode_config.min_height = pResInfo->caps.minHeightInPixels;

    dev->mode_config.max_width  = pResInfo->caps.maxWidthInPixels;
    dev->mode_config.max_height = pResInfo->caps.maxHeightInPixels;

    dev->mode_config.cursor_width  = pResInfo->caps.maxCursorSizeInPixels;
    dev->mode_config.cursor_height = pResInfo->caps.maxCursorSizeInPixels;

    /*
     * NVIDIA GPUs have no preferred depth. Arbitrarily report 24, to be
     * consistent with other DRM drivers.
     */

    dev->mode_config.preferred_depth = 24;
    dev->mode_config.prefer_shadow = 1;

#if defined(NV_DRM_CRTC_STATE_HAS_ASYNC_FLIP) || \
    defined(NV_DRM_CRTC_STATE_HAS_PAGEFLIP_FLAGS)
    dev->mode_config.async_page_flip = true;
#else
    dev->mode_config.async_page_flip = false;
#endif

#if defined(NV_DRM_FORMAT_MODIFIERS_PRESENT) && \
    defined(NV_DRM_MODE_CONFIG_HAS_ALLOW_FB_MODIFIERS)
    /* Allow clients to define framebuffer layouts using DRM format modifiers */
    dev->mode_config.allow_fb_modifiers = true;
#endif

    /*
     * Output polling initialization and disabling.
     * As output polling is no longer explicitly supported in the funcs,
     * the related helper calls remain the same but should align with the
     * kernel's updated expectations.
     */

    drm_kms_helper_poll_init(dev);

    /* Explicitly disable polling since it's not supported */
    drm_kms_helper_poll_disable(dev);
}

static const struct drm_ioctl_desc nv_drm_ioctls[] = {
#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_IMPORT_NVKMS_MEMORY,
                      nv_drm_gem_import_nvkms_memory_ioctl,
                      DRM_RENDER_ALLOW),
#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */

    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_IMPORT_USERSPACE_MEMORY,
                      nv_drm_gem_import_userspace_memory_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_MAP_OFFSET,
                      nv_drm_gem_map_offset_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GET_DEV_INFO,
                      nv_drm_get_dev_info_ioctl,
                      DRM_RENDER_ALLOW),

#if defined(NV_DRM_FENCE_AVAILABLE)
    DRM_IOCTL_DEF_DRV(NVIDIA_FENCE_SUPPORTED,
                      nv_drm_fence_supported_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_PRIME_FENCE_CONTEXT_CREATE,
                      nv_drm_prime_fence_context_create_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_PRIME_FENCE_ATTACH,
                      nv_drm_gem_prime_fence_attach_ioctl,
                      DRM_RENDER_ALLOW),
#endif

    DRM_IOCTL_DEF_DRV(NVIDIA_GET_CLIENT_CAPABILITY,
                      nv_drm_get_client_capability_ioctl,
                      0),
#if defined(NV_DRM_ATOMIC_MODESET_AVAILABLE)
    DRM_IOCTL_DEF_DRV(NVIDIA_GET_CRTC_CRC32,
                      nv_drm_get_crtc_crc32_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GET_CRTC_CRC32_V2,
                      nv_drm_get_crtc_crc32_v2_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_EXPORT_NVKMS_MEMORY,
                      nv_drm_gem_export_nvkms_memory_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_ALLOC_NVKMS_MEMORY,
                      nv_drm_gem_alloc_nvkms_memory_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_EXPORT_DMABUF_MEMORY,
                      nv_drm_gem_export_dmabuf_memory_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GEM_IDENTIFY_OBJECT,
                      nv_drm_gem_identify_object_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_DMABUF_SUPPORTED,
                      nv_drm_dmabuf_supported_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GET_DPY_ID_FOR_CONNECTOR_ID,
                      nv_drm_get_dpy_id_for_connector_id_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GET_CONNECTOR_ID_FOR_DPY_ID,
                      nv_drm_get_connector_id_for_dpy_id_ioctl,
                      DRM_RENDER_ALLOW),
    DRM_IOCTL_DEF_DRV(NVIDIA_GRANT_PERMISSIONS,
                      nv_drm_grant_permission_ioctl,
                      DRM_MASTER),
    DRM_IOCTL_DEF_DRV(NVIDIA_REVOKE_PERMISSIONS,
                      nv_drm_revoke_permission_ioctl,
                      DRM_MASTER),
#endif /* NV_DRM_ATOMIC_MODESET_AVAILABLE */
};

There is also one change required in the kernel to change in mm/memory.c two lines that contains EXPORT_SYMBOL_GPL(follow_pfnmap_start) and same with follow_pfnmap_end to EXPORT_SYMBOL(follow_pfnmap_start) and same for end.

bertieb wrote:

7000k wrote:
Does anyone know what combination of kernel/driver/kernel options works?
To answer this specific question:
pacman & uname & nvidia-debugdump * wrote:
$ pacman -Q nvidia
nvidia-535xx-dkms 535.183.01-2
$ uname -r
6.1.91-1-lts61
$ nvidia-debugdump --list
Found 1 NVIDIA devices
Device ID: 0
Device name: NVIDIA GeForce GTX 970 (*PrimaryCard)
GPU internal ID: GPU-bf73e6ad-0567-643c-5a9d-369b88e45323
modprobe.d/nvidia.conf wrote:
options nvidia_drm modeset=1 fbdev=1
options nvidia NVreg_PreserveVideoMemoryAllocations=1 NVreg_TemporaryFilePath=/var/tmp
- driver: 535
- kernel: 6.1
- options: preserve VM allocs & tempfile path in /var (I could probably use RAM tmpfs as seth suggested to Tharbad)
- card: GTX 970
There may be more recent versions that work, but this combination works for me.

Thank you for answering.

Last edited by 7000k (2025-01-13 14:48:00)

obap74 · 2025-01-15 17:32:03

Something that has been working reliably for me in the last few weeks (GTX 970 / 6.6.67 kernel / NVIDIA 550.142) is to suspend from a TTY while connected with another user (works with root user as well) with:

exec systemctl suspend

exec allows to log out after executing the command. This avoids getting an unlocked session after resuming the machine, I'm back to the login prompt.

Then, I can switch to TTY1 where my regular user is connected with my X session. I'm then met with xsecurelock (that is run before suspend via a systemd service unit) and I can unlock my session just fine.

Unfortunately, this doesn't work with 6.12.9 kernel + NVIDIA 565.77-3. Although the machine resumes fine, after switching to TTY1, all I get is a black screen with my mouse cursor and all I can do is to restart AwesomeWM.

Arch Linux

#151 2024-12-22 07:43:50

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#152 2024-12-22 08:26:00

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#153 2024-12-22 09:53:45

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#154 2024-12-22 10:35:43

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#155 2024-12-22 14:16:54

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#156 2024-12-23 15:48:48

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#157 2024-12-24 19:38:48

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#158 2024-12-24 20:12:03

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#159 2024-12-25 11:31:01

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#160 2024-12-25 14:04:58

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#161 2024-12-25 17:40:22

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#162 2024-12-26 21:52:20

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#163 2024-12-29 10:57:56

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#164 2025-01-02 22:10:11

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#165 2025-01-09 15:17:26

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#166 2025-01-09 16:56:52

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#167 2025-01-12 02:30:56

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#168 2025-01-12 08:35:26

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#169 2025-01-12 12:51:00

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#170 2025-01-12 14:32:32

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#171 2025-01-13 08:17:49

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#172 2025-01-13 09:17:19

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#173 2025-01-13 11:44:34

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#174 2025-01-13 14:46:48

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

#175 2025-01-15 17:32:03

Re: NVIDIA - cannot resume from suspend with PreserveVideoMemoryAllocation

Board footer