Arch Linux

RadiatedExodus

Member

Registered: 2021-11-12

Posts: 5

[SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

Hi, I have this issue where the NVIDIA dedicated GPU on my laptop never enters the D3 power state even though nothing is using it:

# cat /sys/bus/pci/devices/0000\:01\:00.0/power_state 
D0
# cat /sys/bus/pci/devices/0000\:01\:00.0/power/runtime_status 
active
# cat /sys/bus/pci/devices/0000\:01\:00.0/power/runtime_suspended_time 
272697
# cat /sys/bus/pci/devices/0000\:01\:00.0/power/runtime_suspended_time 
272697

nvidia-smi and nvtop does not show that the GPU is being used.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 575.51.02              Driver Version: 575.51.02      CUDA Version: 12.9     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3050 ...    On  |   00000000:01:00.0 Off |                  N/A |
| N/A   34C    P8              6W /   40W |       4MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I should note that I'm also dynamically binding and unbinding the dedicated GPU for VFIO usage, by default the GPU starts with the vfio modules taking over and then it will be restored back to the NVIDIA kernel module using a systemd service that runs on boot.
I also usually at home use a external monitor via HDMI (directly to the dedicated), however even when I'm outside (so, not using a external monitor nor plugged in to power) it never enters D3 state.

Laptop: Lenovo Legion 5 82JW 15ACH6
CPU: AMD Ryzen 7 5800H
Integrated: AMD Radeon
Dedicated: RTX 3050 Ti Mobile

lspci:

01:00.0 VGA compatible controller: NVIDIA Corporation GA107BM [GeForce RTX 3050 Ti Mobile] (rev a1)
	Subsystem: Lenovo Device 3aac
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device: NVIDIA Corporation GA107 High Definition Audio Controller (rev a1)
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel
06:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [Radeon Vega Series / Radeon Vega Mobile Series] (rev c5)
	Subsystem: Lenovo Device 3aac
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu

Kernel cmdline:

intel_pstate=passive kernel.split_lock_mitigate=0 BOOT_IMAGE=/boot/vmlinuz-linux613-tkg-eevdf root=UUID=<part uuid was here> rw loglevel=3 quiet splash amd_iommu=1 iommu=pt kvm_amd.nested=1 vfio-pci.ids=10de:25e0,10de:2291

Unbinding dgpu from vfio:

virsh nodedev-reattach pci_0000_01_00_0
virsh nodedev-reattach pci_0000_01_00_1
rmmod vfio_pci vfio_pci_core vfio_iommu_type1 vfio
modprobe nvidia_drm modeset=1 fbdev=1
modprobe nvidia NVreg_DynamicPowerManagement=0x02
modprobe nvidia_modeset nvidia_uvm
systemctl start nvidia-persistenced

Binding vfio to dgpu:

systemctl stop nvidia-persistenced
sleep 1s
udevadm trigger --action=remove /dev/dri/card1
sleep 1s
lsof /dev/nvidia0 | awk '{print $2}' | xargs -I {} kill {}
lsof /dev/dri/card1 | awk '{print $2}' | xargs -I {} kill {}
lsof /dev/dri/renderD129 | awk '{print $2}' | xargs -I {} kill {}
sleep 2s
rmmod nvidia_drm
rmmod nvidia_modeset
rmmod nvidia_uvm
rmmod nvidia
sleep 2s
modprobe vfio_pci vfio_pci_core vfio_iommu_type1 vfio
virsh nodedev-detach pci_0000_01_00_0
virsh nodedev-detach pci_0000_01_00_1
sleep 1s

EDIT:
I forgot to mention, I've already followed the articles below from the wiki, and I do use PRIME:
- https://wiki.archlinux.org/title/NVIDIA
- https://wiki.archlinux.org/title/NVIDIA/Tips_and_tricks
- https://wiki.archlinux.org/title/NVIDIA/Troubleshooting
- https://wiki.archlinux.org/title/PRIME# … Management

The GPU does support D3 aswell:

# cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

S0ix Power Management:
 Platform Support:          Not Supported
 Status:                    Disabled

Notebook Dynamic Boost:     Supported

Last edited by RadiatedExodus (Yesterday 09:14:09)

seth

Member

Registered: 2012-09-03

Posts: 63,460

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

Please use [code][/code] tags. Edit your post in this regard.

nb. that nvidia-smi will be "using" (ie. waking) the GPU - do you maybe also run any frequent scripts to poll it for sensor data (conky, temperature, GPU load indicators etc)?

I should note that I'm also dynamically binding and unbinding the dedicated GPU for VFIO usage, by default the GPU starts with the vfio modules taking over and then it will be restored back to the NVIDIA kernel module using a systemd service that runs on boot.
I also usually at home use a external monitor via HDMI (directly to the dedicated), however even when I'm outside (so, not using a external monitor nor plugged in to power) it never enters D3 state.

Focus the analysis on when the GPU is neither passed through nor used by any outputs (the latter will require the GPU to be active and when passed through you should not even be able to query it?)

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

RadiatedExodus

Member

Registered: 2021-11-12

Posts: 5

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

Please use [code][/code] tags. Edit your post in this regard.

nb. that nvidia-smi will be "using" (ie. waking) the GPU - do you maybe also run any frequent scripts to poll it for sensor data (conky, temperature, GPU load indicators etc)?

I should note that I'm also dynamically binding and unbinding the dedicated GPU for VFIO usage, by default the GPU starts with the vfio modules taking over and then it will be restored back to the NVIDIA kernel module using a systemd service that runs on boot.
I also usually at home use a external monitor via HDMI (directly to the dedicated), however even when I'm outside (so, not using a external monitor nor plugged in to power) it never enters D3 state.

Focus the analysis on when the GPU is neither passed through nor used by any outputs (the latter will require the GPU to be active and when passed through you should not even be able to query it?)

Thanks, I wasn't sure nor could find any hints on how to get code blocks in the forum, guess I just didn't dig deep enough.

Of course, I've done all my checks when the GPU is not being passthrough or used as a output. All the commands I've ran at op is without the HDMI plugged nor is passed through (nvidia modules loaded). And I believe I do not have anything that runs to check sensors of the GPU when I did those commands (+ on a fresh boot)

I should note that I ran the runtime_suspended_time commands before I did nvidia-smi... AFAIK I should still be able to query the power status even if the GPU is on D3?

Last edited by RadiatedExodus (2025-05-11 15:13:18)

seth

Member

Registered: 2012-09-03

Posts: 63,460

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

cat /sys/bus/pci/devices/0000\:01\:00.0/power*

is fine and not supposed to wake anything

Is this nvidia or nvidia-open?
If the latter, try the former. If the former try to disable the https://wiki.archlinux.org/title/NVIDIA … P_firmware

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

topcat01

Member

Registered: 2019-09-17

Posts: 167

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

I noticed that persistence mode is enabled on your gpu (in nvidia-smi). This is controlled by nvidia-persistenced.service and will keep the card on.

RadiatedExodus

Member

Registered: 2021-11-12

Posts: 5

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

@seth @topcat01

Thanks, I've also tried disabling the GSP firmware, however AFAIK the persistence daemon shouldn't be keeping the GPU active, I've used it on other NVIDIA laptops and they do not cause the GPU to stay up permanently...

A new discovery I made, apparently plasmashell is the cause of the GPU staying awake:

# lsof {/dev/dri/card1,/dev/dri/renderD129,/dev/nvidia0}
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
      Output information may be incomplete.
COMMAND    PID                USER  FD   TYPE DEVICE SIZE/OFF NODE NAME
nvidia-pe  970 nvidia-persistenced   3u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe  970 nvidia-persistenced   5u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe  970 nvidia-persistenced   6u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe  970 nvidia-persistenced   8u   CHR  195,0      0t0 1154 /dev/nvidia0
plasmashe 1491          maidenless mem    CHR  195,0          1154 /dev/nvidia0
plasmashe 1491          maidenless  54u   CHR  195,0      0t0 1154 /dev/nvidia0
plasmashe 1491          maidenless  55u   CHR  195,0      0t0 1154 /dev/nvidia0
plasmashe 1491          maidenless  56u   CHR  195,0      0t0 1154 /dev/nvidia0
plasmashe 1491          maidenless  57u   CHR  195,0      0t0 1154 /dev/nvidia0

Every fresh boot, plasmashell always takes the nvidia device file (the gpu is binded back before any graphical interface start)

When I unbind the GPU to VFIO and bind it back to my host, the GPU was finally able to enter D3cold without any issues:

# cat /sys/bus/pci/devices/0000\:01\:00.0/power_state
D3cold

Checking with lsof again, plasmashell stops taking the nvidia dev device after rebinding:

# lsof {/dev/dri/card1,/dev/dri/renderD129,/dev/nvidia0}
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
      Output information may be incomplete.
COMMAND    PID                USER FD   TYPE DEVICE SIZE/OFF NODE NAME
nvidia-pe 3542 nvidia-persistenced 3u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe 3542 nvidia-persistenced 5u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe 3542 nvidia-persistenced 6u   CHR  195,0      0t0 1154 /dev/nvidia0
nvidia-pe 3542 nvidia-persistenced 8u   CHR  195,0      0t0 1154 /dev/nvidia0

Are there any reasons to why plasmashell needs to read to the nvidia device file?

Last edited by RadiatedExodus (2025-05-12 04:47:55)

seth

Member

Registered: 2012-09-03

Posts: 63,460

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

do you maybe also run any frequent scripts to poll it for sensor data (conky, temperature, GPU load indicators etc)?

Some indicator/monitor widget thingy?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

RadiatedExodus

Member

Registered: 2021-11-12

Posts: 5

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

do you maybe also run any frequent scripts to poll it for sensor data (conky, temperature, GPU load indicators etc)?

Some indicator/monitor widget thingy?

I don't use any kind of widgets or indicators to show any kind of system information, when I need to check for system information I either open KDE's System Monitor or use the available cli-tools (usually htop and nvtop).

seth

Member

Registered: 2012-09-03

Posts: 63,460

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

https://discuss.kde.org/t/nvidia-dgpu-w … ll/30719/3

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

RadiatedExodus

Member

Registered: 2021-11-12

Posts: 5

Re: [SOLVED] NVIDIA dGPU never going to D3 power on Lenovo Legion laptop

https://discuss.kde.org/t/nvidia-dgpu-w … ll/30719/3

OH, now I see the issue on my end...

I use the Wallpaper Engine for KDE plugin as my desktop wallpaper, apparently it's still selecting the dedicated GPU to render the video for some reason. After changing the wallpaper and doing a reboot, at fresh boot now the GPU is on D3cold.

Thanks for the help!