Arch Linux

navidmafi

Member

Registered: 2022-06-07

Posts: 20

[SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

The nvidia card does not automatically drop to lower power states when unutilized.

My device is a laptop with an Intel 11400H + an RTX3050 dGPU (Muxless). I am using Wayland configured to only use the iGPU. I am using modesetting on both cards to have Wayland rendering. I have read most of what I could find about similar issues on Arch forums and elsewhere. I'm certain about the hardware support of this feature. I'm not using nvidia-prime-rtd3pm from AUR.

Driver version: 580.105.08

According to the default specified nvidia docs I shouldn't need any configuration (udev or module parameters) for this to work.

As such, I don't have udev rules targeting the card (such as an 80-nvidia-pm.rules file).

My module params look like this:

# /etc/modprobe.d/nvidia.conf
options nvidia_drm modeset=1 # I would like VK_KHR_wayland_surface for e.g. blender
options nvidia_drm fbdev=0 
options nvidia NVreg_EnableGpuFirmware=0 
options nvidia Nvreg_PreserveVideoMemoryAllocations=1

fbdev param doesn't change results. Included because of https://github.com/NVIDIA/open-gpu-kern … issues/759 which I guess should only be relevant to the open drivers. I'm using the proprietary ones.
NVreg_DynamicPowerManagement=0x03 is not included as it's the default for my platform anyways.
I do rebuild initramfs after every change.

Mkinitcpio hooks are as such:

MODULES=( nvidia nvidia_modeset nvidia_uvm nvidia_drm )

The driver reports fine-grained RTD3 control as available:

cat /proc/driver/nvidia/gpus/0000:01:00.0/power
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

S0ix Power Management:
 Platform Support:          Supported
 Status:                    Disabled

Notebook Dynamic Boost:     Supported

sudo lsof /dev/nvidia* /dev/dri/card1 does not output any open handles.

As a result of modesetting, 1MiB /   4096MiB is reported by nvidia-smi.

I do manually set this to auto because it's "on" by default. I confirm that it persists.

~ cat /sys/bus/pci/devices/0000:01:00.0/power/control
on
~ echo auto | sudo tee /sys/bus/pci/devices/0000:01:00.0/power/control
auto
~ sleep 10; cat /sys/bus/pci/devices/0000:01:00.0/power/control
auto

But:

~ cat /sys/class/drm/card1/device/power_state    
D0

I confirm multiple times that nothing is using the device by unloading all nvidia* kernel modules and reloading them back.
The upsetting fact is that this doesn't even work* with

modprobe nvidia_drm modeset=0

* as much as sysfs is concerned

~ cat /sys/devices/pci0000:00/0000:00:01.0/power_state
D0

Doing all above in a tty - with no graphical interface loaded whatsoever - does not make any difference and the card still has power, which is quite shocking.

Sorry for the long post

Last edited by navidmafi (2025-12-08 14:15:05)

seth

Member

From: Don't DM me only for attention

Registered: 2012-09-03

Posts: 73,317

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

Do you have any output attached to the GPU?

What's the output of nvidia-smi ?
(nb. that nvidia-smi itself will wake the GPU so do not use that to monitor it. It will take a moment for the GPU to enter D3 again afterwards)

Is this btw. on nvidia or nvidia-open?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

No output is attached to the GPU.

Also yes, I don't use nvidia-smi for that exact reason. It shows no processes with 1MB of VRAM usage (presumably because of modeset).

I have tried both with  {nvidia-persistenced, nvidia-powerd} enabled or disabled.

Fri Dec  5 21:28:10 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08             Driver Version: 580.105.08     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3050 ...    On  |   00000000:01:00.0 Off |                  N/A |
| N/A   37C    P8              9W /   60W |       1MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I'm using the proprietary drivers (dkms) on linux-zen. I have since removed nvidia* hooks from mkinitcpio.

Last edited by navidmafi (2025-12-05 23:50:00)

seth

Member

From: Don't DM me only for attention

Registered: 2012-09-03

Posts: 73,317

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

tbc, https://archlinux.org/packages/extra/x86_64/nvidia/ - not https://archlinux.org/packages/extra/x8 … idia-open/
Can you try to add "nvidia.NVreg_EnableGpuFirmware=0" to the https://wiki.archlinux.org/title/Kernel_parameters to make sure it's really applied?

I am using modesetting on both cards to have Wayland rendering.

What's the situation if you only boot the multi-user.target (2nd link below)?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

I have already tried without any GUI whatsoever and I also threw in a cheesy pun haha

Doing all above in a tty - with no graphical interface loaded whatsoever - does not make any difference and the card still has power, which is quite shocking.

Also doing cat /proc/driver/nvidia/params shows

ResmanDebugLevel: 4294967295
RmLogonRC: 1
ModifyDeviceFiles: 1
DeviceFileUID: 0
DeviceFileGID: 0
DeviceFileMode: 438
InitializeSystemMemoryAllocations: 1
UsePageAttributeTable: 4294967295
EnableMSI: 1
EnablePCIeGen3: 0
MemoryPoolSize: 0
KMallocHeapMaxSize: 0
VMallocHeapMaxSize: 0
IgnoreMMIOCheck: 0
EnableStreamMemOPs: 0
EnableUserNUMAManagement: 1
NvLinkDisable: 0
RmProfilingAdminOnly: 1
PreserveVideoMemoryAllocations: 1
EnableS0ixPowerManagement: 0
S0ixPowerManagementVideoMemoryThreshold: 256
DynamicPowerManagement: 3
DynamicPowerManagementVideoMemoryThreshold: 200
RegisterPCIDriver: 1
EnablePCIERelaxedOrderingMode: 0
EnableResizableBar: 0
EnableGpuFirmware: 18
EnableGpuFirmwareLogs: 2
RmNvlinkBandwidthLinkCount: 0
EnableDbgBreakpoint: 0
OpenRmEnableUnsupportedGpus: 1
DmaRemapPeerMmio: 1
ImexChannelCount: 2048
CreateImexChannel0: 0
GrdmaPciTopoCheckOverride: 0
CoherentGPUMemoryMode: ""
RegistryDwords: ""
RegistryDwordsPerDevice: ""
RmMsg: ""
GpuBlacklist: ""
TemporaryFilePath: "/var/tmp"
ExcludedGpus: ""

does this mean I messed up? EnableGpuFirmware is 18 instead of 0.

seth

Member

From: Don't DM me only for attention

Registered: 2012-09-03

Posts: 73,317

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

EnableGpuFirmware is 18 instead of 0.

It certainly looks wrong.
Try the kernel commandline parameter to bypass all questions reg. modprobe and initramfs.

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

I have added your suggested modification to the kernel's cmdline. Still on D0 though.

 ~ sudo lsof /dev/nvidia* /dev/dri/card1
lsof: WARNING: can't stat() fuse.portal file system /run/user/1000/doc
      Output information may be incomplete.
lsof: WARNING: can't stat() fuse.gvfsd-fuse file system /run/user/1000/gvfs
      Output information may be incomplete.
~  sudo nvidia-smi
Sun Dec  7 20:03:02 2025       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.105.08             Driver Version: 580.105.08     CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3050 ...    Off |   00000000:01:00.0 Off |                  N/A |
| N/A   36C    P8              9W /   60W |       1MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
 ~  sleep 30; cat /sys/class/drm/card1/device/power_state
D0
 ~  cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-linux-zen root=UUID=7cd715b5-26d3-4686-903d-187ed69f482f rw loglevel=6 xe.force_probe=9a68 fbcon=font:TER16x32 module_blacklist=i915,usblp hibernate.compressor=lz4 nvidia.NVreg_EnableGpuFirmware=0
 ~   cat /proc/driver/nvidia/params | grep Gpu           
EnableGpuFirmware: 0
EnableGpuFirmwareLogs: 2
OpenRmEnableUnsupportedGpus: 1
GpuBlacklist: ""
ExcludedGpus: ""

Last edited by navidmafi (2025-12-07 16:37:05)

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

I still have runtime D3 support available as reported by

 ~ cat /proc/driver/nvidia/gpus/0000:01:00.0/power     
Runtime D3 status:          Enabled (fine-grained)
Video Memory:               Active

GPU Hardware Support:
 Video Memory Self Refresh: Supported
 Video Memory Off:          Supported

S0ix Power Management:
 Platform Support:          Supported
 Status:                    Disabled

Notebook Dynamic Boost:     Supported
 ~ 

seth

Member

From: Don't DM me only for attention

Registered: 2012-09-03

Posts: 73,317

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

Err… does it work w/ i915 ?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

I just checked. It does not.

power_state         
 ~ cat /proc/cmdline                              
BOOT_IMAGE=/vmlinuz-linux-zen root=UUID=7cd715b5-26d3-4686-903d-187ed69f482f rw loglevel=6 i915.force_probe=9a68 i915.modeset=1 fbcon=font:TER16x32 module_blacklist=xe,usblp hibernate.compressor=lz4 nvidia.NVreg_EnableGpuFirmware=0
 ~ cat /sys/devices/pci0000:00/0000:00:01.0/power_state
D0
 ~ lsmod | grep i915
i915                 5107712  49

seth

Member

From: Don't DM me only for attention

Registered: 2012-09-03

Posts: 73,317

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

"module_blacklist=xe,usblp"
What if you blacklist ucsi_ccg and nvidia_uvm (first is the USB device on the GPU, afaiu for VR devices, the other one is cuda related)

Also maybe post your complete system journal for the boot:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

in case anything shows up that might be dogging on the pci device …

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

navidmafi

Member

Registered: 2022-06-07

Posts: 20

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

Before posting the boot journal here, I read it once for myself in glorious tty1 to see what's included (that would be my first time posting it online).

I saw TLP complaining about how it's not in full effect since its systemd service is not enabled (It indeed wasn't). I wasn't using TLP anyways so I decided to remove it (sheer randomness). Pacman log of today shows:

[2025-12-08T15:41:23+0330] [PACMAN] Running 'pacman -R --config /etc/pacman.conf -- tlp tlpui'
[2025-12-08T15:41:24+0330] [ALPM] transaction started
[2025-12-08T15:41:24+0330] [ALPM] removed tlpui (2:1.9.0-1)
[2025-12-08T15:41:24+0330] [ALPM] warning: /etc/tlp.conf saved as /etc/tlp.conf.pacsave
[2025-12-08T15:41:24+0330] [ALPM] removed tlp (1.9.0-1)
[2025-12-08T15:41:25+0330] [ALPM] transaction completed
[2025-12-08T15:41:25+0330] [ALPM] running '30-systemd-daemon-reload-system.hook'...
[2025-12-08T15:41:25+0330] [ALPM] running '30-systemd-udev-reload.hook'...
[2025-12-08T15:41:26+0330] [ALPM] running '30-systemd-update.hook'...
[2025-12-08T15:41:26+0330] [ALPM] running 'gtk-update-icon-cache.hook'...
[2025-12-08T15:41:26+0330] [ALPM] running 'update-desktop-database.hook'...
[2025-12-08T15:43:19+0330] [PACMAN] Running 'pacman -R -c -n -s --config /etc/pacman.conf -- swww'
[2025-12-08T16:04:29+0330] [PACMAN] Running 'pacman -R -c -n -s --config /etc/pacman.conf -- swww'
[2025-12-08T16:04:33+0330] [ALPM] transaction started
[2025-12-08T16:04:33+0330] [ALPM] removed swww (0.11.2-1)
[2025-12-08T16:04:34+0330] [ALPM] transaction completed
[2025-12-08T16:04:34+0330] [ALPM] running '30-systemd-update.hook'...

And rebooted to a nice surprise

 cat /sys/devices/pci0000:00/0000:00:01.0/power_state              
D3cold

For reference I have also removed all my module blacklists and also using xe (as opposed to i915). EnableGpuFirmware is still 0, though.

I'm not confident that tlp was the problem. However this information might prove useful to someone else.

This problem is now solved, thank you for bearing with me.

Last edited by navidmafi (2025-12-08 14:14:48)

gnox

Member

Registered: 2013-05-18

Posts: 90

Re: [SOLVED] NVIDIA not dropping to D3 on laptop PRIME Setup

when using TLP if you set the udev rules that nvidia recommended you have to add the pci to TLP exclude list :

# Exclude listed PCIe device adresses from Runtime PM.
...
RUNTIME_PM_DENYLIST="01:00.0"

if you didn't set the udev rules, you can enable the auto in TLP with :

# Runtime Power Management for PCIe bus devices: on=disable, auto=enable.
RUNTIME_PM_ON_AC=auto