NVIDIA Discrete GPU/DGPU powers on on every program that opens.

Fijxu · 2023-03-19 23:07:16

Simple problem, i am currently using a Thinkpad with a 3070 Max-Q on it and i am using RTD3 on it for power usage reasons; But every time that i open a program on KDE Plasma, the program has a delay because it's waiting to the NVIDIA dGPU to be active even if it will not use it.

I attach a demo video about it.

https://i.ayaya.beauty/PKedsi.mp4

Using Wayland, with proprietary NVIDIA drivers (since NVIDIA Open ones doesn't work), mainline up to date kernel using with ibt=off as kernel parameter (to make the proprietary NVIDIA driver to work)

Thanks in advance. If i need to give more details, just say it

Last edited by Fijxu (2023-05-09 19:43:35)

Lone_Wolf · 2023-03-20 10:47:34

Does it work if you are running an X session instead of wayland ?

Fijxu · 2023-03-20 23:26:28

Lone_Wolf wrote:

Does it work if you are running an X session instead of wayland ?

Woops, it works as intended, if i open System settings, the file manager, Firefox or the terminal under X, the GPU doesn't wake up. But for some reason, opening chromium turns on the dGPU (Under KDE Plama Wayland and X11, GNOME Wayland and X11) but i don't use it all the time, anyways is still kinda anoying.

I tried GNOME on Wayland and just some programs wake up (probably because they are using QT and not GTK, not sure) the dGPU, like Firefox and Chromium and Bottles (Probably becuase it uses vulkan and it's checking for GPUs)

I could use KDE X11 but i am using a HIDPI screen and it's very anoying to use X11 on it and also that Wayland is now going mainstream replacing X11.

Then i should change someting on mkinitcpio, modprobe, enviroment variables or what?

Thanks in advance

Last edited by Fijxu (2023-03-20 23:28:01)

Lone_Wolf · 2023-03-21 09:54:59

firefox uses a small part of GTK3 plus their own renderer*, not QT .

Chromium and HW acceleration is not a good combo.

It could be a wayland compositor issue, does Bottles wakeup the dGPU on X ?

* used to be Gecko, but seems to be WebRender now ?

Fijxu · 2023-03-25 22:19:44

Sorry for the late response..

Lone_Wolf wrote:

It could be a wayland compositor issue, does Bottles wakeup the dGPU on X ?

Yes, it wakes up the GPU and then it suspend it after some seconds. I recently upgraded to the latest nvidia version 530.41.03 and it's still not fixed. Every graphical program in wayland wakes up the dGPU delaying the start up of them

Bino_95 · 2023-04-07 06:34:11

I have exactly the same issue on a laptop with AMD iGPU and AMD dGPU (ASUS GA402RJ):

- In GNOME Wayland, opening any app wakes up the dGPU (D3cold->D0) for a few seconds, leading to a lag.
- In GNOME X11, the dGPU stays in D3cold as expected.

So far, I have no idea how to start fixing this, or even where to report it.

PrismaticCuttlefish · 2023-04-11 17:21:33

I also have this issue with an amd iGPU and nvidia dGPU. I've been looking for a solution for a while and can't find one.

gnox · 2023-04-14 00:27:21

The problem is .. nvidia
https://github.com/NVIDIA/libglvnd/blob … eration.md

Otherwise, if the environment variable __EGL_VENDOR_LIBRARY_DIRS is set, it is a colon-separated list of directories. Each directory in turn is scanned for files named *.json and the ICDs that they describe are loaded. Files in the same directory are loaded in strcmp() order, so 40_myvendor.json is considered to be higher-priority than 50_yourvendor.json.
If neither environment variable is set, GLVND behaves as though __EGL_VENDOR_LIBRARY_DIRS had been set to ${sysconfdir}/glvnd/egl_vendor.d:${datadir}/glvnd/egl_vendor.d, replacing ${sysconfdir} and ${datadir} with the values that were set when GLVND was compiled.

So in that directory (/usr/share/glvnd/egl_vendor.d/) when using an iGPU and Nvidia dGPU you have, for example in my case:

$ls /usr/share/glvnd/egl_vendor.d/
... 10_nvidia.json
... 50_mesa.json

10_nvidia.json will be used (see first quote paragraph), making some programs load the nvidia drivers (waking it up) and/or .so (libEGL_nvidia...blah) files even if is not going to be used. The solution/workaround in my case using wayland (KDE or sway) is to set those variables to the iGPU json file before calling the DE/WM, example of my .zprofile (I'm not using Login Manager)

export __EGL_VENDOR_LIBRARY_FILENAMES="/usr/share/glvnd/egl_vendor.d/50_mesa.json"
export __GLX_VENDOR_LIBRARY_NAME="mesa"
exec sway --unsupported-gpu > ~/.sway.log 2>&1
# exec startplasma-wayland

And when I want to use the nvidia I have to set the envs variables pointing to nvidia :

# a prime-run modified
__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/10_nvidia.json __NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only __GLX_VENDOR_LIBRARY_NAME=nvidia "$@"

gnome is the only one that I couldnt make it work with the workaround, gnome-shell keeps grabing the nvidia driver, maybe someone can figure it out how .

edit: mesa should be 00- and nvidia 99999999-

Last edited by gnox (2023-04-14 00:29:54)

Bino_95 · 2023-04-14 18:27:06

gnox wrote:

The problem is .. nvidia
[...]

Thanks for your insights!

Removing the packages nvidia-utils and lib32-nvidia-utils, which provide in particular the EGL libraries you mention, solved the problem for me !

(Even though I have an AMD dGPU, I am using an Arch installation copied over from a system with a nvidia dGPU. I had deleted the nvidia driver but not all the nvidia packages.)

espritlibre · 2023-05-09 19:21:49

i've been also affected by this with intel/nvidia hybrid graphics, gnome/wayland. nearly every app i opened turned the nvidia into D0 state.
thanks for the hint gnox, i partially fixed it in /etc/environment
...
__EGL_VENDOR_LIBRARY_FILENAMES=/usr/share/glvnd/egl_vendor.d/50_mesa.json:/usr/share/glvnd/egl_vendor.d/10_nvidia.json
...
i tested with 10 apps which previously turned the nvidia gpu into D0, now the gpu stays in D3cold state with these apps. chromium and freetube for example still wakes the gpu, but yeah...

Fijxu · 2023-05-09 20:03:55

Thanks gnox, it worked well without problems. Now programs don't try to power up the DGPU (except for Chromium based "apps" which is a pain in the ass, but i can live with it.) I will test it for some days, if i find other problem related to this i will report it here.

I will not mark this as [SOLVED] for now, since other people can have the same problem.

Btw, how i can check if my NVIDIA GPU is in D3Cold power state? Because using nvidia-smi wakes up the GPU.

espritlibre · 2023-05-09 22:21:40

Fijxu wrote:

Btw, how i can check if my NVIDIA GPU is in D3Cold power state? Because using nvidia-smi wakes up the GPU.

watch cat /sys/class/drm/card*/device/power_state

PrismaticCuttlefish · 2023-06-13 03:53:26

espiritlibre's addition to /etc/environment does resolve the issue for me with most programs on gnome, although firefox activates the nvidia gpu for 5-10 seconds when first opening it, then it goes back into d3cold. Chromium (and electron apps) will keep it out of d3cold for the full duration they are open. The nvidia gpu will be in d0 after waking from sleep, also for about 10 seconds before going back into d3cold. Strangely enough, this issue also occurs for me on x11 identically to on wayland (i have no idea why). Has anyone filed a bug report with nvidia about this, or found any fully functional solution in the meantime? I didn't experience this issue at all on or before nvidia version 525.85.05. Perhaps this is unique to my hardware? (amd+nvidia razer blade 14 2022 laptop). I'm not sure.

Last edited by PrismaticCuttlefish (2023-06-13 03:56:51)

mwohah · 2023-07-21 08:58:58

After accidentally creating a duplicate thread about this, I have two peculiarities to report after using the above fix by changing the EGL loader order.

The first is that, as mentioned in the other thread, this 'fixes' a lingering gnome-shell process using 1 MB of VRAM from nvidia-smi that I always thought existed because GNOME needed a process on each GPU for some technical reason.

The other is that, like PrismaticCuttlefish, the issue is now solved for several applications, but not all. What's weird is that Nautilus now no longer causes this behaviour, but e.g. BlackBox does, which is also an 'ordinary' GTK4 application. It has the same behaviour as the other applications where it spins up the NVIDIA GPU according to the power_state files on startup, but when it's already on, the NVIDIA GPU goes back to sleep and isn't evaluated by it again, not even when spawning new windows. The same applied to Nautilus, but only before the fix.

So certainly an improvement, but there seems to be more going on than just the ordering of EGL libraries or some applications are not evaluating them as expected.

I'm also wondering if changing the order in Arch can be done generally as there might be side-effects for other setups where having NVIDIA first is desirable?

Finally, perhaps there is something NVIDIA can do about this? Is it really necessary for the driver to spin up the GPU during evaluation, whilst at that point it's not even certain that an application needs to be rendered by the NVIDIA GPU? Perhaps it is possible for them to delay the wake-up to a later phase.

Last edited by mwohah (2023-07-21 09:00:05)

below-avg · 2023-07-21 21:21:47

espritlibre wrote:

Fijxu wrote:
Btw, how i can check if my NVIDIA GPU is in D3Cold power state? Because using nvidia-smi wakes up the GPU.
watch cat /sys/class/drm/card*/device/power_state

The output says D0 D0 two times :
D0
D0

What does this mean? Is it on or off?

mwohah · 2023-07-22 08:46:54

below-avg wrote:

espritlibre wrote:
Fijxu wrote:
Btw, how i can check if my NVIDIA GPU is in D3Cold power state? Because using nvidia-smi wakes up the GPU.
watch cat /sys/class/drm/card*/device/power_state
The output says D0 D0 two times :
D0
D0
What does this mean? Is it on or off?

Pretty sure D0 means it's on. If it's off, it says e.g. D3Cold.

When it starts to turn off in my case, after about 10 seconds of not being used and in D0, it transitions to D3Hot and then to D3Cold or immediately to D3Cold.

(There might perhaps be more levels depending on the GPU.)

Last edited by mwohah (2023-07-22 08:47:30)

mwohah · 2023-08-03 19:24:17

mwohah wrote:

The other is that, like PrismaticCuttlefish, the issue is now solved for several applications, but not all. What's weird is that Nautilus now no longer causes this behaviour, but e.g. BlackBox does, which is also an 'ordinary' GTK4 application. It has the same behaviour as the other applications where it spins up the NVIDIA GPU according to the power_state files on startup, but when it's already on, the NVIDIA GPU goes back to sleep and isn't evaluated by it again, not even when spawning new windows. The same applied to Nautilus, but only before the fix.

To provide an update on my own peculiarity: it seems that the other applications are coincidentally all Flatpaks. I can reproduce the same problem with GNOME clocks, but if I run the version from the Arch repositories, the problem does not occur.

Is it possible that Flatpaks have their own EGL loader list since they also install special packages for each GL driver and do not rely on the Arch one?

EDIT: It appears so:

/var/lib/flatpak/runtime/org.freedesktop.Platform.GL.default/x86_64/22.08/active/files/glvnd/egl_vendor.d contains 50_mesa.json and;
/var/lib/flatpak/runtime/org.freedesktop.Platform.GL.nvidia-535-86-05/x86_64/1.4/active/files/glvnd/egl_vendor.d contains 10_nvidia.json

EDIT2: It appears the file names, thus also the ordering of the files, are coming from the NVIDIA driver directly (looking at the PKGBUILD in Arch). This would mean we'd ideally plead our case with NVIDIA about this issue. Unfortunately I can't seem to create an account on their forums to do so because they use an allowlist for email domains on registration and my domain is not on it (ugh).

Last edited by mwohah (2023-08-03 19:49:36)

V1del · 2023-08-03 20:01:36

If that's all there is to it and it doesn't regress other use cases (... I could see this being potentially troublesome to simply change the prefix on actual nvidia only/primary systems) then it might be worth a report for the Arch bug tracker to be done as part of the package, alternatively it would be trivial to create e.g. a pacman hook to do the renaming on your own system whenever the nvidia-utils package updates.

mwohah · 2023-09-19 15:37:18

To whom it may concern: apparently the underlying issue is already known upstream.

Last edited by mwohah (2023-09-19 15:37:24)

cyq · 2024-02-21 06:28:27

I only have this problem with Electron applications, even with an empty one created following their getting started guide. So I filed a bug report with Electron: https://github.com/electron/electron/issues/41389.

gulafaran · 2024-02-21 17:33:08

i have worked around all sorts of powering on issues by simply forcing all the environment variables i can to the internal gpu

export DXVK_FILTER_DEVICE_NAME="Intel"
export VKD3D_FILTER_DEVICE_NAME="Intel"
export MESA_VK_DEVICE_SELECT="8086:a788"
export __GLX_VENDOR_LIBRARY_NAME="mesa"
export __EGL_VENDOR_LIBRARY_FILENAMES="/usr/share/glvnd/egl_vendor.d/50_mesa.json"
export __NV_PRIME_RENDER_OFFLOAD="0"
export __VK_LAYER_NV_optimus="non_NVIDIA_only"
export LIBVA_DRIVER_NAME="iHD"
export VDPAU_DRIVER="va_gl"
export WLR_RENDER_DRM_DEVICE="/dev/dri/renderD128"

and then in my own "prime-run" script inside of $PATH

export GBM_BACKEND="nvidia-drm"
export DXVK_FILTER_DEVICE_NAME="NVIDIA"
export VKD3D_FILTER_DEVICE_NAME="NVIDIA"
export __EGL_VENDOR_LIBRARY_FILENAMES="/usr/share/glvnd/egl_vendor.d/10_nvidia.json"
export VK_ICD_FILENAMES="/usr/share/vulkan/icd.d/nvidia_icd.json"
export VK_LOADER_DRIVERS_SELECT="*nvidia*"
export __NV_PRIME_RENDER_OFFLOAD="1"
export __VK_LAYER_NV_optimus="NVIDIA_only"
export __GLX_VENDOR_LIBRARY_NAME="nvidia"
unset MESA_VK_DEVICE_SELECT
"$@"

this way nothing besides lspci and actually touching the dgpu with nvidia-smi or similiar powers it on.
i also have this udev rule, not sure if its needed anymore but its an old remnant from when nvidias (RTD3) Power Management began appearing.

# Enable runtime PM for all pci devices
SUBSYSTEM=="pci", ATTR{power/control}="auto"

# Remove NVIDIA USB xHCI Host Controller devices, if present
ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c0330", ATTR{remove}="1"

# Remove NVIDIA USB Type-C UCSI devices, if present
ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x0c8000", ATTR{remove}="1"

# Remove NVIDIA Audio devices, if present
ACTION=="add", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x040300", ATTR{remove}="1"

# Enable runtime PM for NVIDIA VGA/3D controller devices on driver bind
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="auto"
ACTION=="bind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="auto"

# Disable runtime PM for NVIDIA VGA/3D controller devices on driver unbind
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030000", TEST=="power/control", ATTR{power/control}="on"
ACTION=="unbind", SUBSYSTEM=="pci", ATTR{vendor}=="0x10de", ATTR{class}=="0x030200", TEST=="power/control", ATTR{power/control}="on"

cristatus · 2025-04-11 07:48:58

Anyone facing a similar issue with latest GNOME, forcing GTK4 to use ngl seems to fix the issue. See https://wiki.archlinux.org/title/GTK#GT … mus_setups for more details.

mkdir -p ~/.config/environment.d && echo GSK_RENDERER=ngl > ~/.config/environment.d/gsk.conf

and for flatpak:

flatpak override --user --env=GSK_RENDERER=ngl

However, some flatpak apps are still waking up dGPU on exit. For example, app.devsuite.Ptyxis.

Forcing intel vuklan driver on them fixes the issue:

flatpak override --user --env=VK_DRIVER_FILES=/lib/x86_64-linux-gnu/GL/vulkan/icd.d/intel_icd.x86_64.json app.devsuite.Ptyxis

Last edited by cristatus (2025-04-11 08:04:41)

Lone_Wolf · 2025-04-11 09:18:45

You did notice this thread was started 2 years ago ?

Closing this old thread.

Arch Linux

#1 2023-03-19 23:07:16

NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#2 2023-03-20 10:47:34

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#3 2023-03-20 23:26:28

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#4 2023-03-21 09:54:59

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#5 2023-03-25 22:19:44

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#6 2023-04-07 06:34:11

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#7 2023-04-11 17:21:33

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#8 2023-04-14 00:27:21

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#9 2023-04-14 18:27:06

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#10 2023-05-09 19:21:49

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#11 2023-05-09 20:03:55

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#12 2023-05-09 22:21:40

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#13 2023-06-13 03:53:26

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#14 2023-07-21 08:58:58

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#15 2023-07-21 21:21:47

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#16 2023-07-22 08:46:54

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#17 2023-08-03 19:24:17

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#18 2023-08-03 20:01:36

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#19 2023-09-19 15:37:18

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#20 2024-02-21 06:28:27

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#21 2024-02-21 17:33:08

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#22 2025-04-11 07:48:58

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

#23 2025-04-11 09:18:45

Re: NVIDIA Discrete GPU/DGPU powers on on every program that opens.

Board footer