You are not logged in.
Hello,
I have a laptop with a GeForce RTX 3060 GPU and integrated Intel graphics. I have followed the instructions for PRIME in the wiki and the NVIDIA documentation and I also have dynamic power management active. My problem is that the behaviour of my system doesn't match the behaviour described in those articles.
In my case the system seems to try to be "smart" and handles choosing whether applications run on the iGPU or the dGPU by itself. For example, if I am on AC power, the system seems to default to keep the dGPU powered at all times (though it still runs many low graphics programs on the iGPU). When I go onto battery, it will power down the dGPU if it isn't being used. However, no matter the situation, the system always runs anything using vulkan on the dGPU. For example, if I run vkcube it will always run on the NVIDIA GPU no matter whether I use prime-run or not. However, if I run glxgears by itself it will run on the iGPU and with prime-run it will run on the dGPU (expected behaviour).
Ultimately, I want to be able to control when applications run on the dGPU. In my ideal situation the dGPU would remain powered off, even on battery unless I run an application with prime-run.
At the moment, I have been experiencing problems with an application using vulkan and I really want to test it on the iGPU but no matter how I call the application it always gets sent to the dGPU.
Any advice on how to get PRIME to give me full control would be appreciated.
Last edited by cecure (2022-10-07 12:20:51)
Offline
Do you have vulkan-intel installed so that there even is any other choice? IF you have you can enforce that by explicitly setting the VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json:usr/share/vulkan/icd.d/intel_icd.i686.json environment variable so that it can only find the intel one, or use some tool that shifts this appropriately like https://aur.archlinux.org/packages/vkdevicechooser
Generally vulkan allows to find and enumerate multiple devices by design and it's normally expected the application itself has an option to select which to use. But since this won't be a given you'll need to force things with these environment variables.
Last edited by V1del (2022-10-05 15:22:07)
Offline
Do you have vulkan-intel installed so that there even is any other choice? IF you have you can enforce that by explicitly setting the VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json:usr/share/vulkan/icd.d/intel_icd.i686.json environment variable so that it can only find the intel one, or use some tool that shifts this appropriately like https://aur.archlinux.org/packages/vkdevicechooser
Generally vulkan allows to find and enumerate multiple devices by design and it's normally expected the application itself has an option to select which to use. But since this won't be a given you'll need to force things with these environment variables.
Okay, setting that environment variable before launching vkcube forces it onto the iGPU so at least this is a good strategy for me to test things on the iGPU as needed.
Do you know why this behaviour of needing to be forced to only see the intel card is present ie why running:
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=non_NVIDIA_only vkcube
does not run vkcube on the iGPU as expected based on the instructions from NVIDIA (http://us.download.nvidia.com/XFree86/L … fload.html).
I am wondering if there is something that I need to do to get normal Prime offloading behaviour.
Offline
Not sure, that should work as well. compare vulkaninfo --summary with/without the variables?
Offline
Not sure, that should work as well. compare vulkaninfo --summary with/without the variables?
vulkaninfo --summary ~
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
==========
VULKANINFO
==========
Vulkan Instance Version: 1.3.226
Instance Extensions: count = 20
-------------------------------
VK_EXT_acquire_drm_display : extension revision 1
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_portability_enumeration : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
Instance Layers: count = 5
--------------------------
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.3.205 version 1
VK_LAYER_VALVE_steam_fossilize_32 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_fossilize_64 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_32 Steam Overlay Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_64 Steam Overlay Layer 1.3.207 version 1
Devices:
========
GPU0:
apiVersion = 4206797 (1.3.205)
driverVersion = 2161311744 (0x80d30000)
vendorID = 0x10de
deviceID = 0x2520
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = NVIDIA GeForce RTX 3060 Laptop GPU
driverID = DRIVER_ID_NVIDIA_PROPRIETARY
driverName = NVIDIA
driverInfo = 515.76
conformanceVersion = 1.3.1.0
deviceUUID = e40b19f4-f60b-e0f9-f1b7-15b0fca204ab
driverUUID = 0d91d1a5-7bc0-5373-923a-1f40774e46a8
GPU1:
apiVersion = 4206803 (1.3.211)
driverVersion = 92278791 (0x5801007)
vendorID = 0x8086
deviceID = 0x46a6
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Intel(R) Graphics (ADL GT2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 22.1.7
conformanceVersion = 1.3.0.0
deviceUUID = 3fc767c9-e985-a63e-93d2-cc522c14963b
driverUUID = 4bb42d09-c7a1-618d-f1d1-05403a44aa75
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json:usr/share/vulkan/icd.d/intel_icd.i686.json vulkaninfo --summary ~
ERROR: [Loader Message] Code 0 : loader_get_json: Failed to open JSON file usr/share/vulkan/icd.d/intel_icd.i686.json
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
==========
VULKANINFO
==========
Vulkan Instance Version: 1.3.226
Instance Extensions: count = 20
-------------------------------
VK_EXT_acquire_drm_display : extension revision 1
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_portability_enumeration : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
Instance Layers: count = 5
--------------------------
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.3.205 version 1
VK_LAYER_VALVE_steam_fossilize_32 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_fossilize_64 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_32 Steam Overlay Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_64 Steam Overlay Layer 1.3.207 version 1
Devices:
========
GPU0:
apiVersion = 4206803 (1.3.211)
driverVersion = 92278791 (0x5801007)
vendorID = 0x8086
deviceID = 0x46a6
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Intel(R) Graphics (ADL GT2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 22.1.7
conformanceVersion = 1.3.0.0
deviceUUID = 3fc767c9-e985-a63e-93d2-cc522c14963b
driverUUID = 4bb42d09-c7a1-618d-f1d1-05403a44aa75
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=non_NVIDIA_only vulkaninfo --summary ~
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
==========
VULKANINFO
==========
Vulkan Instance Version: 1.3.226
Instance Extensions: count = 20
-------------------------------
VK_EXT_acquire_drm_display : extension revision 1
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_portability_enumeration : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
Instance Layers: count = 5
--------------------------
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.3.205 version 1
VK_LAYER_VALVE_steam_fossilize_32 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_fossilize_64 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_32 Steam Overlay Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_64 Steam Overlay Layer 1.3.207 version 1
Devices:
========
GPU0:
apiVersion = 4206803 (1.3.211)
driverVersion = 92278791 (0x5801007)
vendorID = 0x8086
deviceID = 0x46a6
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Intel(R) Graphics (ADL GT2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 22.1.7
conformanceVersion = 1.3.0.0
deviceUUID = 3fc767c9-e985-a63e-93d2-cc522c14963b
driverUUID = 4bb42d09-c7a1-618d-f1d1-05403a44aa75
GPU1:
apiVersion = 4206797 (1.3.205)
driverVersion = 2161311744 (0x80d30000)
vendorID = 0x10de
deviceID = 0x2520
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = NVIDIA GeForce RTX 3060 Laptop GPU
driverID = DRIVER_ID_NVIDIA_PROPRIETARY
driverName = NVIDIA
driverInfo = 515.76
conformanceVersion = 1.3.1.0
deviceUUID = e40b19f4-f60b-e0f9-f1b7-15b0fca204ab
driverUUID = 0d91d1a5-7bc0-5373-923a-1f40774e46a8
As you will see, when I run with
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=non_NVIDIA_only
, the Intel card is assigned GPU0 but still things like vkcube run on the NVIDIA card (at least according to nvtop)
Offline
ERROR: [Loader Message] Code 0 : loader_get_json: Failed to open JSON file usr/share/vulkan/icd.d/intel_icd.i686.json
Install lib32-vulkan-intel to fix that.
You only have valve and nvidia layers installed for vulkan , none for mesa / your intel card .
Check the optional dependencies for vulkan-intel .
Last edited by Lone_Wolf (2022-10-06 12:11:25)
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Offline
Well from these outputs the logical behaviour of swapping the presented GPU indices seems to work correctly. does the process you see in nvtop actually generate any noticeable load on the dGPU (from what I'm guessing it shouldn't)
Offline
ERROR: [Loader Message] Code 0 : loader_get_json: Failed to open JSON file usr/share/vulkan/icd.d/intel_icd.i686.json
Install lib32-vulkan-intel to fix that.
You only have valve and nvidia layers installed for vulkan , none for mesa / your intel card .
Check the optional dependencies for vulkan-intel .
I already had vulkan-intel installed, I tried installing lib32-vulkan-intel and the command
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=non_NVIDIA_only vkcube
still runs on the dGPU
Offline
Well from these outputs the logical behaviour of swapping the presented GPU indices seems to work correctly. does the process you see in nvtop actually generate any noticeable load on the dGPU (from what I'm guessing it shouldn't)
Just checked and running vkcube by itself or with
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=non_NVIDIA_only vkcube
does put a load in nvtop under the GPU column (varies between 10%-30%) next to the vkcube process.
If I run with
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json:usr/share/vulkan/icd.d/intel_icd.i686.json vkcube
It will run on the Intel device but the expected behaviour in Prime offloading is that it will run by default on Intel and require user intervention to run on the GPU. I have consulted the latest NVIDIA documentation and cannot find any explanation for this behaviour.
Last edited by cecure (2022-10-06 13:14:26)
Offline
You only have valve and nvidia layers installed for vulkan , none for mesa / your intel card .
Check the optional dependencies for vulkan-intel .
It seems I wasn't clear enough, 2nd attempt
Both vulkaninfo --summary outputs show you have 5 vulkan layers installed .
Instance Layers: count = 5
--------------------------
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.3.205 version 1
VK_LAYER_VALVE_steam_fossilize_32 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_fossilize_64 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_32 Steam Overlay Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_64 Steam Overlay Layer 1.3.207 version 1
The first layer is from nvidia, the other 4 are from valve ((steam) .
There are no vulkan layers present that deal with the intel videocard.
To figure out what package you'are missing, run
$ pacman -Qi vulkan-intel
and look at its optional dependencies .
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Offline
To figure out what package you'are missing, run
$ pacman -Qi vulkan-intel
and look at its optional dependencies .
Thanks for the clarification. I ran it and then installed vulkan-mesa-layers (the only optional dependency).
My default vulkaninfo summary (ie no environment variables) is now:
vulkaninfo --summary ~
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0
==========
VULKANINFO
==========
Vulkan Instance Version: 1.3.226
Instance Extensions: count = 20
-------------------------------
VK_EXT_acquire_drm_display : extension revision 1
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_portability_enumeration : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
Instance Layers: count = 8
--------------------------
VK_LAYER_INTEL_nullhw INTEL NULL HW 1.1.73 version 1
VK_LAYER_MESA_device_select Linux device selection layer 1.3.211 version 1
VK_LAYER_MESA_overlay Mesa Overlay layer 1.3.211 version 1
VK_LAYER_NV_optimus NVIDIA Optimus layer 1.3.205 version 1
VK_LAYER_VALVE_steam_fossilize_32 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_fossilize_64 Steam Pipeline Caching Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_32 Steam Overlay Layer 1.3.207 version 1
VK_LAYER_VALVE_steam_overlay_64 Steam Overlay Layer 1.3.207 version 1
Devices:
========
GPU0:
apiVersion = 4206803 (1.3.211)
driverVersion = 92278791 (0x5801007)
vendorID = 0x8086
deviceID = 0x46a6
deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU
deviceName = Intel(R) Graphics (ADL GT2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 22.1.7
conformanceVersion = 1.3.0.0
deviceUUID = 3fc767c9-e985-a63e-93d2-cc522c14963b
driverUUID = 4bb42d09-c7a1-618d-f1d1-05403a44aa75
GPU1:
apiVersion = 4206797 (1.3.205)
driverVersion = 2161311744 (0x80d30000)
vendorID = 0x10de
deviceID = 0x2520
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = NVIDIA GeForce RTX 3060 Laptop GPU
driverID = DRIVER_ID_NVIDIA_PROPRIETARY
driverName = NVIDIA
driverInfo = 515.76
conformanceVersion = 1.3.1.0
deviceUUID = e40b19f4-f60b-e0f9-f1b7-15b0fca204ab
driverUUID = 0d91d1a5-7bc0-5373-923a-1f40774e46a8
Still no change though, even though vulkan lists the NVIDIA GPU as the second entry, things like vkcube still run on the NVIDIA GPU by default and even when __VK_LAYER_NV_optimus=non_NVIDIA_only
Offline
I did some investigating and your "issue" is due to vkcube specifically and actively seeking out a discrete gpu to render to unless you override that via the gpu_number parameter. https://github.com/KhronosGroup/Vulkan- … be.c#L3419
Moral of the story, applications can do whatever, layers/GPU ordering are a suggestion not a requirement, if you want to be absolutely certain, omit one or the other driver via the VK_ICD_FILENAMES param.
Last edited by V1del (2022-10-07 11:18:32)
Offline
I did some investigating and your "issue" is due to vkcube specifically and actively seeking out a discrete gpu to render to unless you override that via the gpu_number parameter. https://github.com/KhronosGroup/Vulkan- … be.c#L3419
That certainly explains the behaviour, I imagine that steam is probably doing something similar (the other place I have seen this behaviour). Thanks so much for looking into it.
Offline
Steam (... or rather DXVK/VKD3D if you mean proton) certainly also does this yes.
Offline