Arch Linux

danni140c

Member

Registered: 2025-04-12

Posts: 6

VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

Hey, I have a system with a RTX 4080 Super and a GTX 980 ti.

I am trying to passthrough the 4080S to a Windows 10 VM while using the 980 ti on the host. I used to have a similar setup working, using a GTX 750 ti on the host and a GTX 980 ti for passthrough, but I recently upgraded.

At first I was using the nouveau driver, which seemed to work fine-ish for loading vfio driver for the 4080S and nouveau for the 980 ti. Using this setup I was able to start my Win 10 VM with GPU passthrough, but there were issues with using Looking Glass, my cursor would have significant delay and Looking Glass would use llvmpipe as renderer. I was not able to find a solution to the llvmpipe and cursor delay issue, so I thought I’d try using the NVIDIA drivers.

I uninstalled nouveau drivers and installed NVIDIA drivers. Now when I boot using vfio-pci.ids=10de:2702,10de:22bb kernel parameter I have issues, if I remove the vfio-pci.ids parameter, everything seems to be working, in the sense that NVIDIA drivers are loaded and I can use both GPUs in X11 with i3, but I of course can not passthrough my 4080S.
The issue when using the vfio-pci.ids parameter, is that I get some very scrambled video output on the 980 ti. I am able to login, though if I run startx I lose video output on the 980 ti (monitor says no signal). The video output is so scrambled you can’t tell what is shown on screen, but I can type and run commands.

I included some images and logs. Please let me know if there is any other information I should provide. Any help, assistance and suggestions are appreciated! Thank you in advance!

dmesg output WITHOUT vfio-pci.ids kernel parameter: https://termbin.com/fud0

dmesg output WITH vfio-pci.ids kernel parameter: https://termbin.com/sf7e

Images showing output on both cards when using vfio-pci.ids kernel parameter: https://imgur.com/a/LfOptCE

I made the same post on https://forum.level1techs.com/t/vfio-pa … sue/229168

Last edited by danni140c (2025-04-12 14:28:17)

cryptearth

Member

Registered: 2024-02-03

Posts: 1,411

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

one option could be the nvidia driver is too new for the 900-series
which driver do you use EXACTLY? 900-series is maxwell and with lts kernel standard nvidia-lts should be the right one
otherwise it could ve the 900-series and 40-series are just too far apart and hence one driver can't handle both gpus at once properly
also: what's the rest of the hardware? board and cpu specifically
could be the 900-series is connected via the chipset which can cause the issues

danni140c

Member

Registered: 2025-04-12

Posts: 6

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

Thank you for the response, much appreciated.

These are the nvidia packages I have installed

sudo pacman -Qs | grep nvidia
local/nvidia-lts 1:570.133.07-5
local/nvidia-settings 570.133.07-1
local/nvidia-utils 570.133.07-1

Just to clarify, if I was not clear in my original post, I do NOT want to use the NVIDIA driver for both GPUs, I want to use NVIDIA driver for the 980 ti and vfio driver for the 4080 Super, so I can passthrough the 4080 Super.

The motherboard is a ProArt X870E-CREATOR WIFI.
MB specs: https://www.asus.com/dk/motherboards-co … /techspec/
MB manual: https://dlcdnets.asus.com/pub/ASUS/mb/S … TOR%20WIFI

The CPU is an AMD Ryzen 9 9950X.

I do not believe the 900-series is connected via the chipset, as far as I can tell.

Last edited by danni140c (2025-04-12 13:42:15)

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 13,607

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

The closed 570 driver should be ok for the GTX 980, both logs show signs of simpledrm getting initialised .

Nvidia doesn't cooperate well with simpledrm .

Try adding nvidia_drm.modeset=1 nvidia_drm.fbdev=1 as kernel boot parameters .

Moderator Note :
Please don't use pastebin.com , see https://wiki.archlinux.org/title/List_o … n_services for reasons & alternatives.

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

danni140c

Member

Registered: 2025-04-12

Posts: 6

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

Thank you for letting me know, about the pastebin services. I edited the original post to use termbin.com instead.

I will try the kernel boot parameters you suggested and report back.

danni140c

Member

Registered: 2025-04-12

Posts: 6

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

I tried adding the kernel boot parameters suggested, I still got the scrambled video output.

Here is dmesg from when I used the extra nvidia_drm parameters AND vfio-pci.ids: https://termbin.com/u0sa

here is dmesg from my current boot, also using the extra nvidia_drm parameters but without vfio-pci.ids, in case this is useful: https://termbin.com/7ahk

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 13,607

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

It seems your processor has a builtin amdgpu, making this a trio videocard setup .

Please post the full output of lspci -knn  so we can verify that.

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

danni140c

Member

Registered: 2025-04-12

Posts: 6

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

I did enable integrated GPU in the BIOS recently (a day or two ago, while trying to make this work), I wanted to see if I could get any output from the iGPU while using vfio-pci.ids, but had no luck with this. I did not disable it again afterward.

Output of lspci -knn is here: https://termbin.com/1xkx

Last edited by danni140c (2025-04-12 15:23:55)

cryptearth

Member

Registered: 2024-02-03

Posts: 1,411

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

as the two x16 slots are in either primary x16 / secondary x0 or both x8 it doesn't matter which gpu is set into which slot
from the lspci you have placed the 4080 into the primary slot and the 980 into the secondary slot - which makes the 4080 the primary framebuffer target - which breaks as soon as the kernel boots
have you tried to swap the cards physical? as in: place the 980 in the primary slot and the 4080 in the secondary - this way the 4080 should not be loaded at all which is what you want for a passthru while the 980 as primary framebuffer won't break on boot but should show a regular boot sequence
as for the iGPU: could be the bios just kills it when it detects a dGPU - according to the bios manual you seem to be able to both set the igpu as primary and force it even if a dGPU is installed - maybe this could help if you just use the iGPU and remove the 980 entirely (at least to see if you can get it working at all) - if this works you should be able to disablethe iGPU, re-add the 980 in the primary slot and ues it as primary gpu
also: have you enabled re-bar? I'm not sure but I guess the 980 doesn't support it - so enabling it could maybe cause issues while disabling will decrease the 4080 performance - again: these two cards are several generations apart - it could be the 980 just being too old for the rest of the platform and for what you want to use it for

danni140c

Member

Registered: 2025-04-12

Posts: 6

Re: VFIO passthrough, dual GPU issue, scrambled video output, maybe driver

I now tried swapping the GPUs physically, so the 980 ti is now in the first/primary slot (closest to CPU on MB) and the 4080S in the secondary slot. I also disabled the iGPU in BIOS.
This did not seem to help, I no longer get the scrambled output, I now only get the "Starting grubx64.efi Using load options" (same as in the image with scrambled video output), but I get this on the 4080S instead of the 980 ti. I also the text "Booting a command list Loading Linux linux-lts... Loading initial ramdisk...", but I get this on the 980 ti instead of the 4080S.
If I remove the vfio-pci.ids from my kernel boot parameters, the same thing as described above happens, I am no longer able to login and start xorg, as I was previously when I removed vfio-pci.ids from my kernel boot parameters.

I was curious to see what happened, if I changed the kernel boot parameter to load the vfio driver on the 980 ti, so using vfio-pci.ids=10de:17c8,10de:0fb0 (IDs for 980 ti) instead of vfio-pci.ids=10de:2702,10de:22bb (IDs for 4080S). This actually worked as expected. I was able to login and start xorg using the 4080S and I could passthrough the 980 ti to my VM. So using the GPUs opposite of what I want seems to work fine (i.e. passing through the 980 ti and using the 4080S on the host works, other way around does not).
See https://termbin.com/elct for IOMMU groups for the GPUs.
Output from dmesg when using the 980 ti for passthrough with vfio drivers: https://termbin.com/u0sa
Output from lspci -knn when using the 980 ti for passthrough with vfio drivers: https://termbin.com/2lxj

I have no idea why it works when I try using the GPUs opposite of how I want.

Tomorrow I might try removing the 980 ti and use the iGPU for the host and the 4080S for passthrough/vfio drivers and see how that works.

Last edited by danni140c (2025-04-12 21:26:49)