[SOLVED] No Output From GPU via OVMF PCI Passthrough

Caelence · 2023-09-06 22:59:05

Hi there,

I must admit that I am new to trying PCI passthrough, and have little to no experience with it. I do, however, have some experience with Arch Linux, so I'm not a total beginner here.

Here are my system specs:

CPU: AMD Ryzen 9 7900X
MOBO: MSI MPG X670E Carbon Wifi Gaming
RAM: G.SKILL Trident Z5 Neo DDR5 64GB (2x32GB)
GPU: ASRock Radeon RX 7900XTX Taichi

Just to give more info for anyone that is unaware, this CPU has integrated graphics.

What I'm trying to do/info:

While the integrated GPU certainly isn't the best thing to use for graphics, it's honestly all I need to be able to use Arch, scroll the web, run some programs on Arch, and do basic things.
Most of the heavy lifting that I plan on doing will be in Windows.
My plan was to use Arch Linux on the iGPU(7900X integrated GPU), and Windows 11 with the dGPU(7900XTX dedicated GPU).
I am not using any modified kernel for my Arch Linux install. I'm just running the latest kernel.

What I have tried so far:

Tried following the PCI passthrough via OVMF Guide as best as I could.
Updating my host system via Pacman.
Checked that my GPU is indeed using vfio-pci drivers.
Completely removing every step done from previous guide mentioned, and tried to do it again (a few times).
Rebooting. Lots of times...
Searching the Arch forums for similar problems.
Disabling Re-sizable bar and/or Above 4G Decoding.
Initiate Graphic Adapter from PEG to IGD.
Initiate Graphic Adapter from IGD to PEG, opening a new tty and logging in, and starting the VM via virsh. (still nothing)
Completely reinstalling Arch Linux.
Disabling Re-sizable bar in the UEFI.
Disabling Above 4G Decoding in the UEFI
Adding "<vendor_id state='on' value='123456789ab'/>" to the XML.
Adding "<kvm><hidden state='on'/></kvm>" to the XML
Ripping out my hair.

Technical INFO

Checking DMESG that IOMMU is enabled:

[username@hostname ~]$ sudo dmesg | grep -i -e DMAR -e IOMMU
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=5bdd24f1-8c88-4829-8200-ff09f4353a18 rw loglevel=3 quiet amd_iommu=on iommu=pt vfio-pci.ids=1002:744c,1002:ab30
[    0.044697] Kernel command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=5bdd24f1-8c88-4829-8200-ff09f4353a18 rw loglevel=3 quiet amd_iommu=on iommu=pt vfio-pci.ids=1002:744c,1002:ab30
[    0.436776] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.462858] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.462900] pci 0000:00:01.0: Adding to iommu group 0
[    0.462913] pci 0000:00:01.1: Adding to iommu group 1
[    0.462924] pci 0000:00:01.2: Adding to iommu group 2
[    0.462942] pci 0000:00:02.0: Adding to iommu group 3
[    0.462954] pci 0000:00:02.1: Adding to iommu group 4
[    0.462965] pci 0000:00:02.2: Adding to iommu group 5
[    0.462982] pci 0000:00:03.0: Adding to iommu group 6
[    0.462998] pci 0000:00:04.0: Adding to iommu group 7
[    0.463018] pci 0000:00:08.0: Adding to iommu group 8
[    0.463029] pci 0000:00:08.1: Adding to iommu group 9
[    0.463040] pci 0000:00:08.3: Adding to iommu group 10
[    0.463063] pci 0000:00:14.0: Adding to iommu group 11
[    0.463074] pci 0000:00:14.3: Adding to iommu group 11
[    0.463137] pci 0000:00:18.0: Adding to iommu group 12
[    0.463148] pci 0000:00:18.1: Adding to iommu group 12
[    0.463159] pci 0000:00:18.2: Adding to iommu group 12
[    0.463169] pci 0000:00:18.3: Adding to iommu group 12
[    0.463180] pci 0000:00:18.4: Adding to iommu group 12
[    0.463191] pci 0000:00:18.5: Adding to iommu group 12
[    0.463201] pci 0000:00:18.6: Adding to iommu group 12
[    0.463213] pci 0000:00:18.7: Adding to iommu group 12
[    0.463224] pci 0000:01:00.0: Adding to iommu group 13
[    0.463235] pci 0000:02:00.0: Adding to iommu group 14
[    0.463254] pci 0000:03:00.0: Adding to iommu group 15
[    0.463268] pci 0000:03:00.1: Adding to iommu group 16
[    0.463280] pci 0000:04:00.0: Adding to iommu group 17
[    0.463291] pci 0000:05:00.0: Adding to iommu group 18
[    0.463302] pci 0000:06:00.0: Adding to iommu group 19
[    0.463313] pci 0000:06:04.0: Adding to iommu group 20
[    0.463325] pci 0000:06:05.0: Adding to iommu group 21
[    0.463336] pci 0000:06:06.0: Adding to iommu group 22
[    0.463347] pci 0000:06:07.0: Adding to iommu group 23
[    0.463360] pci 0000:06:08.0: Adding to iommu group 24
[    0.463371] pci 0000:06:0c.0: Adding to iommu group 25
[    0.463382] pci 0000:06:0d.0: Adding to iommu group 26
[    0.463384] pci 0000:0c:00.0: Adding to iommu group 24
[    0.463387] pci 0000:0d:00.0: Adding to iommu group 24
[    0.463390] pci 0000:0d:04.0: Adding to iommu group 24
[    0.463393] pci 0000:0d:05.0: Adding to iommu group 24
[    0.463396] pci 0000:0d:06.0: Adding to iommu group 24
[    0.463398] pci 0000:0d:07.0: Adding to iommu group 24
[    0.463401] pci 0000:0d:08.0: Adding to iommu group 24
[    0.463403] pci 0000:0d:0c.0: Adding to iommu group 24
[    0.463406] pci 0000:0d:0d.0: Adding to iommu group 24
[    0.463408] pci 0000:0f:00.0: Adding to iommu group 24
[    0.463411] pci 0000:10:00.0: Adding to iommu group 24
[    0.463414] pci 0000:14:00.0: Adding to iommu group 24
[    0.463417] pci 0000:15:00.0: Adding to iommu group 24
[    0.463420] pci 0000:16:00.0: Adding to iommu group 25
[    0.463422] pci 0000:17:00.0: Adding to iommu group 26
[    0.463433] pci 0000:18:00.0: Adding to iommu group 27
[    0.463457] pci 0000:19:00.0: Adding to iommu group 28
[    0.463469] pci 0000:19:00.1: Adding to iommu group 29
[    0.463480] pci 0000:19:00.2: Adding to iommu group 30
[    0.463492] pci 0000:19:00.3: Adding to iommu group 31
[    0.463503] pci 0000:19:00.4: Adding to iommu group 32
[    0.463515] pci 0000:19:00.6: Adding to iommu group 33
[    0.463526] pci 0000:1a:00.0: Adding to iommu group 34
[    0.463752] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.639571] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    0.700982] AMD-Vi: AMD IOMMUv2 loaded and initialized

Everything seems to be okay, as far as I can tell. Yes, I explicitly set amd_iommu=on iommu=pt in the kernel parameters. I was/am getting desperate.

Checking the IOMMU groups:

[username@hostname ~]$ shopt -s nullglob
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;
IOMMU Group 0:
        00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 1:
        00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 2:
        00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 3:
        00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 4:
        00:02.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 5:
        00:02.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14db]
IOMMU Group 6:
        00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 7:
        00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 8:
        00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14da]
IOMMU Group 9:
        00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14dd]
IOMMU Group 10:
        00:08.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:14dd]
IOMMU Group 11:
        00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 71)
        00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 12:
        00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e0]
        00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e1]
        00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e2]
        00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e3]
        00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e4]
        00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e5]
        00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e6]
        00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Device [1022:14e7]
IOMMU Group 13:
        01:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev 10)
IOMMU Group 14:
        02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479] (rev 10)
IOMMU Group 15:
        03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] [1002:744c] (rev c8)
IOMMU Group 16:
        03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30]
IOMMU Group 17:
        04:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO [144d:a80a]
IOMMU Group 18:
        05:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f4] (rev 01)
IOMMU Group 19:
        06:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 20:
        06:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 21:
        06:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 22:
        06:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 23:
        06:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
IOMMU Group 24:
        06:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f4] (rev 01)
        0d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0d:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        0f:00.0 Network controller [0280]: MEDIATEK Corp. MT7922 802.11ax PCI Express Wireless Network Adapter [14c3:0616]
        10:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8125 2.5GbE Controller [10ec:8125] (rev 05)
        14:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f7] (rev 01)
        15:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f6] (rev 01)
IOMMU Group 25:
        06:0c.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        16:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f7] (rev 01)
IOMMU Group 26:
        06:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f5] (rev 01)
        17:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43f6] (rev 01)
IOMMU Group 27:
        18:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller PM9A1/PM9A3/980PRO [144d:a80a]
IOMMU Group 28:
        19:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raphael [1002:164e] (rev c2)
IOMMU Group 29:
        19:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]
IOMMU Group 30:
        19:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] VanGogh PSP/CCP [1022:1649]
IOMMU Group 31:
        19:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b6]
IOMMU Group 32:
        19:00.4 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b7]
IOMMU Group 33:
        19:00.6 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h/19h HD Audio Controller [1022:15e3]
IOMMU Group 34:
        1a:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:15b8]

I can clearly see that the IOMMU Groups 15 and 16 are the groups that I would want to pass to VMM.

Grub default config:

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet amd_iommu=on iommu=pt vfio-pci.ids=1002:744c,1002:ab30"

After which, I ran

 sudo grub-mkconfig -o /boot/grub/grub.cfg

Loading vfio-pci early is done via mkinitcpio:

...
MODULES=(vfio_pci vfio vfio_iommu_type1)
...
HOOKS=(... modconf ...)

After that, I made sure to run

sudo mkinitcpio -P

and I rebooted my system.

Verifying that the configuration worked

[username@hostname ~]$ sudo dmesg | grep -i vfio
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=5bdd24f1-8c88-4829-8200-ff09f4353a18 rw loglevel=3 quiet amd_iommu=on iommu=pt vfio-pci.ids=1002:744c,1002:ab30
[    0.044697] Kernel command line: BOOT_IMAGE=/vmlinuz-linux root=UUID=5bdd24f1-8c88-4829-8200-ff09f4353a18 rw loglevel=3 quiet amd_iommu=on iommu=pt vfio-pci.ids=1002:744c,1002:ab30
[    2.947247] VFIO - User Level meta-driver version: 0.3
[    3.027721] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    3.027849] vfio_pci: add [1002:744c[ffffffff:ffffffff]] class 0x000000/00000000
[    3.124523] vfio_pci: add [1002:ab30[ffffffff:ffffffff]] class 0x000000/00000000
[   11.527303] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[ 5679.077326] vfio-pci 0000:03:00.0: enabling device (0002 -> 0003)
[ 5679.077607] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x19@0x270
[ 5679.077612] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0
[ 5679.077616] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x26@0x410
[ 5679.077618] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x27@0x450
[ 5679.093736] vfio-pci 0000:03:00.1: enabling device (0000 -> 0002)
[ 5853.567181] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x19@0x270
[ 5853.567188] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x1b@0x2d0
[ 5853.567193] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x26@0x410
[ 5853.567195] vfio-pci 0000:03:00.0: vfio_ecap_init: hiding ecap 0x27@0x450

[username@hostname ~]$ lspci -nnk -d 1002:744c
03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] [1002:744c] (rev c8)
        Subsystem: ASRock Incorporation Navi 31 [Radeon RX 7900 XT/7900 XTX] [1849:5302]
        Kernel driver in use: vfio-pci
        Kernel modules: amdgpu

[username@hostname ~]$ lspci -nnk -d 1002:ab30
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 HDMI/DP Audio [1002:ab30]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

As far as I can tell, my system should be setup so that I can use my 7900XTX with VMM(Virtual Machine Manager).
I made sure to install qemu-desktop, libvirt, edk2-ovmf, virt-manager, and dnsmasq.
I also made sure to enable/start them with systemctl.

Following the guide, I get as far as installing Windows inside of Virtual Machine Manager using the Spice connection.
The moment that I remove the spice devices, add my PCI devices, and start the VM, I constantly get a black screen when trying to switch to the GPU on my monitor. In fact, I'm pretty sure that I don't even get a signal out of the GPU at all, as my monitor automatically switches back to my iGPU after trying and waiting a couple of seconds for a picture.

At first, I thought it was this, however even after adding

 <vendor_id state='on' value='randomid'/>

to my XML, I still don't get any output.
I'm honestly stumped at this point, and have no idea what to do next.

Extra INFO

In my UEFI, I have my iGPU set as the default GPU. There is an option for HybridGraphics in there as well, but enabling it always causes my system to no post, and so I have to clear the CMOS and run again. Changing the default GPU to the dGPU just causes Arch to not load to the desktop, and I assume that's because the dGPU has it's kernel drivers running as vfio, so it makes sense that it doesn't show anything.
CSM is disabled in my motherboard, as well as Secure boot.

XML config of VM:

<domain type="kvm">
  <name>win11</name>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/11"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">33554432</memory>
  <currentMemory unit="KiB">33554432</currentMemory>
  <vcpu placement="static">20</vcpu>
  <os firmware="efi">
    <type arch="x86_64" machine="pc-q35-8.1">hvm</type>
    <firmware>
      <feature enabled="no" name="enrolled-keys"/>
      <feature enabled="yes" name="secure-boot"/>
    </firmware>
    <loader readonly="yes" secure="yes" type="pflash">/usr/share/edk2/x64/OVMF_CODE.secboot.fd</loader>
    <nvram template="/usr/share/edk2/x64/OVMF_VARS.fd">/var/lib/libvirt/qemu/nvram/win11_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vendor_id state="on" value="randomid"/>
    </hyperv>
    <vmport state="off"/>
    <smm state="on"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" cores="10" threads="2"/>
  </cpu>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="block" device="disk">
      <driver name="qemu" type="raw" cache="none" io="native" discard="unmap"/>
      <source dev="/dev/nvme1n1p1"/>
      <target dev="vda" bus="virtio"/>
      <boot order="1"/>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x19"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x1a"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x2"/>
    </controller>
    <controller type="pci" index="12" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="12" port="0x1b"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x3"/>
    </controller>
    <controller type="pci" index="13" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="13" port="0x1c"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x4"/>
    </controller>
    <controller type="pci" index="14" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="14" port="0x1d"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x5"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:0e:50:ce"/>
      <source network="default"/>
      <model type="e1000e"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <tpm model="tpm-crb">
      <backend type="emulator" version="2.0"/>
    </tpm>
    <audio id="1" type="none"/>
    <hostdev mode="subsystem" type="usb" managed="yes">
      <source>
        <vendor id="0x045e"/>
        <product id="0x07fd"/>
      </source>
      <address type="usb" bus="0" port="1"/>
    </hostdev>
    <hostdev mode="subsystem" type="usb" managed="yes">
      <source>
        <vendor id="0xc0f4"/>
        <product id="0x05c0"/>
      </source>
      <address type="usb" bus="0" port="2"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x03" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </hostdev>
    <watchdog model="itco" action="reset"/>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
</domain>

I hope that I have provided enough info for someone to be able to help me.

I just want to know if I'm an idiot or not for thinking that this would work. I understand that the usual case is to have two seperate dedicated GPUs. I just don't see how having one iGPU and one dGPU would make a difference.

Any help will be greatly appreciated.

Thanks,
Caelence

Last edited by Caelence (2024-07-25 05:54:24)

Brut · 2023-09-08 14:55:41

You didn't mention if your monitor is connected to your dGPU but it has to be connected to output anything(it won't pass through the iGPU when you fire up the VM). You could connect another cable to the same monitor and then switch the outputs on the monitor or connect it to another monitor.
Also I don't know if this applies to newer AMD GPUs but you should disable Resizable Bar. If I remember correctly you can enable above 4G Encoding which will still allow you to use Resizeable Bar in Linux but the option to enable SAM should be disabled in BIOS(QEMU doesn't support Resizable Bar yet however will advertise as being available if its enabled in BIOS causing the card to be unusable once you install drivers on the Windows VM).

Caelence · 2023-09-08 16:58:02

Brut wrote:

You didn't mention if your monitor is connected to your dGPU but it has to be connected to output anything(it won't pass through the iGPU when you fire up the VM). You could connect another cable to the same monitor and then switch the outputs on the monitor or connect it to another monitor.
Also I don't know if this applies to newer AMD GPUs but you should disable Resizable Bar. If I remember correctly you can enable above 4G Encoding which will still allow you to use Resizeable Bar in Linux but the option to enable SAM should be disabled in BIOS(QEMU doesn't support Resizable Bar yet however will advertise as being available if its enabled in BIOS causing the card to be unusable once you install drivers on the Windows VM).

Thanks for some suggestions, I appreciate it!
I did forget to mention that I do indeed have a cable running from both the iGPU and the dGPU to my monitor.

Unfortunately, I have also tried enabling and disabling Re-sizable bar a few times in my UEFI, but it doesn't seem to change anything. Not sure if that is because my VM is already setup and created or not. But I shall try creating a fresh VM with Re-sizable bar disabled, and see if I have any luck. I'll post an update once I've done this.

Last edited by Caelence (2023-09-09 01:59:36)

Caelence · 2023-09-09 06:01:42

Brut wrote:

You didn't mention if your monitor is connected to your dGPU but it has to be connected to output anything(it won't pass through the iGPU when you fire up the VM). You could connect another cable to the same monitor and then switch the outputs on the monitor or connect it to another monitor.
Also I don't know if this applies to newer AMD GPUs but you should disable Resizable Bar. If I remember correctly you can enable above 4G Encoding which will still allow you to use Resizeable Bar in Linux but the option to enable SAM should be disabled in BIOS(QEMU doesn't support Resizable Bar yet however will advertise as being available if its enabled in BIOS causing the card to be unusable once you install drivers on the Windows VM).

I can confirm that after disabling both Re-sizable bar and Above 4G Decoding I still don't get an output.
I was also checking out some other forum posts, and saw suggestions to add some lines to the XML.
I tried adding:

<vendor_id state='on' value='123456789ab'/>

But had no luck.
I also tried:

<kvm>
     <hidden state='on'/>
</kvm>

Still had no luck.

Going to update my original post with the extra things that I have tried.

JProdman · 2023-09-09 09:47:18

Have you tried to see this? https://wiki.archlinux.org/title/PCI_pa … one_above)

Caelence · 2023-09-09 10:56:34

JProdman wrote:

Have you tried to see this? https://wiki.archlinux.org/title/PCI_pa … one_above)

Funny enough, I was reading this earlier and thought that it never really related to me. I have since tried out the DKMS version.
This never worked on it's own, but I FINALLY got a picture to display from the image, after literal days of trying. So, I thank you for pointing me in the right direction. It turns out that I had a combination of the exact issue that you mentioned and
this one.
I kept seeing

vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none

popup each time I ran the command:

sudo dmesg | grep -i vfio

I never thought anything of it, and never realized it was causing my headaches.

I also want to say that following the comments from this Reddit thread also helped me to solve this.

I also had to change the graphics in my UEFI to PEG. I had it set to IGD, as I was certain that I was supposed to be booting using my iGPU.

To anyone that has these exact same issues with a 7900XTX, I urge you to try the following:

Make sure that you have your UEFI integrated graphics set to your graphics card and not your integrated graphics. It may make sense to want to change it to your integrated graphics card, especially if you're planning on running your host system with your integrated graphics. For some reason, it just doesn't work, so give it a try.
Check all vfio messages with dmesg, and make sure that you don't have
```
vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
```
when you run the command:
```
sudo dmesg | grep -i vfio
```
If you do, follow this section of the guide. It explains perfectly how to solve the problem. You can also view the Reddit link in this message as well.
Whether or not you get the error described in this section of the post, I seriously urge you to try it. I never had this issue once show up, but I believe that this also solved my problem. I tried to boot the VM without this package, and the second I remove it, everything doesn't display anymore. I did need the previous fixes that I mentioned in conjunction with this one for everything to work.

Thanks to everyone that tried to help me with this. I will now be marking this as solved.

Arch Linux

#1 2023-09-06 22:59:05