[PCI Passthrough] NVIDIA reporting incompatible Windows

Heronymous · 2020-11-24 18:15:04

Goal:

To have a Windows 10 guest VM in Virt-Manager with a GPU pass-through on my laptop that I can then view on the same laptop screen using some form of remote desktop viewing (such as, if possible, SPICE on virt-viewer)

Issue:

When attempting to install nvidia drivers into a Windows 10 guest VM on an Archlinux host machine, NVIDIA's Driver Installer (whether that be Geforce Experience or the direct installer itself) keeps reporting:

NVIDIA Installer cannot continue.
This NVIDIA graphics driver is not compatible with this version of Windows.
This graphics driver could not find compatible graphics hardware.

Setup:

HP Envy x360 15t-dr100
Discrete NVIDIA MX250 GPU (dGPU)
Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz with integrated GPU (iGPU)
Arch Linux as host system
Windows 10 Pro as guest system

Context:

I am trying to pass-through an NVIDIA MX250 to a Windows 10 VM using Virt-Manager QEMU/KVM using the guide outlined in PCI_passthrough_via_OVMF.
I have successfully completed the following:

Enabled IOMMU using Kernal Parameters along with other needed parameters

Verified that the dGPU is in its own IOMMU group

element@Lithium ~ % ./bin/check_iommu.sh
IOMMU Group 0:
        00:00.0 Host bridge [0600]: Intel Corporation Device [8086:9b61] (rev 0c)
IOMMU Group 1:
        00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics [8086:9b41] (rev 02)
IOMMU Group 10:
        00:17.0 RAID bus controller [0104]: Intel Corporation 82801 Mobile SATA Controller [RAID mode] [8086:282a]
IOMMU Group 11:
        00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:02b8] (rev f0)
IOMMU Group 12:
        00:1c.4 PCI bridge [0604]: Intel Corporation Device [8086:02bc] (rev f0)
IOMMU Group 13:
        00:1d.0 PCI bridge [0604]: Intel Corporation Device [8086:02b0] (rev f0)
IOMMU Group 14:
        00:1d.4 PCI bridge [0604]: Intel Corporation Device [8086:02b4] (rev f0)
IOMMU Group 15:
        00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:0284]
        00:1f.3 Multimedia audio controller [0401]: Intel Corporation Device [8086:02c8]
        00:1f.4 SMBus [0c05]: Intel Corporation Device [8086:02a3]
        00:1f.5 Serial bus controller [0c80]: Intel Corporation Comet Lake SPI (flash) Controller [8086:02a4]
IOMMU Group 16:
        02:00.0 3D controller [0302]: NVIDIA Corporation GP108M [GeForce MX250] [10de:1d13] (rev a1)
IOMMU Group 17:
        03:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS522A PCI Express Card Reader [10ec:522a] (rev 01)
IOMMU Group 18:
        04:00.0 Non-Volatile memory controller [0108]: Toshiba Corporation Device [1179:011a]
IOMMU Group 2:
        00:04.0 Signal processing controller [1180]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem [8086:1903] (rev 0c)
IOMMU Group 3:
        00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
IOMMU Group 4:
        00:12.0 Signal processing controller [1180]: Intel Corporation Comet Lake Thermal Subsytem [8086:02f9]
IOMMU Group 5:
        00:13.0 Serial controller [0700]: Intel Corporation Comet Lake Integrated Sensor Solution [8086:02fc]
IOMMU Group 6:
        00:14.0 USB controller [0c03]: Intel Corporation Device [8086:02ed]
        00:14.2 RAM memory [0500]: Intel Corporation Device [8086:02ef]
IOMMU Group 7:
        00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9462 [8086:02f0]
IOMMU Group 8:
        00:15.0 Serial bus controller [0c80]: Intel Corporation Serial IO I2C Host Controller [8086:02e8]
        00:15.1 Serial bus controller [0c80]: Intel Corporation Comet Lake Serial IO I2C Host Controller [8086:02e9]
IOMMU Group 9:
        00:16.0 Communication controller [0780]: Intel Corporation Comet Lake Management Engine Interface [8086:02e0]

Isolated the dGPU using kernel parameters

element@Lithium ~ % lspci -nnk -d 10de:1d13                 
02:00.0 3D controller [0302]: NVIDIA Corporation GP108M [GeForce MX250] [10de:1d13] (rev a1)
        Kernel driver in use: vfio-pci
        Kernel modules: nouveau, nvidia_drm, nvidia

Configured the VM, passthrough the GPU, placed XML code to avoid code 43, and placed XML code to avoid "Error 43: Driver failed to load" with mobile (Optimus/max-q) nvidia GPUs"

<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
  <name>win10-2</name>
  <uuid>1cb4bad2-a73f-40dd-8e65-09a3f3d686aa</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">4194304</memory>
  <currentMemory unit="KiB">4194304</currentMemory>
  <vcpu placement="static">8</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-5.1">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/edk2-ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-2_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vendor_id state="on" value="randomid"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on"/>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="raw"/>
      <source file="/var/lib/libvirt/images/win10-2.img"/>
      <target dev="sda" bus="scsi"/>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/element/Torrents/Complete/Windows 10 Pro 19H2 X64 incl Office 2019 en-US NOV 2019 {Gen2}/W10X64.OFF19.ENU.NOV2019.ISO"/>
      <target dev="sdb" bus="scsi"/>
      <readonly/>
      <boot order="2"/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/element/Downloads/virtio-win-0.1.189.iso"/>
      <target dev="sdc" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="2"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="scsi" index="0" model="virtio-scsi">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:c4:7d:30"/>
      <source network="default"/>
      <model type="virtio"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spiceport">
      <source channel="org.spice-space.webdav.0"/>
      <target type="virtio" name="org.spice-space.webdav.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
      <gl enable="no"/>
    </graphics>
    <sound model="ich9">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1b" function="0x0"/>
    </sound>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </hostdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value="-acpitable"/>
    <qemu:arg value="file=/home/element/Public/SSTD1.dat"/>
  </qemu:commandline>
</domain>

Attempted Solutions

I have attempted the solution in Can't install Nvidia driver in VM's Windows 10.. This just lead to the solution pointed to in "Error 43: Driver failed to load" with mobile (Optimus/max-q) nvidia GPUs", which did not help fix the issue.
No code 43 has been observed yet, but it is believed that this would only show up if the NVIDIA driver were actually installed, hence the XML code for that fix has been inserted ahead of time.
The guide pointed to in Use passthrough GPU in KVM/QEMMU and display in host OS in a window, which is [GUIDE] Optimus laptop dGPU passthrough does seem to help with issues to be faced after the NVIDIA driver is installed, but no mention of issues with getting the driver installed itself. However it was noted that in all observed cases that the NVIDIA card had a VGA Compatible Adapter heading while this laptop has a 3D Controller heading. This is believed to be a result of a MUXLESS design, as explained in [GUIDE] Optimus laptop dGPU passthrough
I have attempted to install two different versions of the NVIDIA driver, both the current (457.30) and version 441.2, in case it truly was an issue of compatibility with Windows.

Software Verisons

My host system is Archlinux with the following versions:

element@Lithium ~ % uname -r
5.9.9-arch1-1
element@Lithium ~ % pacman -Q ovmf qemu virt-manager libvirt
edk2-ovmf 202008-1
qemu 5.1.0-3
virt-manager 3.1.0-1
libvirt 1:6.5.0-3

Additional Info

I have tried using different versions of Windows 10 (both the 20H2 and 19H2 versions). I have also tried different configurations of the VM, such as:

With the fake ACPI/Battery code, but no hidden KVM and random Vendor ID
With SATA and VIRTIO SCSI drives (not that I thought this would help, but I was desperate)
Disabling the QXL adapter and SPICE Display just waiting to see if Windows 10 would just magically install the driver.

I have also installed the qemu guest agent and virtio-win drivers, as suggested in Can't install Nvidia driver in VM's Windows 10.. Neither helped with the issue.

Request

If anyone can help me diagnose this issue or point me to the solution that I cannot seem to find, I would greatly appreciate the help.

Last edited by Heronymous (2020-11-25 18:20:03)

Heronymous · 2020-11-26 01:31:35

Update 1

Further reading of [GUIDE] Optimus laptop dGPU passthrough led me to You can now passthrough your dGPU as you wish with an Optimus laptop which finally led me to How to Modify NVIDIA Notebook Driver .INF Files . It is this final link that led me to realize that the hardware ID of the dGPU in Windows showed a SUBSYS_00000000 which leads me to believe that for some reason the Laptop/PC Model and Code Laptop/PC Manufacturer Code are not being properly recognized by Windows. Still looking into this, but wanted to update this post with some progress. My suspicion now is on the idea that these two numbers are needed by the NVIDIA Driver Installer (and even for Windows to install its own driver).

Attempted Solutions

I tried using a VM that uses BIOS instead of UEFI to see if maybe that would get Windows to recognize the proper numbers. This did not work. I'm still looking for a solution.

Additional Info

Found Code 28 which just led back to the "Error 43: Driver failed to load" with mobile (Optimus/max-q) nvidia GPUs"

Last edited by Heronymous (2020-11-26 01:49:38)

Heronymous · 2020-11-26 04:40:31

Update 2: Partial Progress

So following How to Modify NVIDIA Notebook Driver .INF Files up to Step [6], At this step, setting the Model Code and Manufacturer Code to 0000
(so the strings are %NVIDIA_DEV.1D13.0000.0000%=Section039,PCI\VEN_10DE&DEV_1D13&SUBSYS_00000000 and NVIDIA_DEV.1D13.0000.0000="NVIDIA GeForce MX250")
and adding in the new strings to the .INF did cause the NVIDIA Driver Installer to progress beyond the error messages given above. Now the installer reports it just fails to install the drivers. This is expected, as How to Modify NVIDIA Notebook Driver .INF Files stated that the installers need the Model Code and Manufacturer Code to be correct and valid (i.e not 0000) in order for the install process to complete. However, this is now strong evidence that it is the fact that SUBSYS_n(8) needs to be the correct value.

Next step is to figure out how to get the correct string to come up.

Last edited by Heronymous (2020-11-26 04:41:51)

Heronymous · 2020-11-27 15:25:04

Quick Note 1

For anyone that is unsure of what Hardware ID I am talking about, I'm refering to the Hardware ID shown in Windows under Device Manager>3D Controller>Properties>Details>Hardware Ids. Right now it says PCI\VEN_10DE&DEV_1D13&SUBSYS_00000000&REV_A1 and I need it to say PCI\VEN_10DE&DEV_1D13&SUBSYS_86B2103C&REV_A1 inside the VM.

Last edited by Heronymous (2020-11-27 15:26:23)

Heronymous · 2020-11-27 17:09:51

Quick Note 2

The following link seems to be a gold mine of issues people have been having with this process:

Laptop with nvidia gpu passed through, but cannot install drivers, Code 28

jimx · 2020-12-13 14:07:35

Hi, I'm having exactly the same issue on my VM. The device manager shows a wrong subsystem ID and nvidia driver doesn't recognize the GPU. From [GUIDE] Optimus laptop dGPU passthrough I think the problem is the ROM file because it shows an example to masquerade the subsystem ID which just passes a ROM file to the <rom> option in the xml. I tried some ways to extract ROM including using the nvflash program but they didn't work. Did you manage to get it work?

Edit:
I think I get it now. I missed some command line arguments at the end of that example xml file. I read the subsys ID from sysfs and add
<qemu:commandline>
<qemu:arg value='-set'/>
<qemu:arg value='device.hostdev0.x-pci-sub-vendor-id=6058'/>
<qemu:arg value='-set'/>
<qemu:arg value='device.hostdev0.x-pci-sub-device-id=16153'/>
</qemu:commandline>
to my config and it's showing the correct ID now. But I'm still getting error 43 after installing nvidia driver.

Last edited by jimx (2020-12-13 14:22:42)

kiin · 2021-09-07 21:26:17

Heronymous wrote:

Quick Note 2
The following link seems to be a gold mine of issues people have been having with this process:
Laptop with nvidia gpu passed through, but cannot install drivers, Code 28

Hi, were you able to successfully do the passthrough with the MX250? My laptop has the same card, probably in the same fashion as yours, I haven't succeed on this

Arch Linux

#1 2020-11-24 18:15:04