You are not logged in.

#1 2019-12-02 12:17:39

thearcher
Member
Registered: 2010-05-13
Posts: 31

[SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Good day to everyone!

I need your help smile

I did this before but never with the Q35 chipset but as far as I know... it should be ok smile

First: The VGA is passing through well as I get VGA signal but on windows 10 it gives Error 43 even when:

I have OVMF UEFI
I have the hidden state to on
I have the ipic driver to kvm

I have qemu args as:
-cpu
-host, kvm=off, hv_vendor_id=0123456789ab
-machine
q35, kernel_irqchip=on

My grub has:
intel_iommu=on,igfx_off iommu=pt vfio-pci.ids=10de:17c8,10de:0fb0intel_iommu=on,igfx_off iommu=pt vfio-pci.ids=10de:17c8,10de:0fb0

In the old days I was passing the VGA bios from TechPowerUp but now if I do the Windows 10 just does not boot (BSOD even when I do not see it)

I also have /etc/modprobe.d/kvm.conf file with:
options kvm ignore_msrs=1

lspci -nk lists:
01:00.0 0300: 10de:17c8 (rev a1)
    Subsystem: 3842:4998
    Kernel driver in use: vfio-pci
    Kernel modules: nouveau
01:00.1 0403: 10de:0fb0 (rev a1)
    Subsystem: 3842:4998
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

journalctl -b | grep -i vfio show:
Command line: BOOT_IMAGE=/boot/vmlinuz-linux root=UUID=b07ef69d-5390-4cfb-bff5-a04c08a9b48d rw loglevel=3 quiet intel_iommu=on,igfx_off iommu=pt vfio-pci.ids=10de:17c8,10de:0fb0
Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux root=UUID=b07ef69d-5390-4cfb-bff5-a04c08a9b48d rw loglevel=3 quiet intel_iommu=on,igfx_off iommu=pt vfio-pci.ids=10de:17c8,10de:0fb0
VFIO - User Level meta-driver version: 0.3
vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none
vfio_pci: add [10de:17c8[ffffffff:ffffffff]] class 0x000000/00000000
vfio_pci: add [10de:0fb0[ffffffff:ffffffff]] class 0x000000/00000000
vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=none,decodes=none:owns=none

The hardware:

VGA: NVIDIA GTX980Ti
Motherboard AsRock Z97 Extreme 6

Please, if you need anything else, just let me know... I really do not know what else to try.

This was working just a year ago or so.

Thanks in advance.

Best regards,


Juanjo

Last edited by thearcher (2019-12-03 18:50:47)


Keep It Simple, Stupid!

Offline

#2 2019-12-02 12:52:46

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

What is the output of the following script?

#!/bin/bash
shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

Also, perhaps try changing your cpu line to:

-cpu host,kvm=off,hypervisor=false,check

Last edited by markzz (2019-12-02 12:54:04)


I don't want to work.  I want to bang on the drum all day.

Offline

#3 2019-12-02 12:59:47

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello,

Thanks for your reply!!

The output is:

IOMMU Group 0:
	00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
IOMMU Group 1:
	00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
	01:00.1 Audio device [0403]: NVIDIA Corporation GM200 High Definition Audio [10de:0fb0] (rev a1)
IOMMU Group 10:
	00:1c.3 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 [8086:8c96] (rev d0)
IOMMU Group 11:
	00:1c.6 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 7 [8086:8c9c] (rev d0)
IOMMU Group 12:
	00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6]
IOMMU Group 13:
	00:1f.0 ISA bridge [0601]: Intel Corporation Z97 Chipset LPC Controller [8086:8cc4]
	00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82]
	00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2]
IOMMU Group 14:
	03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11)
IOMMU Group 15:
	04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 16:
	05:01.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 17:
	05:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
	07:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 18:
	05:05.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 19:
	05:07.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
	09:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 2:
	00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
IOMMU Group 20:
	0a:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 3:
	00:14.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB xHCI Controller [8086:8cb1]
IOMMU Group 4:
	00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba]
IOMMU Group 5:
	00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I218-V [8086:15a1]
IOMMU Group 6:
	00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad]
IOMMU Group 7:
	00:1b.0 Audio device [0403]: Intel Corporation 9 Series Chipset Family HD Audio Controller [8086:8ca0]
IOMMU Group 8:
	00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0)
IOMMU Group 9:
	00:1c.2 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 3 [8086:8c94] (rev d0)

I will try the rest later at home smile


Keep It Simple, Stupid!

Offline

#4 2019-12-02 13:12:50

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Your IOMMU groups are not separated properly and you will likely need to use the ACS override patch. I would recommend using the linux-vfio package in the AUR so that you can have the NVIDIA cards separated from the Intel chip.

It's an all or nothing thing when doing PCI passthrough with the IOMMU groups. EVERYTHING in the group must be passed-through.

An example of my groups:

IOMMU Group 0:
	00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
IOMMU Group 1:
	00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
IOMMU Group 10:
	00:1c.7 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 8 [8086:8c9e] (rev d0)
IOMMU Group 11:
	00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6]
IOMMU Group 12:
	00:1f.0 ISA bridge [0601]: Intel Corporation Z97 Chipset LPC Controller [8086:8cc4]
	00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82]
	00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2]
IOMMU Group 13:
	01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 6770] [1002:68ba]
	01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58]
IOMMU Group 14:
	02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
	02:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU Group 15:
	04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 16:
	05:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 17:
	06:00.0 Ethernet controller [0200]: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) [8086:107d] (rev 06)
IOMMU Group 2:
	00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06)
IOMMU Group 3:
	00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba]
IOMMU Group 4:
	00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I218-V [8086:15a1]
IOMMU Group 5:
	00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad]
IOMMU Group 6:
	00:1b.0 Audio device [0403]: Intel Corporation 9 Series Chipset Family HD Audio Controller [8086:8ca0]
IOMMU Group 7:
	00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0)
IOMMU Group 8:
	00:1c.3 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 [8086:8c96] (rev d0)
IOMMU Group 9:
	00:1c.6 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 7 [8086:8c9c] (rev d0)

Last edited by markzz (2019-12-02 13:15:14)


I don't want to work.  I want to bang on the drum all day.

Offline

#5 2019-12-02 13:56:57

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Thanks sir; I will check the ACS patch but while I'm arriving home... I'm wondering why it was ok before and as far as I have read it should not be a problem as the other device in that group is the processor root PCI(?)

I have read about changing my graphic card to PCIe 2 but I was not sure if doing it as I will need to open my computer and it was working fine last time (year ago or so).

But again... thank you!!


Keep It Simple, Stupid!

Offline

#6 2019-12-02 14:01:29

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

More than likely you changed your hardware configuration (also that's what it sounds like based on your first post) and the kernel isn't separating IOMMU groups as it did with the other hardware.


I don't want to work.  I want to bang on the drum all day.

Offline

#7 2019-12-02 14:11:28

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Interesting as I have not changed my hardware for long time! But I was using only windows for some. Maybe in the old days I patched my kernel and I didn’t remember it... thanks a lot!

I will give a try as soon as I can smile

I really hope to fix this smile


Keep It Simple, Stupid!

Offline

#8 2019-12-03 07:02:09

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Good morning,

I have just tested the ACS patch (took long time to compile smile).

Having grub with: pcie_acs_override=downstream,multifunction

It gives me:

IOMMU Group 0:
	00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
IOMMU Group 1:
	00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
IOMMU Group 10:
	00:1c.3 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 [8086:8c96] (rev d0)
IOMMU Group 11:
	00:1c.6 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 7 [8086:8c9c] (rev d0)
IOMMU Group 12:
	00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6]
IOMMU Group 13:
	00:1f.0 ISA bridge [0601]: Intel Corporation Z97 Chipset LPC Controller [8086:8cc4]
	00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82]
	00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2]
IOMMU Group 14:
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
IOMMU Group 15:
	01:00.1 Audio device [0403]: NVIDIA Corporation GM200 High Definition Audio [10de:0fb0] (rev a1)
IOMMU Group 16:
	03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 11)
IOMMU Group 17:
	04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 18:
	05:01.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 19:
	05:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 2:
	00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
IOMMU Group 20:
	05:05.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 21:
	05:07.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1184e PCIe Switch Port [1b21:1184]
IOMMU Group 22:
	07:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 23:
	09:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 24:
	0a:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 3:
	00:14.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB xHCI Controller [8086:8cb1]
IOMMU Group 4:
	00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba]
IOMMU Group 5:
	00:19.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I218-V [8086:15a1]
IOMMU Group 6:
	00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad]
IOMMU Group 7:
	00:1b.0 Audio device [0403]: Intel Corporation 9 Series Chipset Family HD Audio Controller [8086:8ca0]
IOMMU Group 8:
	00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0)
IOMMU Group 9:
	00:1c.2 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 3 [8086:8c94] (rev d0)

Unfortunately, still with the 43 error... but getting there I hope!! smile

I will try now with your other qemu settings smile

Thanks a lot!


Keep It Simple, Stupid!

Offline

#9 2019-12-03 07:11:49

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

I have tried with your settings for qemu but:

"check" give me an error

the others seems to be working fine but I still have the error 43...

Any other ideas?

I can share my XML if you like or... should I try again with the VGA bios?

Thank you!


Keep It Simple, Stupid!

Offline

#10 2019-12-03 07:35:32

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello again,

I have tried with the Bios... black screen even with spice vga attached as well (does not seem to be restarting but I have tried RDP and it's not working).

Here it is, my XML file:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10</name>
  <uuid>c7464a60-ba42-4bd3-ad2d-af17466ba295</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>8388608</memory>
  <currentMemory unit='KiB'>8388608</currentMemory>
  <vcpu placement='static'>8</vcpu>
  <os>
    <type arch='x86_64' machine='pc-q35-4.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/usr/share/ovmf/x64/OVMF_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <vendor_id state='on' value='0123456789ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-passthrough' check='none'/>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/home/lhoqvso/VM/win10.img'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='4' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x18'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='10' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='11' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='11' port='0x8'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='12' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='12' port='0x9'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:c9:c7:e0'/>
      <source network='default'/>
      <model type='e1000e'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
      <gl enable='no' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x02' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom file='/home/lhoqvso/Downloads/vgaBios.rom'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='3'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
    <shmem name='looking-glass'>
      <model type='ivshmem-plain'/>
      <size unit='M'>32</size>
      <address type='pci' domain='0x0000' bus='0x0a' slot='0x01' function='0x0'/>
    </shmem>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off,hypervisor=false,hv_vendor_id=0123456789ab'/>
    <qemu:arg value='-machine'/>
    <qemu:arg value='q35,kernel_irqchip=on'/>
  </qemu:commandline>
</domain>

Suggestions are welcome smile

Thank you!!


Keep It Simple, Stupid!

Offline

#11 2019-12-03 16:07:17

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

I already told you that your IOMMU groups are not correct, you must fix this before anything should work.


I don't want to work.  I want to bang on the drum all day.

Offline

#12 2019-12-03 16:19:33

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello sir!

But now the nvidia vga and audio are in different groups isn’t it? It has changed from the first time without the ACS Patch.

Is not what I was trying to achieve? Do I need to do something different?

Thanks a lot!


Keep It Simple, Stupid!

Offline

#13 2019-12-03 16:23:17

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

No, they should not be separated, the ACS patch will properly group those two together. My example from earlier demonstrates this.


I don't want to work.  I want to bang on the drum all day.

Offline

#14 2019-12-03 16:38:42

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Mmmm then maybe I need to do something different... I will do some research and let’s see if I’m able to do it properly smile

I will test it tonight or tomorrow morning smile

Thanks!!


Keep It Simple, Stupid!

Offline

#15 2019-12-03 18:04:20

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

My kernel cmdline is as follows, you can try removing the multifunction parameter:

initrd=\intel-ucode.img initrd=\initramfs-linux-vfio.img root=/dev/sda2 rw intel_iommu=on pcie_acs_override=downstream

I don't want to work.  I want to bang on the drum all day.

Offline

#16 2019-12-03 18:38:01

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Ohhh yeah!!! You made it!!!

I had downstream option but also multifunction and maybe that was the problem... it’s working!!!

Thanks sir!! I owe you a beer!!

Much, much appreciated!!


Keep It Simple, Stupid!

Offline

#17 2019-12-03 18:47:39

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

If you're ever in Michigan, I'll take you up on it. ;-)

For the future though, I do host binary copies of the linux-vfio package on my package repo if you don't want to spend a lot of time compiling it yourself in the future.

https://wiki.archlinux.org/index.php/Un … ies#markzz


I don't want to work.  I want to bang on the drum all day.

Offline

#18 2019-12-03 18:50:16

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Really appreciated mate! I’m in Norway but he sure that you have a paid beer here!
Woooooow súper happy!! smile

Have a nice day there!!

(And I'm taking notes so I don’t repeat this... and your patched kernel is also appreciated!!)

Last edited by thearcher (2019-12-04 06:43:21)


Keep It Simple, Stupid!

Offline

#19 2019-12-04 06:42:55

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Good morning sir,

Sorry for rising this but... My sound through HDMI seems to not work unless I use intel_iommu=on, igfx_off but if I do that it breaks the IOMMU...

Any ideas to have sound working without breaking it?

Thanks again smile


Keep It Simple, Stupid!

Offline

#20 2019-12-04 06:48:36

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Replying to myself smile

Seems like (I cannot test it until I arrive home), it's working fine now using:

iommu=pt (I do not know exactly what it means)

So this is now my working grub.cfg:

GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 quiet intel_iommu=on,igfx_off iommu=pt pcie_acs_override=downstream"

I will edit this post later today when I'm at home for testing it smile

Thanks a lot!!


Keep It Simple, Stupid!

Offline

#21 2019-12-04 09:10:27

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

I try to keep things simple with as few cmdline options as possible. I have not personally tried using sound over HDMI as it's not a concern with my setup, but I also do not try to suppress my iGPU with any options either. It should work just fine as long as you pass through both the video and sound components on the card,

I would definitely be curious of any issues you have though for my own information,

Last edited by markzz (2019-12-04 09:11:15)


I don't want to work.  I want to bang on the drum all day.

Offline

#22 2019-12-04 19:55:02

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello sir!

At least at home!

It seems like the HDMI output from the Intel Graphics card is not working fine if the intel_iommu=on,igfx_off is not there. It's just does not work,

Apart of that, if the iommu=pt is not there the audio is not working either... but I'm not sure why.

I will try removing the igfx_off but keeping the iommu=pt.

I have also tried with the "multifunction" option for the ACS but it seems like sets my graphic card and Nvidia audio in the same group so it does not work the passthrough.

I will keep trying smile

Thanks a lot!!

Best regards,


Juanjo


Keep It Simple, Stupid!

Offline

#23 2019-12-05 20:18:31

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello everyone!

Well after a lot of testing...

You were right... better with intel_iommu=on only... but I was right... sound stops working with HDMI output...

But... took me a while but I found something that it seems to be working fine!

You need to create a file named daemon.conf in your .config/pulse folder with the following content:

default-sample-format = float32le
default-sample-rate = 48000
alternate-sample-rate = 44100
default-sample-channels = 2
default-channel-map = front-left,front-right
default-fragments = 2
default-fragment-size-msec = 125
resample-method = soxr-vhq
enable-lfe-remixing = no
high-priority = yes
nice-level = -11
realtime-scheduling = yes
realtime-priority = 9
rlimit-rtprio = 9
daemonize = no

/etc/asound.conf (even having this only one user was working fine, so I believe the configuration above was the important one but maybe is the combination!!)

  GNU nano 4.6                    /etc/asound.conf                               
# Use PulseAudio by default
#pcm.!default {
#  type pulse
#  fallback "sysdefault"
#  hint {
#    show on
#    description "Default ALSA Output (currently PulseAudio Sound Server)"
#  }
#}

pcm.!default {
  type plug
  slave.pcm hw
  fallback "sysdefault"
}

# vim:set ft=alsaconf:

Information was taken from here:
https://medium.com/@gamunu/enable-high- … 16f3fe7e1f

But I was only trying to improve the sound a few days ago when I realized that in one user (the "improved" one) the HDMI sound was working while not in the other... so... with this file it seems to be working all fine.

As a note for others... both NVIDIA devices are in the same IOMMU group for me and everything works but if I use igfx_off even when the groups are not changing, the VM give me the infamous error 43 again...

So I think I did the whole research and all looks good.

Disclaimer: Your configuration may need to be different. This is only working for me and you may need to do your own research to find a solution that works for you. In any case... I hope this sheds some light when you feel alone smile

Last edited by thearcher (2019-12-05 20:23:30)


Keep It Simple, Stupid!

Offline

#24 2019-12-05 21:01:37

markzz
Member
From: Michigan, United States
Registered: 2013-06-08
Posts: 89
Website

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Oh, I see now, you're trying to use the default sound instead of using the HDMI port on the card.

How I solved that issue is with

QEMU_AUDIO_DRV=pa qemu-system-x86_64 ...

.

Last edited by markzz (2019-12-05 21:01:44)


I don't want to work.  I want to bang on the drum all day.

Offline

#25 2019-12-06 11:24:20

thearcher
Member
Registered: 2010-05-13
Posts: 31

Re: [SOLVED] Infamous error 43 with NVIDIA and KVM Q35

Hello sir,

My solution was not perfect... well honestly I do not know exactly why...

It seems like in one of the users, with Cinnamon the audio when streaming is out of sync... but surprisingly it works fine with another user that uses i3.

With intel_iommu=on,iglx_off everything works but not the VM as it gets the infamous error 43.

I have removed the ich9 audio from the VM as I'm using the NVIDIA audio and HDMI (working fine, tested with a game yesterday).

Any ideas why the audio can be out of sync but only for a user? (they have both the same daemon.conf file). I do not think it's a resources issue and the audio is out of sync without starting the VM and just in a few seconds (and get worst and worst).

Any help is more than welcome!

Best regards,


Juanjo


Keep It Simple, Stupid!

Offline

Board footer

Powered by FluxBB