[Solved] Help with VGA passthrough using libvirt

cruzpach · 2019-01-15 04:55:29

Hello guys,

I was trying to pass-through my dedicated graphics card (GTX 980 Ti) in a virtual machine. After following the guide in https://wiki.archlinux.org/index.php/PC … ng_the_GPU, I tried booting the VM and got the following error:

Error starting domain: internal error: process exited while connecting to monitor: 2019-01-15T04:48:50.385078Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.5,addr=0x0: vfio 0000:01:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1400, in startup
    self._backend.create()
  File "/usr/lib/python3.7/site-packages/libvirt.py", line 1080, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: internal error: process exited while connecting to monitor: 2019-01-15T04:48:50.385078Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.5,addr=0x0: vfio 0000:01:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

This is the output for dmesg | grep -i vfio:

[    1.113372] VFIO - User Level meta-driver version: 0.3
[    1.116818] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[    1.131727] vfio_pci: add [10de:17c8[ffff:ffff]] class 0x000000/00000000
[    1.148408] vfio_pci: add [10de:0fb0[ffff:ffff]] class 0x000000/00000000
[    3.319483] vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none

Ant his is the output for lspci -nnk -d 10de:17c8:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
	Subsystem: Micro-Star International Co., Ltd. [MSI] GM200 [GeForce GTX 980 Ti] [1462:3236]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau

Any help as to how to fix this issue?

Last edited by cruzpach (2019-01-19 18:01:43)

jasonwryan · 2019-01-15 05:13:30

Please use code tags when pasting to the boards: https://wiki.archlinux.org/index.php/Co … s_and_code

Moving to NC...

cruzpach · 2019-01-15 05:23:16

Thanks

Lone_Wolf · 2019-01-15 11:42:55

content of /etc/modprobe.d/vfio.conf ?

Also the output of the script at https://wiki.archlinux.org/index.php/PC … _are_valid

cruzpach · 2019-01-15 20:35:07

This is what I have in /etc/modprobe.d/vfio.conf:

options vfio-pci ids=10de:17c8,10de:0fb0

This is what I get from the script:

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
IOMMU Group 10 00:1c.3 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 4 [8086:8c96] (rev d0)
IOMMU Group 11 00:1c.4 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 5 [8086:8c98] (rev d0)
IOMMU Group 12 00:1d.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #1 [8086:8ca6]
IOMMU Group 13 00:1f.0 ISA bridge [0601]: Intel Corporation Z97 Chipset LPC Controller [8086:8cc4]
IOMMU Group 13 00:1f.2 SATA controller [0106]: Intel Corporation 9 Series Chipset Family SATA Controller [AHCI Mode] [8086:8c82]
IOMMU Group 13 00:1f.3 SMBus [0c05]: Intel Corporation 9 Series Chipset Family SMBus Controller [8086:8ca2]
IOMMU Group 14 03:00.0 SATA controller [0106]: Samsung Electronics Co Ltd Device [144d:a801] (rev 01)
IOMMU Group 15 04:00.0 Ethernet controller [0200]: Qualcomm Atheros Killer E220x Gigabit Ethernet Controller [1969:e091] (rev 10)
IOMMU Group 16 05:00.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 41)
IOMMU Group 17 07:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller [1b4b:9172] (rev 12)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
IOMMU Group 1 00:01.2 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x4 Controller [8086:0c09] (rev 06)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GM200 High Definition Audio [10de:0fb0] (rev a1)
IOMMU Group 1 02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 [144d:a802] (rev 01)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller [8086:0412] (rev 06)
IOMMU Group 3 00:03.0 Audio device [0403]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller [8086:0c0c] (rev 06)
IOMMU Group 4 00:14.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB xHCI Controller [8086:8cb1]
IOMMU Group 5 00:16.0 Communication controller [0780]: Intel Corporation 9 Series Chipset Family ME Interface #1 [8086:8cba]
IOMMU Group 6 00:1a.0 USB controller [0c03]: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 [8086:8cad]
IOMMU Group 7 00:1b.0 Audio device [0403]: Intel Corporation 9 Series Chipset Family HD Audio Controller [8086:8ca0]
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 1 [8086:8c90] (rev d0)
IOMMU Group 9 00:1c.2 PCI bridge [0604]: Intel Corporation 9 Series Chipset Family PCI Express Root Port 3 [8086:8c94] (rev d0)

Lone_Wolf · 2019-01-16 09:29:23

IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
IOMMU Group 1 00:01.2 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x4 Controller [8086:0c09] (rev 06)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GM200 High Definition Audio [10de:0fb0] (rev a1)
IOMMU Group 1 02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 [144d:a802] (rev 01

Your nvme SSDs is in the same iommu group as the nvidia videocard .
https://wiki.archlinux.org/index.php/PC … MF#Gotchas describes that situation.

The user manual for your motherboard should have the info needed to determine if moving the card / ssd is possible.
Post the brand and model of your motherboard .

Last edited by Lone_Wolf (2019-01-16 09:30:08)

cruzpach · 2019-01-18 21:43:49

Removing the SSD from group 1 did the trick! Thank you!

On another note, I just found out that NVIDIA GeForce cards seem to be programmed not work in a VM. I tried the workaround by Angelos Kouratzinos found here https://passthroughpo.st/apply-error-43-workaround/ and it worked perfectly!

Basically, I had to edit the VM XML file by typing

sudo EDITOR=vim virsh edit win10

Once the XML file opens, I had to either write or modify the vendor_id in the hyperv section tag with a value of 12 characters. It should look something like this:

<hyperv>
     <relaxed state='on'/>
     <vapic state='on'/>
     <spinlocks state='on' retries='8191'/>
     <vendor_id state='on' value='123456789ab'/> 
</hyperv>

The '123456789ab' value was given by Angelos Kouratzinos, you can write whatever value we want, as long as it is 12 characters in total.

After doing that, I added a some code to instruct KVM to hide its state as a VM. It looks like this:

Second, I instructed KVM to hide it’s state by adding the following code under the hyperv section:

<kvm>
     <hidden state='on'/>
</kvm>

And that did it!

Last edited by cruzpach (2019-01-28 22:32:06)

2ManyDogs · 2019-01-19 00:32:42

Please remember to mark your thread [SOLVED] (edit the title of your first post).

cruzpach · 2019-01-19 17:18:10

Would you mind guiding me through the process of editing the post title? I have not been able to to fin the "edit" button for the post

ewaller · 2019-01-19 17:34:19

You have to edit the first post in the thread. The edit button is below the text and to the right. When you edit the first post, the title is presented in an edit box.

Arch Linux

#1 2019-01-15 04:55:29

[Solved] Help with VGA passthrough using libvirt

#2 2019-01-15 05:13:30

Re: [Solved] Help with VGA passthrough using libvirt

#3 2019-01-15 05:23:16

Re: [Solved] Help with VGA passthrough using libvirt

#4 2019-01-15 11:42:55

Re: [Solved] Help with VGA passthrough using libvirt

#5 2019-01-15 20:35:07

Re: [Solved] Help with VGA passthrough using libvirt

#6 2019-01-16 09:29:23

Re: [Solved] Help with VGA passthrough using libvirt

#7 2019-01-18 21:43:49

Re: [Solved] Help with VGA passthrough using libvirt

#8 2019-01-19 00:32:42

Re: [Solved] Help with VGA passthrough using libvirt

#9 2019-01-19 17:18:10

Re: [Solved] Help with VGA passthrough using libvirt

#10 2019-01-19 17:34:19

Re: [Solved] Help with VGA passthrough using libvirt

Board footer