Arch Linux

ebol4

Member

Registered: 2014-12-17

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Oh my god, aw, it works. I've been trying to get this running for so long. I get BIOS output and it boots into Windows. Checking to make sure the video drivers are functional... edit: YES THEY ARE!

In case anyone is having a similar issue with Code 12 with Intel Host/Nvidia Guest -- PATCH THE KERNEL!!!!

Now I just need to figure out how to get some decent input available on it... I'm currently doing "sendkey ____" in the QEMU monitor through TightVNC to navigate Windows....

Thank you so much for your help.

Last edited by ebol4 (2014-12-18 02:37:46)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Oh my god, aw, it works. I've been trying to get this running for so long. I get BIOS output and it boots into Windows. Checking to make sure the video drivers are functional... edit: YES THEY ARE!

In case anyone is having a similar issue with Code 12 with Intel Host/Nvidia Guest -- PATCH THE KERNEL!!!!

Now I just need to figure out how to get some decent input available on it... I'm currently doing "sendkey ____" in the QEMU monitor through TightVNC to navigate Windows....

Thank you so much for your help.

You can add -device qxl somewhere in the middle to make qemu show you an SDL surface where it can grab keyboard and mouse. But be aware, it lags seriously in source-based games, sometimes mod-keys(shift, ctrl, etc.) stick and the mouse may stop responding if you press some weird buttons(like win+alt+smth). I've been using this BAD input for a month or so because i'm lazy as hell.

Another options are:
VNC. I think there was a way of getting it working via qemu's built-in, so you could rule the keyboard while it's in BIOS menu, so no software required on guest.
Synergy(it doesn't work in another context, like UAC windows popping up)
Spice(my fedora doesn't have a client in the repos)
Separate keyboard-mouse pair connected through passed-through USB controller(or just as usb devices).
Hardware KVM(heh!)-switch.

Last edited by Duelist (2014-12-18 10:35:59)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

kidchunks

Member

Registered: 2014-12-03

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Setup: Debian, 3.14 Kernel with qemu 2.1.2

Doing some testing with libvirt (xml) config and I'm getting the following when trying to start my guest using virsh.

2014-12-18T13:11:47.567800Z qemu-system-x86_64: -device vfio-pci,host=85:00.0,addr=03.0,multifunction=on,x-vga=on: vfio_dma_map(0x7fe5acea33b0, 0x100000000, 0x40000000, 0x7fe555800000) = -12 (Cannot allocate memory)

In dmesg (a few of htese):

[  350.663041] vfio_pin_pages: RLIMIT_MEMLOCK (65536) exceeded

I looked around the thread and saw others have similar issues when trying to run as a diff user other than root. The problem is that I am as root and still cannot launch the guest.

I checked the RLIMIT using ulimit and got the following:

root@local:~# ulimit
unlimited

/etc/libvirt/qemu.conf has the user/group set as well as the cgroup_device_acl.

libvirt sample config:

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=85:00.0,addr=09.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=85:00.1,addr=09.1'/>
    <qemu:arg value='-bios'/>
    <qemu:arg value='/usr/share/seabios/bios.bin'/>
  </qemu:commandline>

Again, not sure why even as root, I cannot run the guest using virsh. Running qemu directly works.

Last edited by kidchunks (2014-12-18 13:40:50)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

libvirt sample config:

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=85:00.0,addr=09.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=85:00.1,addr=09.1'/>
    <qemu:arg value='-bios'/>
    <qemu:arg value='/usr/share/seabios/bios.bin'/>
  </qemu:commandline>

You're causing your own problems with this <qemu:arg> crap.  The bios option is not needed.  The vfio-pci devices should be created as <hostdev> options.  To add the x-vga=on option, use a wrapper script like shown in this post: https://bbs.archlinux.org/viewtopic.php … 1#p1475541  Set the <emulator> to the wrapper.  All the problems you're having are because you're hiding the assigned device from libvirt and therefore needing to hack the libvirt setup to allow the VM the permissions that libvirt doesn't know it needs.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Nesousx

Member

Registered: 2012-03-27

Posts: 46

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I was looking for the method to build the kernel from first post. Here is a little reminder if anyone had forgotten how to do it like myself.

Download archive, extract it and cd into it, then:

makepkg -s
pacman -U linux-mainline-*

Imho, it would be great if OP could add it to his first post.

Last edited by Nesousx (2014-12-18 21:09:24)

kidchunks

Member

Registered: 2014-12-03

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You're causing your own problems with this <qemu:arg> crap.  The bios option is not needed.  The vfio-pci devices should be created as <hostdev> options.  To add the x-vga=on option, use a wrapper script like shown in this post: https://bbs.archlinux.org/viewtopic.php … 1#p1475541  Set the <emulator> to the wrapper.  All the problems you're having are because you're hiding the assigned device from libvirt and therefore needing to hack the libvirt setup to allow the VM the permissions that libvirt doesn't know it needs.

Thanks for the clarification and link to the wrapper script. I tried searching for a way to build the script after seeing this post from you but came up short. It's strange that this worked using ubuntu 14.04 (qemu args in xml file). It could be that ubuntu does something different regarding the implementation.

Anyways, I'll stick to using hostdev for libvirt and remove the qemu args.

Last edited by kidchunks (2014-12-18 21:40:14)

magemaster

Member

From: Brasil

Registered: 2010-11-24

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to be clear, the people who can do passthrough with stock kernel are not using intel onboard video as primary, right?
Tried a lot with arch stock kernel(3.17) and only was able to have image on the second monitor after
patching a kernel(3.18-ck + i915 vga arbiter patch).

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to be clear, the people who can do passthrough with stock kernel are not using intel onboard video as primary, right?

Or avoiding the whole VGA problem by using OVFM+UEFI

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

thearcherblog

Member

Registered: 2014-10-30

Posts: 165

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to be clear, the people who can do passthrough with stock kernel are not using intel onboard video as primary, right?

Or avoiding the whole VGA problem by using OVFM+UEFI

Yes, I can confirm this.

Mine is working with intel on host and NVIDIA on guest using stock kernel but OVFM and UEFI and windows 8.

magemaster

Member

From: Brasil

Registered: 2010-11-24

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

My vga does not support UEFI and I can't change the card

8400GS
Valid ROM signature found @0h, PCIR offset 188h
    PCIR: type 0, vendor: 10de, device: 10c3, class: 030000
    PCIR: revision 0, vendor revision: 1
    Last image

DelusionalLogic

Member

Registered: 2013-05-07

Posts: 40

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If followed the guide, installed windows 8, and it kinda worked, execept the nividia driver. I enabled "kvm=off" on the cpu parameter, and installed the driver (from a cirrus vga adapter, with the card passthroug as secondary). It installed fine, but now i'm getting BSOD's from the VM. The error is "SYSTEM_SERVICE_EXCEPTION" and it's originating from the nvidia driver.

I then tried installing windows 7, but that's just giving me this whenever i try to boot. the installation went fine (this is without any passthrough).

KVM internal error. Suberror: 1
emulation failure
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080
ESI=0025db2a EDI=0007db2a EBP=00007c00 ESP=00000200
EIP=000000ca EFL=00010002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0020 00000200 0000ffff 00009300
CS =b000 000b0000 0000ffff 00009f00
SS =0020 00000200 0000ffff 00009300
DS =0020 00000200 0000ffff 00009300
FS =0020 00000200 0000ffff 00009300
GS =0020 00000200 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     002b0000 0000001f
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff <ff> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Here's my script for launching the vm:

vfio-bind 0000:03:00.0 0000:03:00.1
 
qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host,kvm=off \
    -smp 6,sockets=1,cores=6,threads=1 \
    -bios /usr/share/qemu/bios.bin -vga cirrus \
    -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
    -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
    -device vfio-pci,host=03:00.1,bus=root.1,addr=00.1 \
    -drive file=/home/delusional/vm/windows.img,id=disk,format=raw -device ide-hd,bus=ide.0,drive=disk \
    -drive file=/home/delusional/vm/install.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd \
    -usb -usbdevice host:1038:1361 -usbdevice host:1532:010d \
    -net nic -net bridge,br=br0 \
    -boot menu=on

I am aware that this would not enable the passthrough anyway (seeing as the nividia card is secondary) but at least it should boot right?

magemaster

Member

From: Brasil

Registered: 2010-11-24

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If followed the guide, installed windows 8, and it kinda worked, execept the nividia driver. I enabled "kvm=off" on the cpu parameter, and installed the driver (from a cirrus vga adapter, with the card passthroug as secondary). It installed fine, but now i'm getting BSOD's from the VM. The error is "SYSTEM_SERVICE_EXCEPTION" and it's originating from the nvidia driver.

I then tried installing windows 7, but that's just giving me this whenever i try to boot. the installation went fine (this is without any passthrough).

KVM internal error. Suberror: 1
emulation failure
EAX=00000010 EBX=00000080 ECX=00000000 EDX=00000080
ESI=0025db2a EDI=0007db2a EBP=00007c00 ESP=00000200
EIP=000000ca EFL=00010002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0020 00000200 0000ffff 00009300
CS =b000 000b0000 0000ffff 00009f00
SS =0020 00000200 0000ffff 00009300
DS =0020 00000200 0000ffff 00009300
FS =0020 00000200 0000ffff 00009300
GS =0020 00000200 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     002b0000 0000001f
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff <ff> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

Here's my script for launching the vm:

vfio-bind 0000:03:00.0 0000:03:00.1
 
qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host,kvm=off \
    -smp 6,sockets=1,cores=6,threads=1 \
    -bios /usr/share/qemu/bios.bin -vga cirrus \
    -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
    -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
    -device vfio-pci,host=03:00.1,bus=root.1,addr=00.1 \
    -drive file=/home/delusional/vm/windows.img,id=disk,format=raw -device ide-hd,bus=ide.0,drive=disk \
    -drive file=/home/delusional/vm/install.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd \
    -usb -usbdevice host:1038:1361 -usbdevice host:1532:010d \
    -net nic -net bridge,br=br0 \
    -boot menu=on

I am aware that this would not enable the passthrough anyway (seeing as the nividia card is secondary) but at least it should boot right?

I was having the same error before I patched the kernel.

ailcp

Member

Registered: 2014-12-20

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I am trying to passthrough the GPU of the following computer:

CPU: Athlon x4 860k (no iGPU)
Motherboard: ASRock FM2A88M Extreme4+
Primary GPU for host: ATI Rage XL (in PCI slot)
GPU for passthrough: MSI GTX 750 Ti OC v1 (in PCIE 3.0 slot)
Host OS: Fedora 21 (Stock Kernel 3.17.4)
Guest OS: Windows 8.1 64 bit

Kernel boot parameter in grub:

pci-stub.ids=10de:1380,10de:0fbc iommu=pt

Libvirt XML:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win8</name>
  <uuid>88b0dc9a-8831-11e4-b116-123b93f75cba</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
    <boot dev='cdrom'/>
    <boot dev='hd'/>
    <bootmenu enable='yes'/>
    <smbios mode='host'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/Windows81.iso'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/disk/by-id/ata-WDC_WD10EALX-759BA1_WD-WCATR8239826'/>
      <target dev='hdb' bus='ide'/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='pci' index='1' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='2' model='pci-bridge'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='28:8F:A0:07:98:C4'/>
      <source bridge='br0'/>
      <model type='e1000'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <input type='tablet' bus='usb'/>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc246'/>
        <address bus='1' device='3'/>
      </source>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='yes'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52e'/>
        <address bus='1' device='4'/>
      </source>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </hostdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-drive'/>
    <qemu:arg value='if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd'/>
  </qemu:commandline>
</domain>

Symptoms:
When I boot the guest the first time, the display can output the OVMF screen and the whole boot process successfully.  After booting into Windows, I installed Nvidia driver 334.89 and hope to avoid code 43 error.  After installation finished, the name "GTX 750 Ti" was shown correctly with a yellow exclamation mark in Device Manager and still come with code 43 error.  The screen resolution was stuck at 800x600 and the option to change resolution was greyed out.

I have looked at the google doc for reporting GPU passthrough results.  It seems that there is no success case of passing through a Nvidia GPU on FM2/FM2+ motherboard + CPU combo.  If someone had successfully passthrough a Nvidia GPU on FM2/FM2+ CPU, would you please share your results.  Since there is some success case of passthrough AMD GPU on FM2/FM2+ CPU, should I just give up on the GTX 750 Ti and buy a AMD GPU instead?  Please give me some advice?  Thank you very much!

Ansa89

Member

Registered: 2014-08-30

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I encountered a problem while running my VM: it freezed and when I tried to stop it with Ctrl+C, I got this message in syslog:

WARNING: CPU: 3 PID: 2679 at fs/block_dev.c:67 bdev_inode_switch_bdi+0x77/0x80()
Modules linked in: xt_mac xt_iprange xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_connmark xt_physdev xt_state xt_multiport
CPU: 3 PID: 2679 Comm: qemu-system-x86 Tainted: G      D    O   3.18.1 #1
Hardware name: MSI MS-7758/ZH77A-G43 (MS-7758), BIOS V1.6 10/25/2012
 0000000000000000 0000000000000009 ffffffff81a33879 0000000000000000
 ffffffff8112cadb ffff88031cc891b0 ffff88031cc89238 ffffffff81ddc400
 0000000000000000 000000000006001f ffffffff81286367 ffff88031cc890c0
Call Trace:
 [<ffffffff81a33879>] ? dump_stack+0x41/0x51
 [<ffffffff8112cadb>] ? warn_slowpath_common+0x8b/0xc0
 [<ffffffff81286367>] ? bdev_inode_switch_bdi+0x77/0x80
 [<ffffffff8128672c>] ? __blkdev_put+0x6c/0x1a0
 [<ffffffff812869d5>] ? blkdev_close+0x25/0x30
 [<ffffffff81250fad>] ? __fput+0xcd/0x1f0
 [<ffffffff8114807f>] ? task_work_run+0x9f/0xe0
 [<ffffffff8105bbb9>] ? do_notify_resume+0x69/0xa0
 [<ffffffff81250e9b>] ? fput+0x4b/0x90
 [<ffffffff81a3cba0>] ? int_signal+0x12/0x17

Host info:
- kernel 3.18.1 (without patches)
- IGD as primary graphic device
- qemu 2.2.0
- OVMF (latest svn)
- LVM used for guest disk

Guest info:
- Windows 8.1 UEFI
- nVidia geForce GTX 650 passed as primary graphic device
- virtio drivers for disk

Script to launch qemu:

qemu-system-x86_64 -enable-kvm -m 4096 -cpu host,hv_relaxed,hv_time,kvm=off \
        -smp 2,sockets=1,cores=2,threads=1 \
        -vga none -rtc base=localtime -k it \
        -device vfio-pci,host=01:00.0,multifunction=on \
        -device vfio-pci,host=01:00.1 \
        -device virtio-scsi-pci,id=scsi \
        -drive file=/dev/qemu/windows,id=disk,format=raw,if=none -device scsi-hd,bus=scsi.0,drive=disk \
        -drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd \
        -drive if=pflash,format=raw,file=/root/OVMF_VARS_Win8.1.fd

Any thoughts about the kernel oops?

maxqia

Member

Registered: 2014-12-20

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[*]Use pci-stub

In my case since i have 2 radeon cards blacklisting the radeon module is not an option, so i use pci-stub.

NOTE: If pci-stub was built as a module, you'll need to modify /etc/mkinitcpio.conf and add pci-stub in the MODULES section, after that you need to update your initramfs like this.

mkinitcpio -p linux-mainline

lspci

07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cayman PRO [Radeon HD 6950] <-- radeon 6950
07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] <-- radeon 6950 audio

lspci -n

07:00.0 0300: 1002:6719 <-- radeon 6950
07:00.1 0403: 1002:aa80 <-- radeon 6950 audio

[/*]

By the Way, "lspci" & "lspci -n" Can be replaced with a single "lspci -nn" command

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I am trying to passthrough the GPU of the following computer:

CPU: Athlon x4 860k (no iGPU)
Motherboard: ASRock FM2A88M Extreme4+
Primary GPU for host: ATI Rage XL (in PCI slot)
GPU for passthrough: MSI GTX 750 Ti OC v1 (in PCIE 3.0 slot)
Host OS: Fedora 21 (Stock Kernel 3.17.4)
Guest OS: Windows 8.1 64 bit

I have looked at the google doc for reporting GPU passthrough results.  It seems that there is no success case of passing through a Nvidia GPU on FM2/FM2+ motherboard + CPU combo.  If someone had successfully passthrough a Nvidia GPU on FM2/FM2+ CPU, would you please share your results.  Since there is some success case of passthrough AMD GPU on FM2/FM2+ CPU, should I just give up on the GTX 750 Ti and buy a AMD GPU instead?  Please give me some advice?  Thank you very much!

1. I really like your CPU. I want that CPU, actually. Can you tell me if there's PCI-E 3.0 support there? Google tells me that there isn't one. What does your lspci tell about it?
2. I have Athlon X4 750K, trinity-based FM2-socket paired with F2A55 motherboard. I am passing through TWO HD7750 with VGA and it works. As a host GPU i have nvidia gt610. Due to ASUS, there is MAJOR problems with IOMMU realization and memory accesses, but it works as long as i have my firefox running during the startup of VM. It reboots perfectly too. There appear to be some slight performance drawbacks related to bandwidth-issues, but i can't find the reasons or any info on how the hell crossfire works. I can post my qemu's launching script if you're interested, but it's awful. Oh, and i have fedora 21 too. Check if you have git-repo of OVMF, it helped me a lot when i fiddled with win7+UEFI+GPT setup.
3. Continue digging.
4. Awesome host gpu, i definitely should dig into old ATI cards, since only PCI ones i have right now are various S3 cards, which won't work.

P.S. I'm really-really tired right now, so my language may be a little bit messy, since my head's contents are messy.

Last edited by Duelist (2014-12-21 18:05:29)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

ailcp

Member

Registered: 2014-12-20

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

1. I really like your CPU. I want that CPU, actually. Can you tell me if there's PCI-E 3.0 support there? Google tells me that there isn't one. What does your lspci tell about it?

Hi Duelist,

I think there is PCI-E 3.0 support.  lspci shows:
LnkCap: Port #0, Speed 8GT/s, Width x16

Actually I have read your success case about passing through the HD7750, so I am thinking to buy a single slot XFX R7 250e which should be equivalent to the HD7750.  However, I really want to make the GTX 750 Ti work because I really like its low power consumption.

Since I am a Linux Newbie, hopefully someone with the FM2/FM2+ CPU and Nvidia GPU combo can share their experience on resolving this dreadful code 43 error.  Thank you.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have looked at the google doc for reporting GPU passthrough results.  It seems that there is no success case of passing through a Nvidia GPU on FM2/FM2+ motherboard + CPU combo.  If someone had successfully passthrough a Nvidia GPU on FM2/FM2+ CPU, would you please share your results.  Since there is some success case of passthrough AMD GPU on FM2/FM2+ CPU, should I just give up on the GTX 750 Ti and buy a AMD GPU instead?  Please give me some advice?  Thank you very much!

I don't know why you're getting a Code 43 and don't have or want any experience on an FM2/+ system, but a few comments on your config:

a) Your <qemu:arg> options can be avoided, libvirt should support OVMF on F21, it certainly does on F20 w/ the virt-preview repo enabled - http://vfio.blogspot.com/2014/09/libvir … -ovmf.html

b) Why are you defining extra PCI bridges in slots 3 & 4?

c) Why aren't you using your distro version of QEMU?  Again, F20 + virt preview repo works just fine on my Intel box with GTX750 and OVMF

d) <smbios mode='host'/>  Why?

I don't know if any of that will help, but better than nothing I suppose.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

ailcp

Member

Registered: 2014-12-20

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't know why you're getting a Code 43 and don't have or want any experience on an FM2/+ system, but a few comments on your config:

Thank you so much for aw's advice.  I guess I should give up on this Nvidia GPU and maybe buy the HD7750 to try.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't know why you're getting a Code 43 and don't have or want any experience on an FM2/+ system, but a few comments on your config:

Thank you so much for aw's advice.  I guess I should give up on this Nvidia GPU and maybe buy the HD7750 to try.

750Ti is better than HD7750 or rare R7 250E. It eats less power, works faster, heats less. It should work. HD7750 isn't compatible with UEFI out-of-the-box, while 750Ti is.(aw has 750 too) Is there any related messages related to VFIO or your GPU in dmesg? Or, maybe, windows doesn't enable MSI? Can you show us "sudo lspci -v" when your VM is running?

BTW, aw, does libvirt gets a copy of OVMF_VARS when ovmf is specified via qemu:arg section? Wasn't there some other way?

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

BTW, aw, does libvirt gets a copy of OVMF_VARS when ovmf is specified via qemu:arg section? Wasn't there some other way?

Not the way it's specified there, that's using the unified OVMF image that contains both CODE and VARS.  The guest doesn't get to persistently store any EFI variables.  It is possible to specify separate CODE and VARS with <qemu:arg>, but libvirt won't automatically do it for you.  libvirt doesn't parse anything specified via <qemu:arg>, it just blindly passes it through.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

BTW, aw, does libvirt gets a copy of OVMF_VARS when ovmf is specified via qemu:arg section? Wasn't there some other way?

Not the way it's specified there, that's using the unified OVMF image that contains both CODE and VARS.  The guest doesn't get to persistently store any EFI variables.

But that's not so crucial. Or is it?

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

BTW, aw, does libvirt gets a copy of OVMF_VARS when ovmf is specified via qemu:arg section? Wasn't there some other way?

Not the way it's specified there, that's using the unified OVMF image that contains both CODE and VARS.  The guest doesn't get to persistently store any EFI variables.

But that's not so crucial. Or is it?

Depends on the guest and whether it can manage to boot without them.  In most cases I've seen, it works fine without VARs.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

kainet

Member

Registered: 2014-12-22

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys! Thanks for this helpful thread.

I managed to start VM with such hardware:

OS: Debian Wheezy with no patches (3.18.1 self compiled kernel)
MB: GA-Z97M-WIFI
Host gpu: Intel HD (unplugged all actions through ssh)
Guest: Asus GTX 670

qemu-system-x86_64 -nographic -enable-kvm -M q35 -m 1024 -cpu host,kvm=off \
-smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on  \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-usb -usbdevice host:099a:0721 -usbdevice host:09da:9090 \
-device piix4-ide,bus=pcie.0,id=piix4-ide \
-drive file=/root/windows.img,id=disk,format=raw -device ide-hd,bus=piix4-ide.0,drive=disk

But i struggling with several issues. At first i used Q35 sata controller but where was to many BSODs. So i switched to piix4-ide. VM becomes more stable and even faster, but in both cases i cannot install nvidia video driver VM simply freezes. I am using Windows 7 x64. Also i tried to install driver manually but windows cannot find specific file for gtx 670 but it managed to identify video card correctly. I need some help here Please

Also a have a lot of such messages but they are refers for some problems with my usb devices. For some reason my keyboard resets many times and shifts picture on monitor to right. After several resets and shifts on monitor all calms down.

Dec 23 03:15:21 hypervisor kernel: [17266.255391] usb 1-10: ep 0x82 - rounding interval to 64 microframes, ep desc says 80 microframes
Dec 23 03:15:21 hypervisor kernel: [17266.408915] usb 1-7: reset full-speed USB device number 2 using xhci_hcd
Dec 23 03:15:21 hypervisor kernel: [17266.409015] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 1.
Dec 23 03:15:21 hypervisor kernel: [17266.409111] usb 1-7: hub failed to enable device, error -22
Dec 23 03:15:21 hypervisor kernel: [17266.561631] usb 1-7: reset full-speed USB device number 2 using xhci_hcd
Dec 23 03:15:21 hypervisor kernel: [17266.561743] xhci_hcd 0000:00:14.0: Setup ERROR: setup context command for slot 1.
Dec 23 03:15:21 hypervisor kernel: [17266.561852] usb 1-7: hub failed to enable device, error -22
Dec 23 03:15:21 hypervisor kernel: [17266.714487] usb 1-7: reset full-speed USB device number 2 using xhci_hcd
Dec 23 03:15:21 hypervisor kernel: [17266.726624] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called with disabled ep ffff88041a0cd840
Dec 23 03:15:21 hypervisor kernel: [17266.726747] xhci_hcd 0000:00:14.0: xHCI xhci_drop_endpoint called with disabled ep ffff88041a0cd7c0

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You need to use OVMF, or patch kernel.

That dmesg, looks normal when u pass usb devices like that.

Last edited by slis (2014-12-23 10:06:14)