Arch Linux

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

                                                                                                       
  <os>                                                                                                                 
    <type arch='x86_64' machine='pc-q35-1.5'>hvm</type>                                                                 
    ...
  </os>                                                                                                                 
  ...
  <devices>                                                                                                             
    <emulator>/usr/local/kvm/bin/qemu-system-x86_64</emulator>                                                         
    ...
  </devices>
  ...
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=08:00.0,bus=root.1,addr=00.0,multifunctioIs there n=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=08:00.1,bus=root.1,addr=00.1'/>
    ...
    <qemu:arg value='-bios'/>
    <qemu:arg value='/usr/src/kvm/seabios-1.7.2-patched/seabios/out/bios.bin'/>
    ...
  </qemu:commandline>

I tried using these <qemu:*> tags in my virt-manager XML, but no matter if I define a new VM from the XML or edit an existing one, all the qemu tags vanish without an error. This means after I insert them, virsh edit/define, and then reopen the XML file all the qemu tags are gone...
I tried virt-manager and libvirt from both official repo and AUR. No difference
Am I missing some setting for qemu integration or why is this happening???

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

This is how the first line should look, otherwise it clears all the <qemu:*> stuff.

CharlieBra7o

Member

Registered: 2014-03-01

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

This is how the first line should look, otherwise it clears all the <qemu:*> stuff.

That did the trick Thank you!

Last edited by CharlieBra7o (2014-07-23 22:51:27)

monchi

Member

Registered: 2014-07-21

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

This worked for me. That flag seems to be "new", at least for me If I remember correctly, the vga arbiter was always on previously.

no success with my configuration...
Any got an idea what to do next?

Hard to say where to start, but did you check your hardware for compatibility? Theres an linked Gdrive table document here, where you could look up your configuration.

i could not find the document ...

But i gave it a try with XenServer 6.2.. GPU passthrough worked out of the box (same hardware). So it should be possible to set up passthrough with KVM either right?

Last edited by monchi (2014-07-24 15:32:25)

Nickolai

Member

Registered: 2014-07-24

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What's the problem with passing through an Intel VGA card? Is it impassable? I would really like to pass through an Intel card on my ThinkPad X series.

Last edited by Nickolai (2014-07-24 22:44:32)

belliash

Member

Registered: 2014-04-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What's the problem with passing through an Intel VGA card? Is it impassable? I would really like to pass through an Intel card on my ThinkPad X series.

no, its possible. check xengt

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What's the problem with passing through an Intel VGA card? Is it impassable? I would really like to pass through an Intel card on my ThinkPad X series.

no, its possible. check xengt

Or take a look at SmartOS. While I haven't used it personally, a friend of mine uses it and is very happy with it.

---

i'm sorry for my poor english wirting skills…

redger

Member

Registered: 2014-02-26

Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Is anyone using Centos 7 for VGA Passthrough ? Do you have a full patch-set for this (kernel 3.10) ... and if so, could you post the patches ....

I'd like to use Centos as a host (for stability) - but wondering about KVM patches for VGA passthrough (including performance enhancements)

Thanks for your help

belliash

Member

Registered: 2014-04-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What's the problem with passing through an Intel VGA card? Is it impassable? I would really like to pass through an Intel card on my ThinkPad X series.

no, its possible. check xengt

Or take a look at SmartOS. While I haven't used it personally, a friend of mine uses it and is very happy with it.

Are you serious? SmartOS = ZFS + DTrace + Zones + KVM
KVM doesn't support mediated GPU passthrough. Only XenGT can do that.

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Or take a look at SmartOS. While I haven't used it personally, a friend of mine uses it and is very happy with it.

Are you serious? SmartOS = ZFS + DTrace + Zones + KVM
KVM doesn't support mediated GPU passthrough. Only XenGT can do that.

It's KVM on the OpenIndia (from SunOS/OpenSolaris) kernel and as said I don't use it, because it only works on Intel CPUs. My friend claimed that it did GPU passthrough with his Intel GPU. I didn't verify this (and can't try myself, because I don't have the hardware), but I trust him enough to at least propose it as a possibility.

I'm sorry however, if this can't do passthrough of an Intel GPU.

---

i'm sorry for my poor english wirting skills…

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I get

 Hunk #1 FAILED at 3292 

when I try to patch the 3.15.6 kernel with the acs patch.

I use this command

 patch -p1 -i ../acs.patch --dry-run 

I am running debian jessie. I have gotten kvm and IOMMU to work on arch linux, but I can't seem to get it on debian. What should I do?

----EDIT----

The error "patch unexpectedly ends in middle of line" might point to a broken acs.patch file. See this link for some ideas about the cause.

I think i got it, let me post again the solution found some pages behind.

If you have any problem applying original acs override patch from Alex Williamson (Hunk #1 FAILED at 3292 in my case) try downloading this package which contains a new version.

This one applies correctly on my Debian system with kernel 3.15.3
Just built a new kernel image, will test it later and update.

Edit: it works! Thanks again to this fantastic thread!

### Group 0 ###
    00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v3 Processor DRAM Controller (rev 06)
### Group 1 ###
    00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
### Group 2 ###
    00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller (rev 06)
### Group 3 ###
    00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)
### Group 4 ###
    00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
### Group 5 ###
    00:1c.2 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 (rev d5)
### Group 6 ###
    00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5)
### Group 7 ###
    00:1f.0 ISA bridge: Intel Corporation C222 Series Chipset Family Server Essential SKU LPC Controller (rev 05)
    00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
    00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)
    00:1f.6 Signal processing controller: Intel Corporation 8 Series Chipset Family Thermal Management Controller (rev 05)
### Group 8 ###
    01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 760] (rev a1)
    01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
### Group 9 ###
    02:00.0 Serial Attached SCSI controller: LSI Logic / Symbios Logic SAS2308 PCI-Express Fusion-MPT SAS-2 (rev 05)
### Group 10 ###
    03:00.0 PCI bridge: ASPEED Technology, Inc. AST1150 PCI-to-PCI Bridge (rev 03)
    04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30)
### Group 11 ###
    05:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
### Group 12 ###
    06:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)

This post solved my issue!

Last edited by shrubuntu (2014-07-26 15:46:17)

belliash

Member

Registered: 2014-04-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Or take a look at SmartOS. While I haven't used it personally, a friend of mine uses it and is very happy with it.

Are you serious? SmartOS = ZFS + DTrace + Zones + KVM
KVM doesn't support mediated GPU passthrough. Only XenGT can do that.

It's KVM on the OpenIndia (from SunOS/OpenSolaris) kernel and as said I don't use it, because it only works on Intel CPUs. My friend claimed that it did GPU passthrough with his Intel GPU. I didn't verify this (and can't try myself, because I don't have the hardware), but I trust him enough to at least propose it as a possibility.

I'm sorry however, if this can't do passthrough of an Intel GPU.

No, it can't. KVM doesnt support such things.

DLWood1001

Member

Registered: 2014-07-17

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Is anyone using Centos 7 for VGA Passthrough ? Do you have a full patch-set for this (kernel 3.10) ... and if so, could you post the patches ....

I'd like to use Centos as a host (for stability) - but wondering about KVM patches for VGA passthrough (including performance enhancements)

Thanks for your help

Hi Redger,

I am using RHEL7 w/3.15.6 kernel.

Needed to download the src rpm files for kernel-ml, Seabios, Qemu, IPXE, EDK2-Tools and built from scratch. Note: The spec files were modded lightly to allow for the latest versions of these projects to compile.

Issues Included:
Kernel, enabled some options in the kernel-config, re-diffed the patches to merge properly. ACS, i915, VGA-ARB, DMA-v4 patches from AW's patch history on patchwork.
Seabios, out of date and some roms were not packaged correctly.
Qemu, Broken links to IPXE directory.
IPXE, Didn't inlcude EFI roms.
EDK2-Tools, required by IPXE to build EFI roms.

Currently, everything is working with VGA passthrough. I have RHEL7 host running both a Win8.1 guest and OS X mavericks guests running simultaneously.

redger

Member

Registered: 2014-02-26

Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

thanks @DLWood,
sounds like it was a bit tricky was it worthwhile ?
I have refactored the patches for Ubuntu 3.13 kernel and am running that but considered Centos with "native" kernel in the hope it might be more stable / suitable for 24*7 ue (before I start moving other servers across to VMs)

did you consider using the "stock" Centos kernel (v3.10) ?

thanks again for your reply

DLWood1001

Member

Registered: 2014-07-17

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi @redger,

I would say it is worth it for me , I've learned a lot in the process.

I did consider using the 3.10 kernel, but had to upgrade due to some unrelated hardware limitations that are resolved in latest 3.11+ mainline releases.

I believe my hardware problem will be resolved in RHEL 7.1, so I may try using the stock RHEL7 kernel after that is released. The stock RHEL7/CentOS7 kernel config doesn't have CONFIG_VFIO_PCI_VGA enabled, so you would need to maintain your own build either way.

CharlieBra7o

Member

Registered: 2014-03-01

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

After I booted up my freshly installed Windows 8.1 for the first time it worked for about 60 seconds or so, then the monitor lost its signal. From this moment on the monitor always loses its signal during the Windows loading screen. The VM's CPU load from this moment on seems to be completely steady using up 1 full thread.

My guess is that Windows tried to install some crappy GPU/chipset drivers or apply some wrong settings...

Does anyone have an idea what might be going on there?

Additional info:
- Radeon HD 7950
- 3.15.6 from the OP unchanged
- qemu-git 2.1.r34391.gf368c33-1 + libvirt 1.2.6-1 from GIT

Last edited by CharlieBra7o (2014-07-27 14:37:12)

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Alright I have got it mostly working except for a few snags.

I get an "Code 43" on the nvidia driver within my windows 7 guest and

I cannot restart the VM once I have shut it down without restarting my computer

Is there any fix for these?

CharlieBra7o

Member

Registered: 2014-03-01

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Alright I have got it mostly working except for a few snags.

I get an "Code 43" on the nvidia driver within my windows 7 guest and

I cannot restart the VM once I have shut it down without restarting my computer

Is there any fix for these?

@Code 43: read https://bbs.archlinux.org/viewtopic.php … 0#p1426160 and following post(s)
@Reboot: This issue definitely has been discussed from page 1 on. I think you need the vga reset patch that aw posted a few pages before, but Im not sure...

zzz3000

Member

Registered: 2014-02-16

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hello it works for me with problem

I get messages

Jul 27 20:00:01 DeltaS systemd: Starting Session 5 of user root.
Jul 27 20:00:01 DeltaS systemd: Started Session 5 of user root.
Jul 27 20:01:01 DeltaS systemd: Starting Session 6 of user root.
Jul 27 20:01:01 DeltaS systemd: Started Session 6 of user root.
Jul 27 20:01:01 DeltaS systemd: Starting Session 7 of user sanek.
Jul 27 20:01:01 DeltaS systemd: Started Session 7 of user sanek.
Jul 27 20:10:01 DeltaS systemd: Starting Session 8 of user root.
Jul 27 20:10:01 DeltaS systemd: Started Session 8 of user root

And I get BSOD in some minutes

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hello it works for me with problem

I get messages

Jul 27 20:00:01 DeltaS systemd: Starting Session 5 of user root.
Jul 27 20:00:01 DeltaS systemd: Started Session 5 of user root.
Jul 27 20:01:01 DeltaS systemd: Starting Session 6 of user root.
Jul 27 20:01:01 DeltaS systemd: Started Session 6 of user root.
Jul 27 20:01:01 DeltaS systemd: Starting Session 7 of user sanek.
Jul 27 20:01:01 DeltaS systemd: Started Session 7 of user sanek.
Jul 27 20:10:01 DeltaS systemd: Starting Session 8 of user root.
Jul 27 20:10:01 DeltaS systemd: Started Session 8 of user root

And I get BSOD in some minutes

You don't really expect that someone can give you useful advice with this tiny bit of information, do you?

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Alright I have got it mostly working except for a few snags.

I get an "Code 43" on the nvidia driver within my windows 7 guest and

I cannot restart the VM once I have shut it down without restarting my computer

Is there any fix for these?

@Code 43: read https://bbs.archlinux.org/viewtopic.php … 0#p1426160 and following post(s)
@Reboot: This issue definitely has been discussed from page 1 on. I think you need the vga reset patch that aw posted a few pages before, but Im not sure...

Awesome, reverting to 335 drivers fixed the issue!
Everything running smoothly

My system :

Debian Jessie with custom 3.15.6 kernel with patches from OP
i7 4790k
MSI gaming 5 motherboard
GTX 580

@aw what is the status of VGA reset patches?

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dump gpu bios, use it in your config and u should be able to reset...

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dump gpu bios, use it in your config and u should be able to reset...

Interesting, can you describe this a bit more?

mv0lnicky

Member

Registered: 2014-03-05

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you are dual booting Windows, just use GPU-Z to dump your GPU BIOS into a file. Then you'll have VGA reset from QEMU.

CharlieBra7o

Member

Registered: 2014-03-01

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

After I booted up my freshly installed Windows 8.1 for the first time it worked for about 60 seconds or so, then the monitor lost its signal. From this moment on the monitor always loses its signal during the Windows loading screen. The VM's CPU load from this moment on seems to be completely steady using up 1 full thread.

My guess is that Windows tried to install some crappy GPU/chipset drivers or apply some wrong settings...

Does anyone have an idea what might be going on there?

Additional info:
- Radeon HD 7950
- 3.15.6 from the OP unchanged
- qemu-git 2.1.r34391.gf368c33-1 + libvirt 1.2.6-1 from GIT

Just tried installing Windows 7 and after the installation I spent about 5 minutes clicking around, then shutdown the VM to make an image and started it up again. Now it always freezes during the "Starting Windows" screen and 'virsh list' lists the VM as 'paused'.
dmesg doesn't show anything peculiar, except maybe

kvm: zapping shadow pages for mmio generation wraparound

Honestly, I have absolutely no clue what to try next...

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

After I booted up my freshly installed Windows 8.1 for the first time it worked for about 60 seconds or so, then the monitor lost its signal. From this moment on the monitor always loses its signal during the Windows loading screen. The VM's CPU load from this moment on seems to be completely steady using up 1 full thread.

My guess is that Windows tried to install some crappy GPU/chipset drivers or apply some wrong settings...

Does anyone have an idea what might be going on there?

Additional info:
- Radeon HD 7950
- 3.15.6 from the OP unchanged
- qemu-git 2.1.r34391.gf368c33-1 + libvirt 1.2.6-1 from GIT

Just tried installing Windows 7 and after the installation I spent about 5 minutes clicking around, then shutdown the VM to make an image and started it up again. Now it always freezes during the "Starting Windows" screen and 'virsh list' lists the VM as 'paused'.
dmesg doesn't show anything peculiar, except maybe

kvm: zapping shadow pages for mmio generation wraparound

Honestly, I have absolutely no clue what to try next...

virsh tells you the reason, why vm was paused. I had similar issue (I/O Error), because of vm's disks being raw file images on btrfs (and having really low performance).

Last edited by dwe11er (2014-07-29 08:42:10)