Arch Linux

Bronek

Member

From: London

Registered: 2014-02-14

Posts: 123

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

RIght now im passing through my primary card (radeon 6950) what i did was, use pci-stub to bind it at boot and i disabled the efifb framebuffer with video=efifb:off on my grub cmdline

that's neat, I didn't know this option

stedaniels

Member

From: UK

Registered: 2013-11-21

Posts: 3

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I've just set up a bare arch linux install, and I'd like to start experimenting with KVM VGA-passthrough using vfio-vga.

I see there are source tarballs on the first page with patches pre-applied, and mentions of AUR, but I'd rather put together a script that I can run every time there's a kernel update.

There are three patches mentioned:

1. acs override patch

2. i935 vga arbiter fixes

3. memleak fix by xani

1. Is the ACS override patch on the kvm mailing list still the latest one?
2. For the i935 vga arbiter fixes patch is version 3 the latest and greatest?
3. Xani posted the first xani memory leak fix but that link is now dead, I read through and saw that xani posted a second memory leak fix but that link si also dead, does anyone have a current source?

Thanks!

I don't doubt the quality of the packages already provided, I just want a quick and easy way to rebuild and reproduce on a whim.

Thank you for all your time and energy already exerted on this adventure!

Steve

Last edited by stedaniels (2014-03-05 13:19:22)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I've just set up a bare arch linux install, and I'd like to start experimenting with KVM VGA-passthrough using vfio-vga.

I see there are source tarballs on the first page with patches pre-applied, and mentions of AUR, but I'd rather put together a script that I can run every time there's a kernel update.

There are three patches mentioned:

1. acs override patch

2. i935 vga arbiter fixes

3. memleak fix by xani

1. Is the ACS override patch on the kvm mailing list still the latest one?
2. For the i935 vga arbiter fixes patch is version 3 the latest and greatest?
3. Xani posted the first xani memory leak fix but that link is now dead, I read through and saw that xani posted a second memory leak fix but that link si also dead, does anyone have a current source?

Thanks!

I don't doubt the quality of the packages already provided, I just want a quick and easy way to rebuild and reproduce on a whim.

Thank you for all your time and energy already exerted on this adventure!

Steve

Those are not source packages, and all the patches you mention are there, you can build it yourself using makepkg

makepkg is used for compiling and building packages suitable for installation with pacman, Arch Linux's package manager. makepkg is a script that automates the building of packages; it can download and validate source files, check dependencies, configure build-time settings, compile the sources, install into a temporary root, make customizations, generate meta-info, and package everything together.

Last edited by nbhs (2014-03-05 13:55:04)

stedaniels

Member

From: UK

Registered: 2013-11-21

Posts: 3

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi nbhs & others,

Sorry for my ignorance and thank you for not calling me out on it!  You learn something new everyday it seems.

Reading through PKGBUILD was enlightening.  I'd previously got this all mostly running on Ubuntu so arch is a whole new experience.

Cheers,

Steve

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi nbhs & others,

Sorry for my ignorance and thank you for not calling me out on it!  You learn something new everyday it seems.

Reading through PKGBUILD was enlightening.  I'd previously got this all mostly running on Ubuntu so arch is a whole new experience.

Cheers,

Steve

Here's how you build and install these packages.

For example the kernel, unpack the linux-mainline.tar.gz file somewhere then:

cd path/to/linux-mainline
makepkg -s
sudo pacman -U linux-mainline-*

EDIT: Dont forget to set MAKEFLAGS to the number of cores you have on your /etc/makepkg.conf file

Last edited by nbhs (2014-03-05 14:41:30)

mv0lnicky

Member

Registered: 2014-03-05

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

VGA passthrough is working great with a Radeon 5850. I have tested with a few games and performance is impressive. Tested guests were Windows 7, Fedora 20, Ubuntu 12.04 and 14.04 and Debian 7.

Using Debian Sid as host. Linux 3.13.5 with patches and QEMU from the first post in this topic.

Current script :

${BIN_PATH}qemu-system-x86_64 --enable-kvm \
-M q35 -m ${RAM} -cpu ${CPU} -name ${NAME} \
-smp ${CORES},sockets=1,cores=${CORES},threads=1 \
-bios ${BIOS} \
-vga none -rtc base=localtime,clock=host \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device piix4-ide,bus=pcie.0,id=ide \
-device vfio-pci,host=${GPU_RADEON},bus=root.1,addr=00.0,multifunction=on,x-vga=on,rombar=0,romfile=${GPU_BIOS} \
-device vfio-pci,host=${GPU_AUDIO},bus=root.1 \
-device vfio-pci,host=${USB_PORTS},bus=pcie.0 \
-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-net nic,macaddr=${MAC_ADDR},model=virtio -net tap,ifname=tap0,script=no,downscript=no \
-drive file=${HD_PATH},id=disk,if=virtio \
-boot order=cd,menu=on

What I'm wondering is how can I rebind some devices so the host after unbinding them from VFIO-PCI?

echo 0000:00:1d.0 > /sys/bus/pci/drivers/vfio-pci/unbind

But I can't seem to find where to reattach it to the host. It's a USB controller hub from the motherboard. If I unbind it from host, then I'm losing at least 6 USB ports. I'm guessing I need to echo $vendor and $device to a new_id but I can't seem to find the right one, all I get is "echo: write error: Invalid argument".

Last edited by mv0lnicky (2014-03-06 15:22:04)

redger

Member

Registered: 2014-02-26

Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

A quick update - another success story ... thanks for all your help.

CPU = Intel 4670, MBoard = Asrock Z87 Extreme6, RAM=16GB 1600, GPU = Radeon HD6950, Arch Host Kernel 3.13, Window 7 guest, Renesas based PCIe add-on card passed to the guest

I get about 70-80% native performance under my test conditions (rfacvtor v1, using 11 AI drivers offline). Most importantly there is no micro-stutter problem and no significant latency issues (any more)
I found that allocating multiple CPUs made a noticeable difference to frame rate and subjective experience (surprisingly since I thought this app was GPU limited not CPU)
Test results reported below are expressed as frames / sec as reported in the application (all a bit subjective, but good enough to indicate what to expect). The test was deliberately set up to present a reasonable maximum load. In practice most situations present a lower load than the test case so "normal" performance will be better than reported here
                         Qemu     Qemu          Native        Native
Track Position    2 CPU    1 CPU          4 CPU       1 CPU
---------------------    ---------     --------           ---------        ---------
Pits                     30+         20+             40+           40+
Start Grid             40+         25+             60+           50+
StartF Straight      70+         30+           100+           90+

2 lap MAX            70+         50+                            100+

Note that Native mode is based on a full 4 CPUs on the Intel 4670 ..... CPU constraints are applied at the App level (there's a parameter with options all-CPUs or 1-CPU) so there's no obvious way to reproduce the guest CPU count

Online racing against other (real) people I can see frame rates over 200 fps depending on track and mod (cars) ie. quite acceptable and very useable. Furthermore there are no significant lag issues which means that it is practical to follow closely without crashing eg. if they brake suddenly you have time to react because there is no more detectable lag than a native installation.

After applying this patch to fix the Qemu assertion error (I had to modify this PCIE: fix regression with coldplugged multifunction device  to get it to apply)

diff --git a/qemu/hw/pci/pcie.c b/qemu/hw/pci/pcie.c
index 8ecd11e..02cde6f 100644
--- a/qemu/hw/pci/pcie.c
+++ b/qemu/hw/pci/pcie.c
@@ -221,29 +221,23 @@
                                          DeviceState *dev,
                                          uint8_t **exp_cap, Error **errp)
 {
-    PCIDevice *pci_dev = PCI_DEVICE(dev);
     *exp_cap = hotplug_dev->config + hotplug_dev->exp.exp_cap;
     uint16_t sltsta = pci_get_word(*exp_cap + PCI_EXP_SLTSTA);
 
-    PCIE_DEV_PRINTF(pci_dev, "hotplug state: %d\n", state);
+    PCIE_DEV_PRINTF(PCI_DEVICE(dev), "hotplug state: %d\n", state);
     if (sltsta & PCI_EXP_SLTSTA_EIS) {
         /* the slot is electromechanically locked.
          * This error is propagated up to qdev and then to HMP/QMP.
          */
         error_setg_errno(errp, -EBUSY, "slot is electromechanically locked");
     }
-
-    /* TODO: multifunction hot-plug.
-     * Right now, only a device of function = 0 is allowed to be
-     * hot plugged/unplugged.
-     */
-    assert(PCI_FUNC(pci_dev->devfn) == 0);
 }
 
 void pcie_cap_slot_hotplug_cb(HotplugHandler *hotplug_dev, DeviceState *dev,
                               Error **errp)
 {
     uint8_t *exp_cap;
+    PCIDevice *pci_dev = PCI_DEVICE(dev);
 
     pcie_cap_slot_hotplug_common(PCI_DEVICE(hotplug_dev), dev, &exp_cap, errp);
 
@@ -255,6 +249,11 @@
                                    PCI_EXP_SLTSTA_PDS);
         return;
     }
+    /* TODO: multifunction hot-plug.
+     * Right now, only a device of function = 0 is allowed to be
+     * hot plugged/unplugged.
+     */
+     assert(PCI_FUNC(pci_dev->devfn) == 0);
 
     pci_word_test_and_set_mask(exp_cap + PCI_EXP_SLTSTA,
                                PCI_EXP_SLTSTA_PDS);

And then away it went (using Qemu 1.7 from the repos failed)

Note that i did apply the following tuning factors -
-  hugetlbfs  (mine set to 3200 - for a 6GB guest)
-  -cpu Haswell,hv-time,hv_relaxed,hv_vapic,hv_spinlocks=0x1000
-  kernel compiled with CONFIG_PCI_STUB=y CONFIG_PREEMPT_VOLUNTARY=y   and  CONFIG_HZ_1000=y    and  CONFIG_HZ=1000    (in addition to the CONFIG_VFIO and parameters set to y)

final command now looks like this

qemu-system-x86_64 -name win7 -enable-kvm -M q35 -m 6144  -mem-path /dev/hugepages \
-cpu Haswell,hv-time,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 \
-boot menu=on \
-smp 2,sockets=1,cores=2,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-usbdevice tablet \
-spice port=5902,disable-ticketing \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device ahci,bus=pcie.0,id=ahci \
-net nic,macaddr=00:00:00:0a:5b:2c,model=virtio \
-net user \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-device vfio-pci,host=02:00.0,bus=pcie.0 \
-device vfio-pci,host=04:00.0,bus=pcie.0 \
-device vfio-pci,host=06:00.0,bus=pcie.0 \
-device vfio-pci,host=05:00.0,bus=pcie.0 \
-drive file=/dev/virt-test-xen/lvwin7a_kvm,id=disk1,format=raw,if=virtio \
-drive file=/dev/sda3,id=disk2,format=raw,if=virtio \
-drive file=/dev/sda2,id=disk3,format=raw,if=virtio \
-drive file=/home/redger/Downloads/isos/virtio-win-0.1-74.iso,id=isocd2 \
-device ide-cd,bus=ahci.2,drive=isocd2 \
-drive file=/home/redger/Downloads/isos/winfiles.iso,id=isocd3 \
-device ide-cd,bus=ahci.3,drive=isocd3

You can see above I'm passing through (a) GPUat 1:00:0 (b) Renesas USB card at 2:00.0 (c) NIC at 04:00.0 (d) Asmedia disk controllers at 05:00.0 and 06:00.0 (e) windows install from LVM partition (f) Virtio drivers on iso image as CD-Rom (g) Asrock motherboard drivers and Renesas drivers on iso image as CD-Rom                 plus a virtual NIC which I disable as soon as the passed through device is running. I also use Spice for initial keyboard communications until the USB card drivers are loaded and working.

Grub command line -

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on nohz=off pci-stub.ids=8086:0c01,8086:0c09,1002:6719,1002:aa80,1912:0014,8086:8c10,8086:8c14,8086:8c16,8086:8c18,1b21:0612,8086:1539 quiet"

Things to do -
- obtain kernel 3.14 with debug register patches (in addition to existing patches, most of which are presumably included in 3.14)
- install Windows on a dedicated disk (requires Asrock Asmedia drivers incorporated into initial install, hence std command includes "Winfiles" "CD-Rom" which makes those drivers available)
- Review sound - the GPU sound device is not working (I use USB wireless headphones so not a current issue)

Thanks again for all your help

anickname

Member

Registered: 2014-01-07

Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

RIght now im passing through my primary card (radeon 6950) what i did was, use pci-stub to bind it at boot and i disabled the efifb framebuffer with video=efifb:off on my grub cmdline

Are you booting using UEFI or BIOS mode ?
The onboard video is it enabled or disabled ?
What about "IGPU Multi Monitor" and "Render standby" options ?

I tried booting using BIOS, onboard GPU, both enabled and disabled; It didn't worked for me.
I see the seabios screen but after it only random pixels. I think that both Windows and Linux are
trying to write to the GPU.

I used pci-stub and video=vesafb:off in the command line.

Thanks.

mv0lnicky

Member

Registered: 2014-03-05

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What I'm wondering is how can I rebind some devices so the host after unbinding them from VFIO-PCI?

echo 0000:00:1d.0 > /sys/bus/pci/drivers/vfio-pci/unbind

But I can't seem to find where to reattach it to the host. It's a USB controller hub from the motherboard. If I unbind it from host, then I'm losing at least 6 USB ports. I'm guessing I need to echo $vendor and $device to a new_id but I can't seem to find the right one, all I get is "echo: write error: Invalid argument".

Ok I found out that rebind to host, I just need to bind to the driver. So in my case

echo "0000:00:1d.0" > /sys/bus/pci/drivers/ehci-pci/bind 

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

RIght now im passing through my primary card (radeon 6950) what i did was, use pci-stub to bind it at boot and i disabled the efifb framebuffer with video=efifb:off on my grub cmdline

Are you booting using UEFI or BIOS mode ?
The onboard video is it enabled or disabled ?
What about "IGPU Multi Monitor" and "Render standby" options ?

I tried booting using BIOS, onboard GPU, both enabled and disabled; It didn't worked for me.
I see the seabios screen but after it only random pixels. I think that both Windows and Linux are
trying to write to the GPU.

I used pci-stub and video=vesafb:off in the command line.

Thanks.

Im booting in EFI mode, and i dont have an onboard gpu, my primary card is a radeon 6950, and my secondary card is a radeon 5450, im passing though the radeon 6950

Last edited by nbhs (2014-03-06 20:19:55)

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello everyone. First time user here and I am really interested in doing something similar to this. I have read a couple of pages, but its starting to get overwhelming, considering all the new patches, kernels and other fixes. Is it perhaps possible to write an up to date guide, perhaps a bit more in depth on how we can set this up? (What do I need if I boot with UEFI or BIOS, with a NVIDIA or AMD card etc) I am perhaps speaking for myself here, but I know I would appreciate it alot.

Either way, I will definitely try this out, hopefully I wont pull out all my hairs in the process!

mv0lnicky

Member

Registered: 2014-03-05

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Some of the patches depend on your hardware. Also, if you're using an up to date distribution, you can get some results with just the compiled kernel +  acs override patch,  memleak fix and the patch for your host graphic adapter.

Last edited by mv0lnicky (2014-03-07 02:12:57)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Paolo has posted new patches to fix performance when debug registers are used in the guest.  Performance should be the same as previous set on Intel, but this series also adds support for AMD.  Patches:

http://marc.info/?l=linux-kernel&m=1394 … 4556&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4517&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4527&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4488&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4502&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4492&q=raw
http://marc.info/?l=linux-kernel&m=1394 … 4499&q=raw

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have been trying with a clean install of Arch Linux, fully updated(per 08 March 2014) and I think Im getting there, but I havent quite made it yet.
i7 processor (Haswell), Kernel 3.13.5-1
Host GPU: Intel HD 4600
Guest GPU: NVIDIA GTX 670

Nouveau and nvidia drivers are blacklisted.

This is the only thing I got in my kernel boot options: intel_iommu=on

This is my dmesg |grep IOMMU:
[    0.000000] Intel-IOMMU: enabled
[    0.036230] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.036235] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
[    0.036300] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.428676] IOMMU 0 0xfed90000: using Queued invalidation
[    0.428676] IOMMU 1 0xfed91000: using Queued invalidation
[    0.428678] IOMMU: Setting RMRR:
[    0.428686] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf800000 - 0xcf9fffff]
[    0.429887] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429907] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429924] IOMMU: Setting identity map for device 0000:00:14.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429936] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.429942] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]

vfio-bind 0000:01:00.0 0000:01:00.1 gives me this in journalctl:
Mar 08 04:26:11 arch kernel: vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Mar 08 04:26:11 arch kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

However, when trying to run this command:
qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

I get the message: Could not initialize SDL(No available video device) - exiting.
Nothing in the journalctl is logged when I try to run qemu-system, besides this message: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

Any help is greatly appreciated, as this is something I really would like to have.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have been trying with a clean install of Arch Linux, fully updated(per 08 March 2014) and I think Im getting there, but I havent quite made it yet.
i7 processor (Haswell), Kernel 3.13.5-1
Host GPU: Intel HD 4600
Guest GPU: NVIDIA GTX 670

Nouveau and nvidia drivers are blacklisted.

This is the only thing I got in my kernel boot options: intel_iommu=on

This is my dmesg |grep IOMMU:
[    0.000000] Intel-IOMMU: enabled
[    0.036230] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.036235] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
[    0.036300] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.428676] IOMMU 0 0xfed90000: using Queued invalidation
[    0.428676] IOMMU 1 0xfed91000: using Queued invalidation
[    0.428678] IOMMU: Setting RMRR:
[    0.428686] IOMMU: Setting identity map for device 0000:00:02.0 [0xbf800000 - 0xcf9fffff]
[    0.429887] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429907] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429924] IOMMU: Setting identity map for device 0000:00:14.0 [0xbc0cf000 - 0xbc0dbfff]
[    0.429936] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.429942] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]

vfio-bind 0000:01:00.0 0000:01:00.1 gives me this in journalctl:
Mar 08 04:26:11 arch kernel: vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Mar 08 04:26:11 arch kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

However, when trying to run this command:
qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

I get the message: Could not initialize SDL(No available video device) - exiting.
Nothing in the journalctl is logged when I try to run qemu-system, besides this message: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

Any help is greatly appreciated, as this is something I really would like to have.

Use the qemu -nographic arg

Last edited by nbhs (2014-03-08 03:15:33)

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Im just going to add the -nographic arg without removing anything from the qemu command?

This gives me the following in journalctl:

Mar 08 12:11:33 arch kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:hsw_unclaimed_reg_check] *ERROR* Unclaimed write to 4400c
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:37 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

Looking at the processes with htop I can see two qemu-system processes running at above 90%. But nothing else is happening.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Im just going to add the -nographic arg without removing anything from the qemu command?

This gives me the following in journalctl:

Mar 08 12:11:33 arch kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:hsw_unclaimed_reg_check] *ERROR* Unclaimed write to 4400c
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:35 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 12:11:37 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

Looking at the processes with htop I can see two qemu-system processes running at above 90%. But nothing else is happening.

Are you using the kernel i provided?

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Of course, I forgot that. I have compiled it now, installed it and rebooted.
Journalctl gives me:

Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:09 arch kernel: [drm:hsw_unclaimed_reg_clear] *ERROR* Unknown unclaimed register before writing to 44040
Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:11 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

Tested with both -nographic and without - nothing happens.

(Will these patches be implemented upstream in the future, so we dont have to compile our own kernel all the time?)

As before, Im grateful for your help.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Of course, I forgot that. I have compiled it now, installed it and rebooted.
Journalctl gives me:

Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:09 arch kernel: [drm:hsw_unclaimed_reg_clear] *ERROR* Unknown unclaimed register before writing to 44040
Mar 08 13:27:09 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
Mar 08 13:27:11 arch kernel: [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

Tested with both -nographic and without - nothing happens.

(Will these patches be implemented upstream in the future, so we dont have to compile our own kernel all the time?)

As before, Im grateful for your help.

Missing i915 patches

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I downloaded this: linux-mainline.tar.gz (3.13.5 includes acs override patch, i935 vga arbiter fixes, memleak fix by xani)
tar xfvz
makepkg -s
pacman -U
reboot

Is there something extra I should have done? By i915 patch you mean the i935 vga arbiter fix?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I downloaded this: linux-mainline.tar.gz (3.13.5 includes acs override patch, i935 vga arbiter fixes, memleak fix by xani)
tar xfvz
makepkg -s
pacman -U
reboot

Is there something extra I should have done? By i915 patch you mean the i935 vga arbiter fix?

That patch is included in my kernel package, make sure you have installed it and you are running it

uname -r

3.13.5-1-mainline

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

uname -r
3.13.5-1-ARCH

What am I doing wrong? (sudo pacman -U linux-mainline-3.13.5-1-x86_64.pkg.tar.xz)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

uname -r
3.13.5-1-ARCH

What am I doing wrong? (sudo pacman -U linux-mainline-3.13.5-1-x86_64.pkg.tar.xz)

sudo pacman -U linux-mainline-*
sudo grub-mkconfig -o /boot/grub/grub.cfg

You should see a linux mainline boot option on the grub menu now

trimm

Member

Registered: 2014-03-06

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Oh stupid, stupid me! I forgot to put the new kernel in my Gummiboot.
Loading into X-session takes a while now, before it was rather instant. But lo and behold my GTX 670 has been passthroughed to my Windows 8.1WM!
Thank you so much, I am now one (major) step closer to getting this just how I want it to be

zoop2

Member

Registered: 2014-03-08

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello all,  first thanks for all the work people are putting into this.  It has been a fun project.

I wanted to share my experience, so that it might help others as well.

My system is a Mac Pro (early 2008)  has 8 cores from Xeon E5462's

The Hosts Video card:
02:00.0 VGA compatible controller: NVIDIA Corporation G92 [GeForce 8800 GT] (rev a2)
Passthrough Video card:
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Turks XT [Radeon HD 6670/7670]

I am using Gentoo with Gentoo's moded kernel 3.12.13  I was using QEMU 1.6 from portage,  Now I have changed to the GIT for some new features.  And I have Seabios 1.7.3 also from Portage

So I first had this VM using libvirt.  I tried playing with it to get it to pass the card to windows without success.  I may try to play wi
th it again at a later date.  But I think getting it going without it is a little easier.  I had a lot of problems getting Windows to Boot
when passing the Vfio stuff.  I tried the various things with -M and other config settings.  I ran into various issues.  Like tracebacks fr
om the host  QEMU crashing on start.  and windows crashing on start with just a blank blue screen etc.  I ended up getting around this by b
ooting the system with the original config again and converting everything to only use virtio drivers.   From what I saw debuging all of th
e different ide/sata controllers I tried to setup was causing these problems.  I am sure I could have been making mistakes, but this is wha
t I did.

After converting to that I had a bit of problems with the Ati driver in windows crashing it, but I think that adding these kernel options to the host stopped that from happening. intel_iommu=on,igfx_off,pass-through

I am still having a problem with audio going through the Hosts audio device.  This seems to be a known issue of using ALSA with windows gue
sts.  I dislike pulseaudio so I won't install it.

Here is the line I am using to start the system for now.

/home/zoop/tmp/qemu-git/qemu/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -M q35 -cpu qemu64,hv-time,hv-relaxed,hv-vapic,hv-spinlocks=0x1000 -mem-path /dev/hugepages -device virtio-balloon -m 6144 -smp 4,sockets=1,cores=4,threads=1 -bios /usr/share/seabios/bios.bin -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 -boot menu=on -vga none -drive if=virtio,file=/mnt/backup/vm/win8-Mar2014snap.qcow2,id=disk,format=qcow2 -usb -usbdevice host:047d:1048 -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=no

From here i am looking to implement the Debug registers patch that was mentioned by @aw a while back to improve performance.  I am wondering what kernel that Patch was for, I am assuming it was for 3.13.x.

To help imporove performance a littl more i also plan on assigning the system to use specific CPUs.