Arch Linux

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

refering to problem on post https://bbs.archlinux.org/viewtopic.php … 2#p1428122

so i have sandy bridge i72600 = pcie2.0
560ti = pcie2.0
660ti = pcie3.0
770ti = pcie3.0
mbo= g1 sniper 3=pcie3.0

could this be reason for lagging hdmi sound and i think lower perfomance on kepler+ cards? 560ti works fine...

spicy

Member

Registered: 2014-06-09

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

    qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: Device does not support requested feature x-vga

This means that either a) your kernel does not support CONFIG_VFIO_PCI_VGA or b) the device is not a VGA device.  To test a):

$ grep CONFIG_VFIO_PCI_VGA /boot/config-`uname -r`

Yep, I had to re-compile with CONFIG_VFIO_PCI_VGA. At the time I think I was using a stock kernel which needed some adjustments. Thanks to your guidance the guides provided by the OP, I was able to get everything running with a GTX 770 – working great.

I'm actually doing two builds with similar hardware (minus the video cards, anyway) and ran into a problem recently with my second box and a capture card. From what I've read (and it's even been brought up on the last couple of pages) this is the whole ACS override issue – the capture card looks like it is behind a bridge so I'm getting the error:

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=pcie.0: vfio: error, group 8 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

I haven't been able to test with the ACS patch and

pcie_acs_override=downstream

on because I can't apply the patch to the kernel for some reason – not sure how to troubleshoot why it won't apply properly. I'm using 3.15 from here and getting the following:

$ patch -p1 < acs.diff
patching file Documentation/kernel-parameters.txt
Hunk #1 succeeded at 2574 with fuzz 2 (offset 225 lines).
patching file drivers/pci/quirks.c
Hunk #1 FAILED at 3292.
1 out of 1 hunk FAILED -- saving rejects to file drivers/pci/quirks.c.rej
patch unexpectedly ends in middle of line

and then the output of the rejects file:

# cat  drivers/pci/quirks.c.rej
--- drivers/pci/quirks.c
+++ drivers/pci/quirks.c
@@ -3292,11 +3292,113 @@
 	return pci_dev_get(dev);
 }

+static bool acs_on_downstream;
+static bool acs_on_multifunction;
+
+#define NUM_ACS_IDS 16
+struct acs_on_id {
+	unsigned short vendor;
+	unsigned short device;
+};
+static struct acs_on_id acs_on_ids[NUM_ACS_IDS];
+static u8 max_acs_id;
+
+static __init int pcie_acs_override_setup(char *p)
+{
+	if (!p)
+		return -EINVAL;
+
+	while (*p) {
+		if (!strncmp(p, "downstream", 10))
+			acs_on_downstream = true;
+		if (!strncmp(p, "multifunction", 13))
+			acs_on_multifunction = true;
+		if (!strncmp(p, "id:", 3)) {
+			char opt[5];
+			int ret;
+			long val;
+
+			if (max_acs_id >= NUM_ACS_IDS - 1) {
+				pr_warn("Out of PCIe ACS override slots (%d)\n",
+					NUM_ACS_IDS);
+				goto next;
+			}
+
+			p += 3;
+			snprintf(opt, 5, "%s", p);
+			ret = kstrtol(opt, 16, &val);
+			if (ret) {
+				pr_warn("PCIe ACS ID parse error %d\n", ret);
+				goto next;
+			}
+			acs_on_ids[max_acs_id].vendor = val;
+
+			p += strcspn(p, ":");
+			if (*p != ':') {
+				pr_warn("PCIe ACS invalid ID\n");
+				goto next;
+			}
+
+			p++;
+			snprintf(opt, 5, "%s", p);
+			ret = kstrtol(opt, 16, &val);
+			if (ret) {
+				pr_warn("PCIe ACS ID parse error %d\n", ret);
+				goto next;
+			}
+			acs_on_ids[max_acs_id].device = val;
+			max_acs_id++;
+		}
+next:
+		p += strcspn(p, ",");
+		if (*p == ',')
+			p++;
+	}
+
+	if (acs_on_downstream || acs_on_multifunction || max_acs_id)
+		pr_warn("Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA\n");
+
+	return 0;
+}
+early_param("pcie_acs_override", pcie_acs_override_setup);
+
+static int pcie_acs_overrides(struct pci_dev *dev, u16 acs_flags)
+{
+	int i;
+
+	/* Never override ACS for legacy devices or devices with ACS caps */
+	if (!pci_is_pcie(dev) ||
+	    pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS))
+		return -ENOTTY;
+
+	for (i = 0; i < max_acs_id; i++)
+		if (acs_on_ids[i].vendor == dev->vendor &&
+		    acs_on_ids[i].device == dev->device)
+			return 1;
+
+	switch (pci_pcie_type(dev)) {
+	case PCI_EXP_TYPE_DOWNSTREAM:
+	case PCI_EXP_TYPE_ROOT_PORT:
+		if (acs_on_downstream)
+			return 1;
+		break;
+	case PCI_EXP_TYPE_ENDPOINT:
+	case PCI_EXP_TYPE_UPSTREAM:
+	case PCI_EXP_TYPE_LEG_END:
+	case PCI_EXP_TYPE_RC_END:
+		if (acs_on_multifunction && dev->multifunction)
+			return 1;
+	}
+
+	return -ENOTTY;
+}
+
 static const struct pci_dev_acs_enabled {
 	u16 vendor;
 	u16 device;
 	int (*acs_enabled)(struct pci_dev *dev, u16 acs_flags);
 } pci_dev_acs_enabled[] = {
+	{ PCI_ANY_ID, PCI_ANY_ID, pcie_acs_overrides },
 	{ 0 }
 };

I'm not super familiar with applying kernel patches, but was able to get the i915 VGA patch applied fine earlier on this same kernel version. Also not a C programmer so I'm kind of useless troubleshooting what didn't apply.

Any tips would be appreciated, it is probably something simple that I am missing here (hope I'm not being too frustrating). Also thanks again aw and everyone else who has contributed information, code and guides for this – awesome stuff! I'm hoping to put together a guide of all the specific steps I had to take for my hardware as a sort of specific mini-guide once everything is working to contribute something back in a small way.

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

    qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: Device does not support requested feature x-vga

This means that either a) your kernel does not support CONFIG_VFIO_PCI_VGA or b) the device is not a VGA device.  To test a):

$ grep CONFIG_VFIO_PCI_VGA /boot/config-`uname -r`

Yep, I had to re-compile with CONFIG_VFIO_PCI_VGA. At the time I think I was using a stock kernel which needed some adjustments. Thanks to your guidance the guides provided by the OP, I was able to get everything running with a GTX 770 – working great.

I'm actually doing two builds with similar hardware (minus the video cards, anyway) and ran into a problem recently with my second box and a capture card. From what I've read (and it's even been brought up on the last couple of pages) this is the whole ACS override issue – the capture card looks like it is behind a bridge so I'm getting the error:

qemu-system-x86_64: -device vfio-pci,host=02:00.0,bus=pcie.0: vfio: error, group 8 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.

I haven't been able to test with the ACS patch and

pcie_acs_override=downstream

on because I can't apply the patch to the kernel for some reason – not sure how to troubleshoot why it won't apply properly. I'm using 3.15 from here and getting the following:

$ patch -p1 < acs.diff
patching file Documentation/kernel-parameters.txt
Hunk #1 succeeded at 2574 with fuzz 2 (offset 225 lines).
patching file drivers/pci/quirks.c
Hunk #1 FAILED at 3292.
1 out of 1 hunk FAILED -- saving rejects to file drivers/pci/quirks.c.rej
patch unexpectedly ends in middle of line

and then the output of the rejects file:

# cat  drivers/pci/quirks.c.rej
--- drivers/pci/quirks.c
+++ drivers/pci/quirks.c
@@ -3292,11 +3292,113 @@
 	return pci_dev_get(dev);
 }

+static bool acs_on_downstream;
+static bool acs_on_multifunction;
+
+#define NUM_ACS_IDS 16
+struct acs_on_id {
+	unsigned short vendor;
+	unsigned short device;
+};
+static struct acs_on_id acs_on_ids[NUM_ACS_IDS];
+static u8 max_acs_id;
+
+static __init int pcie_acs_override_setup(char *p)
+{
+	if (!p)
+		return -EINVAL;
+
+	while (*p) {
+		if (!strncmp(p, "downstream", 10))
+			acs_on_downstream = true;
+		if (!strncmp(p, "multifunction", 13))
+			acs_on_multifunction = true;
+		if (!strncmp(p, "id:", 3)) {
+			char opt[5];
+			int ret;
+			long val;
+
+			if (max_acs_id >= NUM_ACS_IDS - 1) {
+				pr_warn("Out of PCIe ACS override slots (%d)\n",
+					NUM_ACS_IDS);
+				goto next;
+			}
+
+			p += 3;
+			snprintf(opt, 5, "%s", p);
+			ret = kstrtol(opt, 16, &val);
+			if (ret) {
+				pr_warn("PCIe ACS ID parse error %d\n", ret);
+				goto next;
+			}
+			acs_on_ids[max_acs_id].vendor = val;
+
+			p += strcspn(p, ":");
+			if (*p != ':') {
+				pr_warn("PCIe ACS invalid ID\n");
+				goto next;
+			}
+
+			p++;
+			snprintf(opt, 5, "%s", p);
+			ret = kstrtol(opt, 16, &val);
+			if (ret) {
+				pr_warn("PCIe ACS ID parse error %d\n", ret);
+				goto next;
+			}
+			acs_on_ids[max_acs_id].device = val;
+			max_acs_id++;
+		}
+next:
+		p += strcspn(p, ",");
+		if (*p == ',')
+			p++;
+	}
+
+	if (acs_on_downstream || acs_on_multifunction || max_acs_id)
+		pr_warn("Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA\n");
+
+	return 0;
+}
+early_param("pcie_acs_override", pcie_acs_override_setup);
+
+static int pcie_acs_overrides(struct pci_dev *dev, u16 acs_flags)
+{
+	int i;
+
+	/* Never override ACS for legacy devices or devices with ACS caps */
+	if (!pci_is_pcie(dev) ||
+	    pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS))
+		return -ENOTTY;
+
+	for (i = 0; i < max_acs_id; i++)
+		if (acs_on_ids[i].vendor == dev->vendor &&
+		    acs_on_ids[i].device == dev->device)
+			return 1;
+
+	switch (pci_pcie_type(dev)) {
+	case PCI_EXP_TYPE_DOWNSTREAM:
+	case PCI_EXP_TYPE_ROOT_PORT:
+		if (acs_on_downstream)
+			return 1;
+		break;
+	case PCI_EXP_TYPE_ENDPOINT:
+	case PCI_EXP_TYPE_UPSTREAM:
+	case PCI_EXP_TYPE_LEG_END:
+	case PCI_EXP_TYPE_RC_END:
+		if (acs_on_multifunction && dev->multifunction)
+			return 1;
+	}
+
+	return -ENOTTY;
+}
+
 static const struct pci_dev_acs_enabled {
 	u16 vendor;
 	u16 device;
 	int (*acs_enabled)(struct pci_dev *dev, u16 acs_flags);
 } pci_dev_acs_enabled[] = {
+	{ PCI_ANY_ID, PCI_ANY_ID, pcie_acs_overrides },
 	{ 0 }
 };

I'm not super familiar with applying kernel patches, but was able to get the i915 VGA patch applied fine earlier on this same kernel version. Also not a C programmer so I'm kind of useless troubleshooting what didn't apply.

Any tips would be appreciated, it is probably something simple that I am missing here (hope I'm not being too frustrating). Also thanks again aw and everyone else who has contributed information, code and guides for this – awesome stuff! I'm hoping to put together a guide of all the specific steps I had to take for my hardware as a sort of specific mini-guide once everything is working to contribute something back in a small way.

seems like you try to apply "broken" patch:

Hunk #1 FAILED at 3292.
1 out of 1 hunk FAILED -- saving rejects to file drivers/pci/quirks.c.rej
patch unexpectedly ends in middle of line

(note last line)
you probably decided to ignore some "unneded" empty lines, which are in fact important for patch command to verify that applied patch is well-formed/valid

spicy

Member

Registered: 2014-06-09

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

seems like you try to apply "broken" patch:

Hunk #1 FAILED at 3292.
1 out of 1 hunk FAILED -- saving rejects to file drivers/pci/quirks.c.rej
patch unexpectedly ends in middle of line

(note last line)
you probably decided to ignore some "unneded" empty lines, which are in fact important for patch command to verify that applied patch is well-formed/valid

Hi sinny,

Thanks for your reply. As far as I can tell, the patch is the correct one – I've downloaded it directly from Alex's LKML posting using wget to directly download the diff:

wget -O acs.diff https://lkml.org/lkml/diff/2013/5/30/513/1

I should note that the last line about "unexpectedly ending in the middle of line" also occurs in the i915 patch which I can apply successfully, as far as I can tell that is expected behaviour.

Any ideas?

ahl

Member

Registered: 2014-03-16

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

seems like you try to apply "broken" patch:

Hunk #1 FAILED at 3292.
1 out of 1 hunk FAILED -- saving rejects to file drivers/pci/quirks.c.rej
patch unexpectedly ends in middle of line

(note last line)
you probably decided to ignore some "unneded" empty lines, which are in fact important for patch command to verify that applied patch is well-formed/valid

Hi sinny,

Thanks for your reply. As far as I can tell, the patch is the correct one – I've downloaded it directly from Alex's LKML posting using wget to directly download the diff:

wget -O acs.diff https://lkml.org/lkml/diff/2013/5/30/513/1

I should note that the last line about "unexpectedly ending in the middle of line" also occurs in the i915 patch which I can apply successfully, as far as I can tell that is expected behaviour.

Any ideas?

That patch is very old. Download the linux-mainline.tar.gz from first post.

$ tar xf linux-3.15.tar.xz
$ tar xf linux-mainline.tar.gz
$ cd linux-3.15
$ cat ../linux-mainline/override_for_missing_acs_capabilities.patch | patch -p1

Also since 3.15.1 is already out, you might want to upgrade to that too. Here is one example how to do that:

$ wget https://www.kernel.org/pub/linux/kernel/v3.x/patch-3.15.1.xz
$ cd linux-3.15
$ xzcat ../patch-3.15.1.xz | patch -p1
$ cd ..
$ mv linux-3.15 linux-3.15.1

...alternatively, you can just download linux-3.15.1.tar.xz

stefan_vgapass

Member

Registered: 2014-03-17

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have finally had some time to try, again, a Linux guest with kvm vga passthrough.
Last time I tried with an Kubuntu 13.10 x64 guest I got the BSOD after installing the ATI drivers.
To my surprise this time with Kubuntu 14.04 x64 it just worked.

Host specs:
check my previous post, the host system is the same.
https://bbs.archlinux.org/viewtopic.php … 4#p1393614

Guest specs:

OS:      Kubuntu 14.04 x64
CPU:     2 cores x 2 threads Opteron emulated CPU
RAM:     8 GB of ram ( I kept the ram the same size as for the windows guest )
GPU:     XFX Radeon HD 7850 2GB RAM
Disk:    virtio ( running 2 HDD: 60GB for / and 128GB for /home )
Network: virtio bridge
Driver:  latest fglrx ATI driver, at the time of writing, Catalyst 14.4

Guest startup script:

/opt/qemu-git/bin/qemu-system-x86_64 -enable-kvm -M q35 \
-m 8196 -mem-prealloc -mem-path /mnt/hugepages \
-cpu Opteron_G5 \
-smp 4,sockets=1,cores=2,threads=2 \
-bios /root/kvm-vga-passthrough/vfio-post3.13/seabios/seabios/out/bios.bin -vga none \
-boot order=cd,menu=on \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=04:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=00:1d.0,bus=root.1,addr=00.1 \
-device vfio-pci,host=00:1b.0,bus=root.1,addr=00.2 \
-device ahci,bus=pcie.0,id=ahci \
-netdev bridge,br=br0,id=hostnet0 \
-device virtio-net-pci,netdev=hostnet0,id=net0 \
-drive file=/var/lib/libvirt/images/kubuntu_1404_x64-disk01.qcow2,id=disk,format=qcow2 -device ide-hd,bus=ahci.0,drive=disk \
-drive file=/var/lib/libvirt/images/kubuntu_1404_x64-disk02.qcow2,id=disk02,format=qcow2 -device ide-hd,bus=ahci.1,drive=disk02 \
-drive file=/var/lib/libvirt/images/kubuntu-14.04-desktop-amd64.iso,id=isocd -device ide-cd,bus=ahci.2,drive=isocd

Tested:

I've only been able to test using Team Fortress 2 and Dota 2.

In Dota 2 I've measured the FPS running between max 60 fps and min 29 fps usually averaging 45 fps.
Compared with the windows VM were I get max 60 fps and min 30 averaging 58 fps.

In Team fortress 2 I've measured the FPS running between max 120 and min 76 averaging 65 FPS.
Compared with the windows VM were I get max 110 and min 19 averaging 44 FPS.

Looks like TF2 works better in Linux and Dota 2 works better in Windows on a VGA passthrough VM with an ATI graphics card.
The difference is not enourmeous though !
I don't know how they compare versus a native windows install as I have no windows install to compare to.

Steam client on linux related bugs:
For posterity I'm putting here the 2 bugs I've experienced whilst running Steam client on linux with an ATI graphics card.

1. I had to reinstall the ATI driver
I had one very strange occurance that I could only fix by reinstalling the ATI Catalyst driver.
I've played on Steam on this guest a few days ago, but when I tried it again today Steam couldn't detect that the Catalyst driver is running OpenGL 4.4.
It was complaining that it is running OpenGL 1.4 and it can't work with OpenGL < 2.0.
The exact error was:

PROBLEM: You appear to have OpenGL 1.4.0, but we need at least 2.0.0!
Could not find required OpenGL entry point 'glGetError'! Either your video card is unsupported, or your OpenGL driver needs to be updated.

2. Steam doesn't correctly detect that I'm runing fglrx.

I had to forcibly tell the Steam client that I'm using fglrx as the video driver.

sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1.2.0 /usr/lib/i386-linux-gnu/mesa/libGL.so.1
sudo ln -s /usr/lib/fglrx/fglrx-libGL.so.1.2 /usr/lib/i386-linux-gnu/mesa/libGL.so.1.2.0

The link that led me to this fix, it is the last post on this steam thread: http://steamcommunity.com/app/221410/di … =german#p2

Next to try SteamOS with vga passthrough, although I expect it to be just as straightforward and easy to setup as this linux guest was !

thehighhat

Member

Registered: 2014-05-12

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Status update to my setup.  After lots of trial and error and tweaking, the minimum amount of steps that works 100% with virsh is:

- motherboard VGA for host (Intel 5520 chipset)
- nvidia gpu for vga passthrough to guest
- ubuntu 14.04 x64_64
- qemu kvm

-- add "intel_iommu=on" to GRUB_CMDLINE_LINUX in /etc/grub/default 
-- create file /etc/modprobe.d/vfio_iommu_type.conf with contents "options vfio_iommu_type1 allow_unsafe_interrupts=1"
-- add "blacklist nouveau" line to /etc/modprobe.d/blacklist-framebuffer.conf
-- set user and group to "root" in /etc/libvirt/qemu.conf
-- add the correct /dev/vfio/#### entry to cgroup_device_acl in /etc/libvirt/qemu.conf
-- use the OP vfio-bind script with the id found from "lscpi -v | grep -i vga"
-- create a VM (virt-manager is quick and dirty way to do this)
-- download or extract the romfile for your GPU
-- then virsh edit YOUR_VM
-- -- set the first line <domain ...> to "<domain type='kvm' id='3' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>"
-- -- add a block similar to this to the xml
"   <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=02:00.0,bus=pci.0,multifunction=on,x-vga=on,romfile=/path/to/your/gpu.rom'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=02:00.1,bus=pci.0'/>
    <qemu:arg value='-bios'/>
    <qemu:arg value='/usr/share/qemu/bios.bin'/>
  </qemu:commandline>"
-- virt-manager to delete the display and video entries
-- if using apparmor, turn off aa-enforce on the /etc/apparmor.d/libvirt/libvirt-#####-YOUR_VMs_ENTRY

and that's it.  start the VM using virsh start or virt-manager

mv0lnicky

Member

Registered: 2014-03-05

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

refering to problem on post https://bbs.archlinux.org/viewtopic.php … 2#p1428122

so i have sandy bridge i72600 = pcie2.0
560ti = pcie2.0
660ti = pcie3.0
770ti = pcie3.0
mbo= g1 sniper 3=pcie3.0

could this be reason for lagging hdmi sound and i think lower perfomance on kepler+ cards? 560ti works fine...

Try changing the slot of the same GPU you're passing to the guest. If the problem stops, you know you're hitting a bottleneck with the PCIE->PCH->DMI bus.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Try changing the slot of the same GPU you're passing to the guest. If the problem stops, you know you're hitting a bottleneck with the PCIE->PCH->DMI bus.

Can u pls explain what do u mean by changing slots?  bottleneck? cause of pcie2.0? i checked lspci and it says  for geforce 660 LnkSta: Speed 2.5GT/s, Width x16,

yeah i tested... pcie 3.0 gpus crashes with hdmi sound on gigbyte g1 sniper 3 in any port,, i belive pcie gen3 is problem... but there is no option in bios... i have option on my x77-ud3h so ill try later to lower pcie to 2.0.

Last edited by slis (2014-06-22 12:11:13)

ebinmaymay

Member

Registered: 2014-06-22

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I would like to post my own success story too. A video from youtube brought me here and motivated me to move away from vmware.

I have an 8-core AMD 8320 cpu with a ASUS M5A97 EVO R2.0 mobo. It has a AMD970 chipset and supports both VT-i and VT-d. I'm using (k)ubuntu with the latest lowlatency kernel from kernel-ppa. (I should just switch to Arch.)

I'm using a nVidia GTX 550 Ti as my primary card and my Radeon HD 7750 as my passed-through card. (The open source nvidia driver has gotten rather good to my surprise after uninstalling the proprietary driver. )
I had some success going the other way as well, but ran into the usual nvidia driver detecting hypervisor problem. (I read somewhere that you can pass a qemu option to hide kvm's presence to the guest, but I don't feel like swapping them around yet again just to test it. )

I'm going to give a blow-by-blow description of what I did, so feel free to skip most of it.

===BEGIN BLOW BY BLOW===
First of all I did an lspci and inspection of /sys/devices/ and discovered that my second card appears at 07:00:00 and 07:00:01, with its bridge at 00:15:00.

tux@oniichan:~$ lspci
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750 / R7 250E]           
07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
tux@oniichan:~$ lspci -n
00:15.0 0604: 1002:43a0
07:00.0 0300: 1002:683f
07:00.1 0403: 1002:aab0
tux@oniichan:~$ find /sys/devices -iname '0000:07:00.0'
/sys/devices/pci0000:00/0000:00:15.0/0000:07:00.

Then I added pci-stub to the modules inside the initramfs blob at /etc/initramfs-tools/modules.

sudo nano /etc/initramfs-tools/modules
sudo update-initramfs -u -k all

Then added "pci-stub.ids=1002:683f,1002:aab0" to the kernel command by adding it to GRUB_CMDLINE_LINUX in /etc/default/grub.

sudo nano /etc/default/grub
sudo update-grub

It's worth noting that I ended up not blacklisting the kernel's baked-in radeon driver, nor did I turn on unsafe interrupts.

At this stage I rebooted to check that it was catching the card. It was.

[   16.083836] pci-stub: add 1002:683F sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[   16.083866] pci-stub 0000:07:00.0: claimed by stub
[   16.083878] pci-stub: add 1002:AAB0 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[   16.083893] pci-stub 0000:07:00.1: claimed by stub

I added the vfio-script as provided by the OP and made an upstart script at /etc/init/vfio-pci

# vfio-pci - bind certain pci devices to vfio-pci

description "pci to vfio autobinder"

start on runlevel [2345]

task

script
    vfio-bind 0000:00:15.0 0000:07:00.0 0000:07:00.1
end script

Rebooted, and then ran qemu as per everyone else ITT.
===END BLOW BY BLOW==

HOWEVER, I was not able to pass through my graphics card with a copy-paste of the OP's instructions.
So I gave up, installed windows 7 using sdl, first with the old default computer model, then with the virtio drivers, and then with the q35 drivers.
For the benefit of the newbies, I passed an extra qcow2 image through a virtio adapter. win7 picked it up and asked for the virtio driver then.
Thankfully I have been running a ftp server using vsftpd for sharing anime in my household, so it wasn't troublesome to get driver/software isos into the os and mount them.
fyi, I used SlySoft's Virtual CloneDrive instead of Daemon Tools because it doesn't inject shellcode to do so, although feel free to use whatever you like.

After getting everything else sorted, I finally tackled the pcie passthrough problem and only two days ago, after a week of fiddling, did I get it to work with my AMD card. Whenever I tried to pass it through, the card would turn on and do nothing.
I even extracted the romfile from my card via sysfs...

sudo bash
echo '1' > /sys/devices/pci0000:00/0000:00:15.0/0000:07:00.0/romfile
dd if=/sys/devices/pci0000:00/0000:00:15.0/0000:07:00.0/romfile of=radeon.rom

...but it still didn't do anything.

After a week of fiddling, I got OP's test command to work by passing a dummy ISA(!!!) cirrus graphics adapter.
When this isa card was passed through, the cirrus graphics adapter ALWAYS displays blank. The boot screen and splash appears on my second monitor.
I suspect that it boots the cirrus and it spoofs a boot of the pci, because my custom bootsplash of which I am immensely proud has distorted colors.

I also got the emulated audio device to work flawlessly by dropping the audio adapter on my gfx adapter, then passing it the ich9 version rather than the default ich6. Remember to reboot win7 at least once between hw changes. :-)
So my final working script as of this morning is the following:

#!/bin/bash
export DISPLAY=:0
export QEMU_AUDIO_DRV=alsa

xrandr --output DVI-I-1 --off

sudo qemu-system-x86_64 \
  -enable-kvm \
  -cpu host,hv-time \
  -smp 4,cores=4 \
  -m 4G \
  -M q35 \
  -ctrl-grab \
  -vga none \
  -bios /usr/share/qemu/bios-256k.bin \
  -net user,smb=/home/tux/share \
  -net nic,model=virtio \
  -usb \
  -usbdevice host:8564:1000 \
  -usbdevice host:045e:00d1 \
  -device isa-cirrus-vga \
  -device ioh3420,bus=pcie.0,addr=1c,multifunction=on,port=1,chassis=1,id=root.1 \
  -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
  -device ich9-intel-hda \
  -device hda-micro \
  -drive if=virtio,file=/home/tux/vms/kvm/win7.img,media=disk \
  -drive if=ide,file=/home/ani/Packages/iso/win7-sp0-x64.iso,media=cdrom \
  -drive if=ide,file=/home/tux/vms/kvm/amdgfx.iso,media=cdrom \
  -boot menu=on,splash=/home/tux/vms/kvm/boot480.jpg,splash-time=5000 \
  -name win7

xrandr --output HDMI-1 --auto --primary --output DVI-I-1 --auto --left-of HDMI-1

Those two usb devices are a usb thumbdrive and the mouse from my HTPC which I use whenever something goes wrong.

guest system specs

Last edited by ebinmaymay (2014-06-22 09:23:40)

mdm

Member

Registered: 2014-06-22

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have been working through getting this setup on my laptop.  I am at the point now where I just get a black screen when booting up my virtual machine.  Here are my laptop specs:

Sager np8250
CPU: i7-4800MQ w/ HD 4600
MB Chipset: HM87
GPU: GTX 780M using Optimus

Before I get into more troubleshooting, is it possible to even use the GTX 780M with the pass through since it is on a laptop with Optimus?

AKSN74

Member

From: Taichung, Taiwan

Registered: 2014-02-18

Posts: 52

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have been working through getting this setup on my laptop.  I am at the point now where I just get a black screen when booting up my virtual machine.  Here are my laptop specs:

Sager np8250
CPU: i7-4800MQ w/ HD 4600
MB Chipset: HM87
GPU: GTX 780M using Optimus

Before I get into more troubleshooting, is it possible to even use the GTX 780M with the pass through since it is on a laptop with Optimus?

I think it is impossible, because NVIDIA GPU /w Optimus doesn't have any independent layout.
All video layout from GTX780M connected to HD4600.

You can't get any output unless GTX780M got independent output.

Or maybe you can try the previous post that someone mount his Tesla GPU into VM.

rakataka

Member

Registered: 2014-06-19

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, I have been running Windows 8 64bit with an Nvidia GTX 770 successfully for some time now. In the qemu command parameter for the VGA card (-device) I do *not* specify the romfile sub-parameter as suggested in the 1st post and it works seemingly fine. Does anyone know why the romfile parameter is suggested or what it accomplishes?

My qemu command is:

qemu-system-x86_64 -name Windows-8.1-Pro -enable-kvm 
-machine q35,accel=kvm -m 8192 -cpu Haswell,hv_relaxed,hv_vapic,hv_spinlocks=0x1000,hv_time 
-smp 3,sockets=1,cores=3,threads=1 
-bios /usr/share/qemu/bios.bin -balloon none -serial none -parallel none 
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 
-device virtio-scsi-pci,id=scsi 
-drive file=/mnt/windows-8.1.img,id=disk,format=raw -device ide-hd,bus=ide.0,unit=0,drive=disk -boot menu=on 
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on 
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 
-vga none -net none --monitor stdio -device qxl

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, I have been running Windows 8 64bit with an Nvidia GTX 770 successfully for some time now. In the qemu command parameter for the VGA card (-device) I do *not* specify the romfile sub-parameter as suggested in the 1st post and it works seemingly fine. Does anyone know why the romfile parameter is suggested or what it accomplishes?

The romfile parameter is only necessary if you cannot reliably read the ROM from the device.  This may also be necessary in cases where the VGABIOS is not provided via a PCI ROM on the host, such as might be the case on a laptop.  The effect of using romfile is simply that we read the ROM from the file instead of the device.  Since the original instructions were written we've refactored a lot of the VFIO ROM code and added bus resets, so ROM access should be much more reliable than it once was.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

belliash

Member

Registered: 2014-04-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have been working through getting this setup on my laptop.  I am at the point now where I just get a black screen when booting up my virtual machine.  Here are my laptop specs:

Sager np8250
CPU: i7-4800MQ w/ HD 4600
MB Chipset: HM87
GPU: GTX 780M using Optimus

Before I get into more troubleshooting, is it possible to even use the GTX 780M with the pass through since it is on a laptop with Optimus?

Install Windows (8 preferrably) using additional emulated VGA like currus or vmware. Then install nvidia drivers and check if it will be working, or if you get error 43.
If you got error 43, check if disabling and re-enablig nvidia card helps.

ttouch

Member

From: /dev/null

Registered: 2012-05-27

Posts: 130

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,

After kernel update (linux-mainline provided), I recompiled nvidia-beta-al and started the VM and I get this: `device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: VFIO 0000:02:00.0 BAR 0 mmap unsupported. Performance may be slow` and no output. Any help please?
BTW: I had to also attach 02:00.1 to avoid an iommu_group error.

Thank you.

---

GitHub | Git Server | Blog

penetal

Member

Registered: 2014-06-15

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm stuck on black screen from the passed through VGA card. So here's what I got, and where I'm stuck:

Spec:

• ASRock z97 Extreme 6

• Intel i7 4970 /w HD4600 (main gpu)

• Gainwind Phantom Nvidia GTX680 (secondary gpu)

I followed the 1st post and installed the mainline kernel from the package linked there:

3.15.1-1-mainline

Packages come from pacman's repos, so standard qemu, seabios and the likes.

My Nvidia card:

% lspci |grep -i nvidia                                                    :(
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 680] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)

% lspci -n |grep 01:00
01:00.0 0300: 10de:1180 (rev a1)
01:00.1 0403: 10de:0e0a (rev a1)

Current boot command:

BOOT_IMAGE=/rootvol/boot/vmlinuz-linux-mainline root=UUID=341a7e57-91cb-480c-be5f-2eb405ffd658 rw rootflags=subvol=rootvol intel_iommu=on pci-stub.ids=10de:1180,10de:0e0a quiet

Modprobes:

% cat /etc/modprobe.d/blacklist.conf /etc/modprobe.d/kvm.conf /etc/modprobe.d/vfio_iommu_type.conf
blacklist nouveau
options kvm ignore_msrs=1
options vfio_iommu_type1 allow_unsafe_interrupts=1

Configs for scripts from 1st post (I run vfio-bind.sh manually before I start VMs):

% cat /etc/vfio-pci.cfg 
DEVICES="0000:01:00.0 000:01:00.1"

% cat vfio-bind.sh 
#!/bin/sh
vfio-bind 0000:01:00.0 0000:01:00.1

Qemu VM:

% cat StartWindowsVm1.sh 
#!/bin/sh
qemu-system-x86_64 -enable-kvm -M q35 -m 10240 -cpu host \
-smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/qemu/bios.bin \
-vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=0.0,multifunction=on,x-vga=on  \
-device vfio-pci,host=01:00.1,bus=root.1,addr=0.1 \
-boot menu=on

Dmesg output when starting VM:

[Jun24 12:54] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[  +0.000132] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[  +4.085762] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000040] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000008] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000010] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +2.106266] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

In the Qemu window not much appear, some sort of qemu menu shows up if I activate the window and press a key.
On the external monitor nothing happens at all (connected through DVI -> VGA, but also tried via HDMI with same "non-result").
If I active KVM integrated gpu and boot windows the GTX680 card shows up in device manager with error 12, where it says the device needs more ressources.

I'm not sure where to tweak to get the ball rolling, any ideas?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm stuck on black screen from the passed through VGA card. So here's what I got, and where I'm stuck:

Spec:

• ASRock z97 Extreme 6

• Intel i7 4970 /w HD4600 (main gpu)

• Gainwind Phantom Nvidia GTX680 (secondary gpu)

I followed the 1st post and installed the mainline kernel from the package linked there:

3.15.1-1-mainline

Packages come from pacman's repos, so standard qemu, seabios and the likes.

My Nvidia card:

% lspci |grep -i nvidia                                                    :(
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 680] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)

% lspci -n |grep 01:00
01:00.0 0300: 10de:1180 (rev a1)
01:00.1 0403: 10de:0e0a (rev a1)

Current boot command:

BOOT_IMAGE=/rootvol/boot/vmlinuz-linux-mainline root=UUID=341a7e57-91cb-480c-be5f-2eb405ffd658 rw rootflags=subvol=rootvol intel_iommu=on pci-stub.ids=10de:1180,10de:0e0a quiet

Modprobes:

% cat /etc/modprobe.d/blacklist.conf /etc/modprobe.d/kvm.conf /etc/modprobe.d/vfio_iommu_type.conf
blacklist nouveau
options kvm ignore_msrs=1
options vfio_iommu_type1 allow_unsafe_interrupts=1

Configs for scripts from 1st post (I run vfio-bind.sh manually before I start VMs):

% cat /etc/vfio-pci.cfg 
DEVICES="0000:01:00.0 000:01:00.1"

% cat vfio-bind.sh 
#!/bin/sh
vfio-bind 0000:01:00.0 0000:01:00.1

Qemu VM:

% cat StartWindowsVm1.sh 
#!/bin/sh
qemu-system-x86_64 -enable-kvm -M q35 -m 10240 -cpu host \
-smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/qemu/bios.bin \
-vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=0.0,multifunction=on,x-vga=on  \
-device vfio-pci,host=01:00.1,bus=root.1,addr=0.1 \
-boot menu=on

Dmesg output when starting VM:

[Jun24 12:54] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[  +0.000132] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[  +4.085762] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000040] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000008] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000010] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +2.106266] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

In the Qemu window not much appear, some sort of qemu menu shows up if I activate the window and press a key.
On the external monitor nothing happens at all (connected through DVI -> VGA, but also tried via HDMI with same "non-result").
If I active KVM integrated gpu and boot windows the GTX680 card shows up in device manager with error 12, where it says the device needs more ressources.

I'm not sure where to tweak to get the ball rolling, any ideas?

You need to boot with i915.enable_hd_vgaarb=1 as a kernel parameter

penetal

Member

Registered: 2014-06-15

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm stuck on black screen from the passed through VGA card. So here's what I got, and where I'm stuck:

Spec:

• ASRock z97 Extreme 6

• Intel i7 4970 /w HD4600 (main gpu)

...

Dmesg output when starting VM:

[Jun24 12:54] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[  +0.000132] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[  +4.085762] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000040] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000008] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +0.000010] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[  +2.106266] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

In the Qemu window not much appear, some sort of qemu menu shows up if I activate the window and press a key.
On the external monitor nothing happens at all (connected through DVI -> VGA, but also tried via HDMI with same "non-result").
If I active KVM integrated gpu and boot windows the GTX680 card shows up in device manager with error 12, where it says the device needs more ressources.

I'm not sure where to tweak to get the ball rolling, any ideas?

You need to boot with i915.enable_hd_vgaarb=1 as a kernel parameter

Thanks, that got me right up and running.
I assume you/someone has already said this some other place in the thread but honnestly it's getting a bit too big to find everything.
Might be usefull in the OP's 1st post if its necessary for every Intel install.

penetal

Member

Registered: 2014-06-15

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Seems I'm stuck again.
Now at least I do get output from the card to the screen, so its halfways working.
The problem is windows won't "acitvely" use the card,
it has the yellow triangle of fuckery in device manager and report error code 43.

Searching seems to give little insight, has anyone at all been able to solve this problem?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Seems I'm stuck again.
Now at least I do get output from the card to the screen, so its halfways working.
The problem is windows won't "acitvely" use the card,
it has the yellow triangle of fuckery in device manager and report error code 43.

Searching seems to give little insight, has anyone at all been able to solve this problem?

I believe there are some problems with the lastest nvidia drivers, there's a discussion about this a few pages back

Last edited by nbhs (2014-06-24 15:50:48)

MCMjolnir

Member

Registered: 2014-06-19

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you got error 43, check if disabling and re-enabling nvidia card helps.

I tried that (I used this guide to automate it), and while it does get rid of the error code and the drivers show up correctly in device manager, for all other intents and purposes the card is still unusable. It doesn't detect the monitor I have plugged into it and  I have to reinstall the drivers on every VM boot through GeForce Experience before I can do anything within it (even though Device Manager says the drivers are installed already), and after that it produces some weird results, with some fields    ⃠ ed out.

Screenshot of GeForce Experience + Device Manager

I'm using Xeons (so no iGPU) with the host running headless with a GTX 770 assigned to a VM. I'm not sure if that means I need to use any of the patches.

Last edited by MCMjolnir (2014-06-24 19:58:57)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you got error 43, check if disabling and re-enabling nvidia card helps.

I tried that (I used this guide to automate it), and while it does get rid of the error code and the drivers show up correctly in device manager, for all other intents and purposes the card is still unusable. It doesn't detect the monitor I have plugged into it and  I have to reinstall the drivers on every VM boot through GeForce Experience before I can do anything within it (even though Device Manager says the drivers are installed already), and after that it produces some weird results, with some fields    ⃠ ed out.
I'm using Xeons (so no iGPU) with the host running headless with a GTX 770 assigned to a VM. I'm not sure if that means I need to use any of the patches.

Use qemu.git and add kvm=off to your -cpu options or downgrade the guest driver to 335.23.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

penetal

Member

Registered: 2014-06-15

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well as a small update. I got it working, but I have no idea why.

I was about to try to downgrade the nvidia driver, but before I started the VM I did two small changes that I can remember:

• I passed through my keyboard aswell so I had both keyboard and mouse in the VM while its running.

• And I switched input on my screen form DisplayPort (main intel gpu) to HDMI (nvidia) instead of just using PBP (Picture Beside Picture: both inputs at the same time with alot of wasted screen space going to black bars all done by the monitor), this shouldnt make any difference but I cant beleive that adding keyboard makes any difference either.

So I dont know why the error in Device Manager just disapeared, but I'm not sad it did.

Anyway I downgraded to an older driver and installed the graphics driver and physix engine (no 3D or audio drivers from nvidia pack was installed), and after that I ran 3Dmark. Got a lower score than native windows, so next step is gonna be to make it all more user friendly + optimizing for performance.

IT WORKS! WOHO!

belliash

Member

Registered: 2014-04-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you got error 43, check if disabling and re-enabling nvidia card helps.

I tried that (I used this guide to automate it), and while it does get rid of the error code and the drivers show up correctly in device manager, for all other intents and purposes the card is still unusable. It doesn't detect the monitor I have plugged into it and  I have to reinstall the drivers on every VM boot through GeForce Experience before I can do anything within it (even though Device Manager says the drivers are installed already), and after that it produces some weird results, with some fields    ⃠ ed out.

Screenshot of GeForce Experience + Device Manager

I'm using Xeons (so no iGPU) with the host running headless with a GTX 770 assigned to a VM. I'm not sure if that means I need to use any of the patches.

Thats very interesting, as you have exactly the same problem, except I encounter this on notebook, while you got "normal PC".
Thats pretty interesting, why others managed to get GTX 770 working - at least from desktop family. If we find a solution here, it might be step forward in getting mobile cards fully working too.

I'm also looking forward to hearing that anyone tried to pass through mobile radeon, as AMD cards seems to be less problematic than manufactured by nvidia.
Another idea is to try XEN instead of KVM. Especially as XEN supports mediated GPU. This means that laptop owners could pass through both nvidia/amd card as well as intel igp (4th gen cpu only).
I'm not sure how this behaves, but if Windows and forceware detects Intel HD Card, nvidia software should be able to copy video buffer to igp, therefore we would get optimus working in VM.
Shame it needs IGP and nvidia doesnt seem to let its technology work with other GPUs. Shame, as we could get optimus working with emulated cirrus/vmware cards.