Arch Linux

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I can't use the Intel Sata with vfio because of ACS, ACS patch does not work for Intel Sata [00:1f.2]...

If you haven't already done so, try passing through the other devices in that group, such as:
    00:1f.0 ISA bridge [0601]: Intel Corporation C226 Series Chipset Family Server Advanced SKU LPC Controller [8086:8c56] (rev 05)
    00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05)
    00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
    00:1f.6 Signal processing controller [1180]: Intel Corporation 8 Series Chipset Family Thermal Management Controller [8086:8c24] (rev 05)

Does it really seem like a good idea to pass these other devices through to a guest?  I hope not.

Alternately, these devices should be overridable via 'multifunction', or better, via device ID (in my case, it's id:8086:8c02).

Probably because you have the Cirrus VGA card as the primary. That's being treated as the primary monitor, so all the desktop content is showing up there and not on the Radeon.

I actually used Narrator to get the virtual machine's desktop to appear only on the ATI card, and not at all on the Cirrus.  The ATI desktop was alive and "functional", but totally black.

I've rebuilt Qemu from git, and now it seems to be defaulting to pci-assign instead of VFIO.  This works fine, aside from needing the "eject" workaround (which kicks the GPU into high-speed-fan mode).
In fact, now I'm thinking this may have something to do with that black desktop -- perhaps the black desktop is a result of NOT ejecting.
Time for more experimenting...

EDIT:
Okay, I tried switching back to vfio.  No dice:
genirq: Flags mismatch irq 16. 00000000 (vfio-intx(0000:01:00.0)) vs. 00000080 (ehci_hcd:usb1)

Adding the EHCI controller (#2, oddly enough, not #1 as 'usb1' would seem to imply), seems to perhaps fix this.

If I add my sound card to the mix:
genirq: Flags mismatch irq 17. 00000000 (vfio-intx(0000:07:04.0)) vs. 00000000 (vfio-intx(0000:01:00.1))

How the heck is '00000000' a mismatch with '00000000'?

Because neither of them include IRQF_SHARED and therefore require exclusive access to the interrupt line.  This means that neither devices supports PCI2.3 INTx disable.  You can generally fix this, but lose access to the conflicting device, by detaching the device from the host driver.

Last edited by aw (2014-01-05 15:29:05)

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I tried this with Linux 3.12.6-1-ARCH, qemu 1.7.0-1 and seabios 1.7.3.1-2 without success.

That's not very surprising if your kernel doesn't include the i915 fixes and neither kernel nor qemu include the coherency fixes included by the kvm-vfio device.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

I have some new information and I do not know who to share.

The probleme i found for pci passthougr of sata AsMedia 1062 :

[ 404.206866] dmar: DRHD: handling fault status reg 2
[ 404.206870] dmar: DMAR:[DMA Write] Request device [07:00.0] fault addr 2affdf000
[ 404.206870] DMAR:[fault reason 05] PTE Write access is not set
[ 404.206877] dmar: DRHD: handling fault status reg 2
[ 404.206879] dmar: DMAR:[DMA Read] Request device [07:00.0] fault addr 2affda000
[ 404.206879] DMAR:[fault reason 06] PTE Read access is not set

Is the same for Intel Z87 sata.

I can't use the Intel Sata with vfio because of ACS, ACS patch does not work for Intel Sata [00:1f.2] and when i use with pci_stub it's work only one time, when the VM reboot i have this error :

[ 1801.280862] dmar: DRHD: handling fault status reg 2
[ 1801.280866] dmar: DMAR:[DMA Read] Request device [00:1f.2] fault addr 2affdd000 
[ 1801.280866] DMAR:[fault reason 06] PTE Read access is not set

Where i can report this error in the hope of any boby correct this ?

Thanks.

You can run 'info mtree' from the qemu monitor to check whether those fault addresses are actually something that should be mapped to the device.  There's a reasonable chance that the device just does a stray DMA access, in which case it's broken.  If it only works once and the device does not support a standard reset then it needs a device-specific reset.  You might be able to figure out how to do that by looking through the datasheet for the device.  There's no guarantee that such a mechanism exists though.  The other question is whether it's really worthwhile to assign the SATA device.  I'd guess that you're not going to get a massive performance increase by assigning it versus using virtio-block with a raw partition, disk or lvm volume.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone got any working packages for pci-stub only? q35 performance is pretty poor for memory demanding games.

pci-stub is a kernel module that does nothing but sit on a device to prevent other drivers from claiming it.  q35 is a qemu chipset model.  I think you're asking if anyone has this working with the qemu 440fx chipset model, which can still use vfio-pci.  It's possible, try it an see.  q35 is more similar in architecture to today's PCs, so it's more like how drivers expect to see the hardware.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Now I can fine-tune my setup. Have you found a good solution to save power while the VM is shut down or not in use? I am running this on my 24/7 home server and having the nvidia card running without any power management permanently consumes about 20 Watts, which amounts to over 50€ electricity cost per year in my country.

We could think about making vfio-pci put devices into the D3 power state when not opened by a userspace driver like qemu, but I worry that it might cause more problems than it solves and there's also a good chance that it won't make a desktop graphics card consume any less power.  You can use setpci to experiment whether D3 actually saves you any power.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The motherboard has a built in Matrox G200e, I'm hoping to pass this through to a different VM as it's a horribly slow chip. It appears in the same iommu group as the southbridge x4 PCIe slot (the only slot where a graphics card won't be slowed down to x1 speed by the 3210 chipset on this motherboard). This is why I've applied the ACS patch.

FWIW, nvidia/amd require several quirks in qemu to make things work.  Nobody has done these for matrox and I don't have anything with matrox graphics.  Besides that, what's the point of attaching a matrox card to a guest?  It seems like the headache of assigning it is not worth the incremental improvement versus emulated graphics.  Assignment makes sense for high end GPUs, but I'm not sure I'm interested in maintaining code for assigning random 2d-only VGA devices.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Edit: And another question: I'd like to run a media center through the integrated graphics. Is that feasible? Can I pci-stub the integrated graphics from Arch and run it to a VM or are there any known problems?

There are numerous problem with assigning IGD.  Please don't try it unless you plan to contribute to development.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

cataphract

Member

Registered: 2013-08-06

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I tried this with Linux 3.12.6-1-ARCH, qemu 1.7.0-1 and seabios 1.7.3.1-2 without success.

That's not very surprising if your kernel doesn't include the i915 fixes and neither kernel nor qemu include the coherency fixes included by the kvm-vfio device.

OK, I tried with the linux package in the first post (I'd tried before but I couldn't boot it and gave up -- turns out I was loading the wrong initramfs), qemu-git and seabios-git and I was able to get the windows installer displayed on the 6950.

There were some issues with qemu though. First it wouldn't build (error in /usr/include/sys/xattr.h; it seems XATTR_CREATE was already defined when this was included) and then I'd get this error:

WARNING: failed to find q35-acpi-dsdt.aml

I solved this by copying the file into /usr/share/qemu.

Val532

Member

Registered: 2013-11-13

Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You can run 'info mtree' from the qemu monitor to check whether those fault addresses are actually something that should be mapped to the device.  There's a reasonable chance that the device just does a stray DMA access, in which case it's broken.  If it only works once and the device does not support a standard reset then it needs a device-specific reset.  You might be able to figure out how to do that by looking through the datasheet for the device.  There's no guarantee that such a mechanism exists though.  The other question is whether it's really worthwhile to assign the SATA device.  I'd guess that you're not going to get a massive performance increase by assigning it versus using virtio-block with a raw partition, disk or lvm volume.

I don't know how use "info mtree" and when ^^.

multifonction for acs overdrive does not work for intel sata, and yes is not a good idea to pass the entry bloc to the VM, and the real probleme, is it work with pci_stub (but only once) and not with vfio-pci (vfio-pci say the group is not blah blah).

And why use sata controler instead of virtio-scsi it's because i use a SSD ^^. And i don't know if the trim fuction work trough the virtio.

But it's funy to see the error is the same on Intel chipset, for Intel Sata or AsMedia, do anybody can test with a AMD motherboard ?

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If I add my sound card to the mix:
genirq: Flags mismatch irq 17. 00000000 (vfio-intx(0000:07:04.0)) vs. 00000000 (vfio-intx(0000:01:00.1))

How the heck is '00000000' a mismatch with '00000000'?

Because neither of them include IRQF_SHARED and therefore require exclusive access to the interrupt line.  This means that neither devices supports PCI2.3 INTx disable.  You can generally fix this, but lose access to the conflicting device, by detaching the device from the host driver.

Hmm, perhaps that message could use some tweaking.
Also, both devices function when both are attached to the host, and I'm passing both to the SAME virtual machine!
CORRECTION: If snd-virtuoso and radeon are both loaded, then the system hard-locks when trying to start X. 
With just 'radeon' loaded, no lockup occurs.  Maybe that's precisely the problem -- the shared interrupt!  I wonder why the kernel doesn't warn about this...

I'm trying to pass through (to the same VM) the following, at the moment:

01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper PRO [Radeon HD 5750] [1002:68be]
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series] [1002:aa58]
-- if I remove this, the mismatch instead cites 01:00.0.

07:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc CMI8788 [Oxygen HD Audio] [13f6:8788]
-- this lies under 06:00.0 PCI bridge [0604]: PLX Technology, Inc. PEX8112 x1 Lane PCI Express-to-PCI Bridge [10b5:8112] (rev aa)

09:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)
0c:00.0 Ethernet controller [0200]: Intel Corporation I210 Gigabit Network Connection [8086:1533] (rev 03)

I can see several ways to work around this, but none seems ideal:
* Use KVM built-in audio: only stereo; I'd like 5.1 surround.
* Pass the onboard Realtek to the VM, and keep the Xonar sound card for the host: last time I tried that, the Realtek produced horribly broken audio.
* Move to second CPU slot: steals 8 lanes from primary GPU, despite only actually needing 1 lane.  Also leaves only 4 lanes for a possible second GPU.
* Try another x1 slot: the x1 slot below second CPU slot conflicts with the Serial-over-LAN port, which I am definitely using; the x1 slot below the first GPU slot, I'd need a ribbon riser to try, due to dual-slot video card.
* Get a different sound card and/or GPU:  I can't tell before purchase which ones will likely work, and the same issues about slots will apply.
* Use a USB sound card?  I tried with the Xonar sound card either unbound or physically removed, and now the Renesas USB controller reports the same "00000000" conflict.
* Try the X10SAE motherboard I also have: would lose the Thunderbolt port, but more importantly, it has this bug: https://bugzilla.kernel.org/show_bug.cgi?id=44881

Last edited by DanaGoyette (2014-01-05 22:39:57)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

* Try the X10SAE motherboard I also have: would lose the Thunderbolt port, but more importantly, it has this bug: https://bugzilla.kernel.org/show_bug.cgi?id=44881

This is due to the asmedia PCIe-to-PCI bridge, it fails to expose a PCIe capability in direct violation of the PCIe spec.  It's not a trivial problem to solve, which is why it's been around so long.  Personally, I just avoid anything with an asmedia bridge.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

* Try the X10SAE motherboard I also have: would lose the Thunderbolt port, but more importantly, it has this bug: https://bugzilla.kernel.org/show_bug.cgi?id=44881

This is due to the asmedia PCIe-to-PCI bridge, it fails to expose a PCIe capability in direct violation of the PCIe spec.  It's not a trivial problem to solve, which is why it's been around so long.  Personally, I just avoid anything with an asmedia bridge.

That makes it sound like a PCI-PCI bridge sitting directly on a PCIe bus -- so yeah, that's broken.

Incidentally, the markings on the chip closest to the PCI slots are:
IDT ©
89HMPEB38
3ZBEMG
ZB1242CH

CH09493MX TWP BU

Last edited by DanaGoyette (2014-01-06 00:11:57)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've gone back and re-tested a bunch of the slots.  With pci-assign, I can use the video card in the top x16 and the sound card in the middle x16, just fine.  With vfio, even that configuration fails due to interrupt flag conflicts.  I can add or remove the firewire controller and the ethernet controller, and I'll get varying collisions between the devices I noted above, and sometimes with the Intel chipset KT (serial-over-lan) or SMBUS controller. 
I'm going to try re-adding all those assignments with the sound card in the bottom slot, using pci-assign again.

EDIT: Okay, with sound card back in an x1 slot, and the Ethernet controller included in the VM, I get a kernel oops.  Without ethernet controller, I'm noticing that the "genirq" message mentions IRQs that aren't actually shared in lspci.  For example, genirq will mention the hdmi audio and the C-Media card as conflicting on IRQ 17 -- yet, lspci -nnvv mentions only the C-Media card as using IRQ 17.

EDIT #2:  With sound card in x1 slot, VM boots if I don't bother forwarding HDMI audio, even when including the ethernet controller.  Again, this is still with pci-assign, not vfio.

It looks like vfio is protecting me from certain conflicts -- but unfortunately, all that protection must be somewhat over-zealous.

Also, in case you're wondering why I'm passing the USB controller through, it's because the USB passthrough in Qemu wasn't working for me when I tried it.  Devices stayed attached to the host, not the guest.

Last edited by DanaGoyette (2014-01-06 00:59:48)

MichaelLong

Member

Registered: 2014-01-06

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone,
thx a lot for this great thread, it helped me a lot. So I'd like to report another success story on the following hardware and software:

Hardware
Mainboard: Intel DP67BG
CPU: Intel i7-2600
GPU0 (host): PowerColor Go! Green HD5450 512MB PCIEx1
GPU1 (guest): Gainward GeForce GTX 560 Ti 2048MB
RAM: 8 GB

Software
Kernel: 3.13-rc7
QEMU: qemu-git

What works:
Using the AMD PCIe x1 card makes it convenient to avoid any ACS patches at the moment. For my tests I'm using a clone of my native Windows 7 gaming installation so it is quite easy to make comparisons. The VM runs without any general problems so far. I ran several benchmarks like Cinebench R15, 3DMark (2013), Unigine Heaven 4.0, PassMark. Performance ranges from about 85% (Cinemax) to even 99% (Unigine Heaven) compared to the native installation.

What does not work:
I can run the QEMU instance only once between a power cycle of the host system. Rebooting the VM is not a problem but shutting it down and starting it later results in:

vfio-pci 0000:01:00.0: Invalid ROM contents

Supplying a VBIOS to QEMU makes the message go away but does not change the result. QEMU hangs. This might not be any news at all but I'm not aware of the current state.

What is a problem:
My only remaining issue right now is a quite specific one. Running the benchmarks or even running Crysis 2 Maximum Edition were possible without any noticeable problems but as soon as I start Battlefield 3 or Battlefield 4 the VMs starts behaving very very sluggish. Mouse cursor moves choppy, task manager is updating slowly and  the performance is completely off.
BF3 shows around 30-40 FPS were it should 90-120, BF4 is even worse: 7-12 FPS no matter what configuration (normally around 70-90 FPS).
Sometimes I get a BSOF BUGCODE_USB_DRIVER, never seen before.

I didn't find any benchmark to reproduce this behavior yet.

So any advice to that matter is much appreciated.

xani

Member

Registered: 2013-12-24

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@xani: Do you know exactly the root cause of the memory leak and how you fixed it in your kernel? Or are you unsure on the exact reason? It would be awesome if we get this fixed also in the main kernel sources :-) Or are you already in contact with kernel devs about this topic?

I didn't contact upstream, I thought about it but there's too much on my head right now. It would be great if you could report it.

Here's what I found:

EDIT:
Further investigation showed that this problem can lie within gcc itself. Here's link that can give you explanation. I will report on my findings asap.

In short, Linus wanted to use some better asm goto instructions. It was implemented in commit

0c44c2d0f459c

But it was causing problems and they discovered that problem lies in gcc. It compiled new code in wrong way.
The patches for gcc will be released with 4.9, I think. Anyway, Ingo Molnar repaired problem they had with a hack in commit

88f182dd77

This commit is not fixing mem leak issue. I read in article that is in my qoute, that problem folks had upstream was repaired by disabling ASM_GOTO, so I did the same and problem disappeared. Patch that I included in latest version is very simple. Mem leak patch.

I found out that ASM_GOTO is a cause of the problem by bisecting kernel.

xani

Member

Registered: 2013-12-24

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Edit: Nevermind. I can't re-produce the problem.

With that patched kernel to stop the memory leak I can confirm it works unless I use hugetlbfs. Using hugetlbfs causes the memory leak again.

I'm not using hugetlbfs. After my exams maybe I'll test it, so it will be mid February.

syllopsium

Member

Registered: 2013-12-18

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The motherboard has a built in Matrox G200e, I'm hoping to pass this through to a different VM as it's a horribly slow chip. It appears in the same iommu group as the southbridge x4 PCIe slot (the only slot where a graphics card won't be slowed down to x1 speed by the 3210 chipset on this motherboard). This is why I've applied the ACS patch.

FWIW, nvidia/amd require several quirks in qemu to make things work.  Nobody has done these for matrox and I don't have anything with matrox graphics.  Besides that, what's the point of attaching a matrox card to a guest?  It seems like the headache of assigning it is not worth the incremental improvement versus emulated graphics.  Assignment makes sense for high end GPUs, but I'm not sure I'm interested in maintaining code for assigning random 2d-only VGA devices.

Point. Although it's currently a moot one because a) I'm stupid and forgot to turn rombar back to one - it works when correctly configured, at least in VGA mode. b) QEMU now fails with an emulation failure on the creaky old OS I'm trying to run, but that's not a vfio problem.. Thanks for all the help anyway

mbirkis

Member

Registered: 2013-12-12

Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi! great stuff. This has inspired me to try this on my own. I had to buy a new cpu/motherboard as i didn't have the virtualization stuff needed in my old Intel 2600K.

My setup (once parts get delivered) will be:
AMD FX8350
Gigabyte GA-990XA-UD3, Socket-AM3+
8GB ram
Nvidia GTX 570 (host)
Nvidia GTX580 (passed through)

I will be using the kernel/qemu/seabios from OP.

Results will be posted, if anyone have any starter tips feel free to let me know.

ISSUES:
Using NVIDIA proprietary drivers on the host requires a patch: see http://www.mail-archive.com/qemu-devel@ … 74066.html, patch (posted by andy123):  http://bpaste.net/show/109185/

One question is that i have been testing qemu on my old hardware just to get a feel of it (without passthrough) and i noticed that my Nvidia driver works without patching on the Host even with kvm modules inserted. Is this because i am not passing anything through yet or is the patch obsolete?

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[…]

ISSUES:
Using NVIDIA proprietary drivers on the host requires a patch: see http://www.mail-archive.com/qemu-devel@ … 74066.html, patch (posted by andy123):  http://bpaste.net/show/109185/

One question is that i have been testing qemu on my old hardware just to get a feel of it (without passthrough) and i noticed that my Nvidia driver works without patching on the Host even with kvm modules inserted. Is this because i am not passing anything through yet or is the patch obsolete?

The patch is to fix some problems with passthough (something about the vga arbiter?) and I think I'm still using it.

---

i'm sorry for my poor english wirting skills…

noctavian

Member

Registered: 2013-07-11

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks everybody for this great thread, I managed to get VGA passthrough working.

Motherboard: AsRock H67M-GE
CPU: Intel i7-2600
Host GPU: Intel HD 2500 (this is a headless machine, X isn't installed)
Guest GPU: AMD HD6770

I used the kernel, qemu and seabios PKBUILDs from the first post on a fresh Arch install and everything seems to be working just fine.

I created a Windows 7 VM and a Xubuntu VM, installed the drivers (the Windows Catalyst installer crashes a lot) for the card and run Uningine without any problems in either VM. I can also shutdown/restart/switch between the two VMs at will, awesome!

beardedchimp

Member

Registered: 2014-01-09

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've been running KVM for a few months now all thanks to this thread so thanks for that.

My setup is:
archlinux host
Windows 8.1 Guest
Gigabyte 990FXA-UD3
Radeon X700 PRO for host. I didn't even know what card is was until I looked it up for this post, all I knew was that it was ati and old.
gtx 760 for guest.
amd 8350fx, 6 cores for guest, 2 for host
16GB ram, 8GB guest, 8GB host
I pass through a 200GB hard drive through /dev/sda
I also passthrough a few usb ports

So far it's been working well, I love having full access to arch while I mess about playing games but I have run into issues with what I can run on the guest.

For some reason only some games will start. Other games (such as CS:GO) start launcing from steam but crash with no error message.

Games that do work are:
Counter Strike Source
Borderlands 2
Just Cause 2
Bad company 2
F1 2012

Games that don't work:
CS:GO
Shogun 2
Tomb Raider (the new one)
Civ 5
A load of others but can't remember which

The games all crash in the same way, no error message and no error log file generated. I've tried launching them in various ways with no success.

Because I run the guest from /dev/sda, I can also boot windows directly instead of through KVM. I was really surpised that this work actually but it's fine. When windows is booted directly all the games run, no hint of crashing.

I've had this setup working since August and I was hoping by now updates to linux/kvm/windows would have sorted this issue but it's still there and I'm no closer to working out why.

Has anyone experianced anything similar or know what might be causing it?

Here's the script I've cobled together from this thread, please let me know if I've done anything stupid.

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64
USE_CPUSET=true

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
#NET="-net nic -net bridge,br=br0"
NET=""
SND=true
SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES=""
PCI_DEVICES="00:16.0 00:16.2"
GPU="04:00.0"
GPU_SND="04:00.1"
ROM="/var/lib/vm/GTX760.rom"
VBIOS=""
HDD="/dev/sda"
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1,romfile=$ROM"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d,bus=pcie.0"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,if=virtio,cache=none"
  fi
  if [ -n "$CDROM" ]; then
      isonum=1
      for d in $CDROM; do
        isonum=$[$isonum +1]
        STORAGE+=" -drive file=$d,id=isocd$isonum -device ide-cd,bus=ahci.$isonum,drive=isocd$isonum"
        #STORAGE+=" -drive file=$CDROM,id=isocd$isonum -device ide-cd,bus=ahci.1,drive=isocd$isonum"
    done
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  #SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0  -soundhw ac97"
  #SND="-soundhw ac97"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ $USE_CPUSET ]; then
    if [ ! -d  $CPUSET/system ]; then
        mkdir $CPUSET/system
    fi

    if [ ! -d  $CPUSET/qemu ]; then
        mkdir $CPUSET/qemu
    fi

    echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
    echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
    echo 0 > $CPUSET/qemu/cpuset.mems

    echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
    echo 1 > $CPUSET/system/cpuset.cpu_exclusive
    echo 0 > $CPUSET/system/cpuset.mems
      
    for t in `cat $CPUSET/tasks`; do 
      echo $t > $CPUSET/system/tasks 
    done > /dev/null 2>&1
fi

set -m

#netctl start bridge

QEMU_PA_SAMPLES=128 QEMU_AUDIO_DRV=pa $QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &
#$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $STORAGE $EXTRA_ARGS &

PID=$!

if [ $USE_CPUSET ]; then
    (
    while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
      if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
        i=$((CPU_CORE_NR-VCPUS))
        for t in `ls /proc/$PID/task/`; do
          if [ $i -ne $((CPU_CORE_NR)) ]; then
        if [ $t -ne $PID ];then
          echo $t > $CPUSET/qemu/tasks
          taskset -pc $i $t; let i++
        fi
          fi
        done
        break
      fi
    done
    ) &

    fg %1

    wait

    for t in `cat $CPUSET/system/tasks`; do
      echo $t > $CPUSET/tasks 
    done > /dev/null 2>&1

    rmdir $CPUSET/system
    rmdir $CPUSET/qemu
fi

echo "All Done!"

pascal257

Member

Registered: 2014-01-02

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Because I run the guest from /dev/sda, I can also boot windows directly instead of through KVM. I was really surpised that this work actually but it's fine. When windows is booted directly all the games run, no hint of crashing.

I've had this setup working since August and I was hoping by now updates to linux/kvm/windows would have sorted this issue but it's still there and I'm no closer to working out why.

I'd try a fresh virtualisation-only installation. I suspect some driver issues or something related to that. I had no issues so far playing games.

But I haven't tried Civ 5 yet, I'll check that tonight. Also I am running Windows 7 instead of 8.

Last edited by pascal257 (2014-01-10 13:16:37)

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[…] I haven't tried Civ 5 yet, I'll check that tonight. Also I am running Windows 7 instead of 8.

I've played countless hours (>100) of Civ 5 on my setup, most likely even in different windows versions. My VM started out with windows 7 and was upgraded to 8 and 8.1 from there, iirc.

---

i'm sorry for my poor english wirting skills…

beardedchimp

Member

Registered: 2014-01-09

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Because I run the guest from /dev/sda, I can also boot windows directly instead of through KVM. I was really surpised that this work actually but it's fine. When windows is booted directly all the games run, no hint of crashing.

I've had this setup working since August and I was hoping by now updates to linux/kvm/windows would have sorted this issue but it's still there and I'm no closer to working out why.

I'd try a fresh virtualisation-only installation. I suspect some driver issues or something related to that. I had no issues so far playing games.

But I haven't tried Civ 5 yet, I'll check that tonight. Also I am running Windows 7 instead of 8.

I actually ran windows 7 first, then switched to windows 8 to see if it would fix the problem and didn't.

I've also upgraded the video drivers etc. multiple times with no effect so I don't know what drivers could be causing it.

Ahh, I might actually be wrong about Civ 5 being broken as I just checked and this win8 vm doesn't even have it installed. I'm going to work out a more complete list of what doesn't work.

Since I've already installed the VM twice with the same results I'm not confident about installing it again but I'll try any suggestions.

Cheers

alexis_evo

Member

Registered: 2013-07-04

Posts: 33

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just ordered an FX-8350 with an ASUS M5A97 R2.0. Has anyone managed to get this working on this motherboard?

Also, has anyone figured out a way around the issue with catalyst initializing both GPUs if they are the exact same model? I have 2x HD 6870, both from XFX, exact same model, that I would like to use (one in windows guest, one on arch host). I don't mind buying a new nvidia GPU for the host, but it may be a couple months until I can do that.