Arch Linux

Tyrewt

Member

Registered: 2014-09-13

Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I suffer from the booting problem too...
When I use the VFIO GPU passthrough in a VM, shutdown the VM, but I cannot boot the VM unless I restart my host system.
Does anyone know this problem?

Yes. Same issue here with an R9 280. Windows 7 VM works perfectly fine on 1st run, 2nd run locks up the Host system to the point where it needs a hard reboot.

I believe it to be a problem with AMD cards PCI bus reset on guest reboot. Please let me know if you find a solution.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I suffer from the booting problem too...
When I use the VFIO GPU passthrough in a VM, shutdown the VM, but I cannot boot the VM unless I restart my host system.
Does anyone know this problem?

Yes. Same issue here with an R9 280. Windows 7 VM works perfectly fine on 1st run, 2nd run locks up the Host system to the point where it needs a hard reboot.

I believe it to be a problem with AMD cards PCI bus reset on guest reboot. Please let me know if you find a solution.

What cpus are you guys are runinng this on ? Intel or amd? I have A88X-PRO with amd cpu and it all works fine. The other poster has nvidia card so I dont think nvidia or ati are responsible.
I just shutdown and restarted both windows 10 and win 8.1 10 times no issues. I am using ovmf also. I have cople amd vidoe cards.

Last edited by tritron4 (2015-03-24 14:27:47)

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did anyone tried to install ubuntu 14.10 under kvm ? Any attempts to install it or try it shutdown guest

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

http://documentation-devel.engineering. … e-GPU.html

Your hideous redhat stuff isn't available for everyone. Here, i fixed it. Maybe your -devel version is more fresh..

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

http://documentation-devel.engineering. … e-GPU.html

Your hideous redhat stuff isn't available for everyone. Here, i fixed it. Maybe your -devel version is more fresh..

Same version, thanks.  I thought I was being smart by trying to view it in incognito mode... but I was still on the vpn.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Tyrewt

Member

Registered: 2014-09-13

Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I suffer from the booting problem too...
When I use the VFIO GPU passthrough in a VM, shutdown the VM, but I cannot boot the VM unless I restart my host system.
Does anyone know this problem?

Yes. Same issue here with an R9 280. Windows 7 VM works perfectly fine on 1st run, 2nd run locks up the Host system to the point where it needs a hard reboot.

I believe it to be a problem with AMD cards PCI bus reset on guest reboot. Please let me know if you find a solution.

What cpus are you guys are runinng this on ? Intel or amd? I have A88X-PRO with amd cpu and it all works fine. The other poster has nvidia card so I dont think nvidia or ati are responsible.
I just shutdown and restarted both windows 10 and win 8.1 10 times no issues. I am using ovmf also. I have cople amd vidoe cards.

Intel i7 4790. Can replicate the issue every time on 2nd boot of Win7 guest.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well, seems like there's no drivers for my ES1371 card for win7 x64. There are drivers in the linux kernel, but they aren't included in fedora21, so...

Did anybody try using NetJACK as a mean of transferring audio from guest to host? JACK is available for windows..

Last edited by Duelist (2015-03-24 15:12:17)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well, seems like there's no drivers for my ES1371 card for win7 x64. There's drivers in the linux kernel, but they aren't included in fedora21, so...

Did anybody try using NetJACK as a mean of transferring audio from guest to host? JACK is available for windows..

What about this driver http://www.driverscape.com/download/cre … -%28wdm%29

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well, seems like there's no drivers for my ES1371 card for win7 x64. There's drivers in the linux kernel, but they aren't included in fedora21, so...

Did anybody try using NetJACK as a mean of transferring audio from guest to host? JACK is available for windows..

What about this driver http://www.driverscape.com/download/cre … -%28wdm%29

Dude, now that's offensive. I can definitely use google. There is no drivers for x64 version of windows 7. There's for xp x64, for xp and win7 x86.
Please, stop.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well, seems like there's no drivers for my ES1371 card for win7 x64. There's drivers in the linux kernel, but they aren't included in fedora21, so...

Did anybody try using NetJACK as a mean of transferring audio from guest to host? JACK is available for windows..

What about this driver http://www.driverscape.com/download/cre … -%28wdm%29

Dude, now that's offensive. I can definitely use google. There is no drivers for x64 version of windows 7. There's for xp x64, for xp and win7 x86.
Please, stop.

That is strange the download page clearly states Supported OS: Windows XP, Windows Vista 32 & 64bit, Windows 7 32 & 64bit, Windows 8 32 & 64bit. I have no experience with this card so that why I am posting the link I dont mean any disrespect.

You can buy this sound card
http://www.ebay.com/itm/Creative-Sound- … 27fbf1bea2

Last edited by tritron4 (2015-03-24 15:45:49)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What about this driver http://www.driverscape.com/download/cre … -%28wdm%29

Dude, now that's offensive. I can definitely use google. There is no drivers for x64 version of windows 7. There's for xp x64, for xp and win7 x86.
Please, stop.

That is strange the download page clearly states Supported OS: Windows XP, Windows Vista 32 & 64bit, Windows 7 32 & 64bit, Windows 8 32 & 64bit. I have no experience with this card so that why I am posting the link I dont mean any disrespect.

You can buy this sound card
http://www.ebay.com/itm/Creative-Sound- … 27fbf1bea2

Well, seems like i have to take my words back.
I took that card out for proper examination, it turned out to be CB5880/CT4810 which is ES1371-based.
Reinserting the card back... changed it's PCI IDS, so it's detected properly now(but doesn't get along with vfio well):

03:05.0 Multimedia audio controller [0401]: Ensoniq 5880B [AudioPCI] [1274:5880] (rev 02)
        Subsystem: Ensoniq Creative Sound Blaster AudioPCI128 [1274:2000]
        Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort+ <MAbort+ >SERR- <PERR- INTx-
        Latency: 32 (3000ns min, 32000ns max)
        Interrupt: pin A routed to IRQ 20
        Region 0: I/O ports at c000 [size=64]
        Capabilities: <access denied>
        Kernel driver in use: pci-stub
        Kernel modules: snd_ens1371

After a numerous tries of some generic drivers, that driver that you've linked(and i've found it before many times), which was from "VMware VMaudio (VMAUDIO) (WDM)", installed and worked.

Weird ass legacy. Hell, i thought i was going insane. That card seems to be some fakey copy, since it lacks TDA 2x6w amplifier and it has a different PCB layout than others found on the interwebs, and a different audio codec also.

The old Gentoo wisdom remains true once again: if something doesn't work quite right, try recompiling it.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I just ordered sound card on ebay so I will let you know if that works with 8.1.
I wonder if we can improve sound from qemu by using asound.conf. I found out that spice outputs almost perfect sound. 

Dude, now that's offensive. I can definitely use google. There is no drivers for x64 version of windows 7. There's for xp x64, for xp and win7 x86.
Please, stop.

That is strange the download page clearly states Supported OS: Windows XP, Windows Vista 32 & 64bit, Windows 7 32 & 64bit, Windows 8 32 & 64bit. I have no experience with this card so that why I am posting the link I dont mean any disrespect.

You can buy this sound card
http://www.ebay.com/itm/Creative-Sound- … 27fbf1bea2

Well, seems like i have to take my words back.
I took that card out for proper examination, it turned out to be CB5880/CT4810 which is ES1371-based.
Reinserting the card back... changed it's PCI IDS, so it's detected properly now(but doesn't get along with vfio well):

03:05.0 Multimedia audio controller [0401]: Ensoniq 5880B [AudioPCI] [1274:5880] (rev 02)
        Subsystem: Ensoniq Creative Sound Blaster AudioPCI128 [1274:2000]
        Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=slow >TAbort- <TAbort+ <MAbort+ >SERR- <PERR- INTx-
        Latency: 32 (3000ns min, 32000ns max)
        Interrupt: pin A routed to IRQ 20
        Region 0: I/O ports at c000 [size=64]
        Capabilities: <access denied>
        Kernel driver in use: pci-stub
        Kernel modules: snd_ens1371

After a numerous tries of some generic drivers, that driver that you've linked(and i've found it before many times), which was from "VMware VMaudio (VMAUDIO) (WDM)", installed and worked.

Weird ass legacy. Hell, i thought i was going insane. That card seems to be some fakey copy, since it lacks TDA 2x6w amplifier and it has a different PCB layout than others found on the interwebs, and a different audio codec also.

The old Gentoo wisdom remains true once again: if something doesn't work quite right, try recompiling it.

The_Moves

Member

Registered: 2015-01-06

Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have no issues passing through my Sound Blaster Z to a Windows vm. Once I install the drivers, everything comes up properly. No issues what so ever.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

My 2 cents... If you're looking at purchasing a sound card, please default to a pci-express card.  Not necessarily a Z, but something more like it than that old ebay card above.  Conventional PCI often has issues with broken interrupt masking (the feature wasn't even added to the spec until PCI2.3) and conventional PCI devices do not have a concept of requester IDs, which is what the IOMMU uses to distinguish devices.  Not even the ACS override patch will separate devices behind a conventional PCI bridge, nor would it work correctly if it was hacked to do so.  You may need to unload drivers to get an exclusive interrupt for the device if masking is broken.  I've tested with an old Sound Blaster (Live!) and it works, but it wouldn't be my first choice to use regularly.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

The_Moves

Member

Registered: 2015-01-06

Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Oh wow, I didn't even look to see what their card was, only that its slot ID was strange. This is mine, PCI-Express, no issues:

http://www.soundblaster.com/products/so … ter-z.aspx

Last edited by The_Moves (2015-03-25 00:06:24)

Cubex

Member

Registered: 2014-04-06

Posts: 24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Crashes still occurs randomly, dmesg -w doesn't show anything relevant when crash occur, debug log neigher, i'm starting to suspect that might be a hardware lockup?
Seems to occur more when using CPU intensively but still happens in normal use, Im without clue ATM

Also discovered something interesting accidentally and I need someone to confirm this, when I compiled the lastest linux-mainline 4.0 rc5.1 I forgot to enable pci stub module, so after shutting down machine Dmesg showed "released" devices loading their respective module, after reboot I discovered that pci stub was not enabled and devices where working normaly before and after running KVM...

seems that we can grab PCI-E devices runtime without pci-stub? is this really safe? I passed secondary gpu + audio, USB controller and Net adapter, and 4 devices where restored aparently... dmesg of testing:

0000:02:00.0 is the GTX970
0000:04:00.0 is the USB controller
0000:05:00.0 is the NET adapter

[  115.651756] usbcore: registered new interface driver snd-usb-audio
[  115.687194] usb 1-14.1: current rate 33186 is different from the runtime rate 16000
[  115.699927] usb 1-14.1: current rate 33186 is different from the runtime rate 16000
[  115.724838] usb 1-14.1: current rate 198 is different from the runtime rate 16000
[  115.732585] usb 1-14.1: current rate 8 is different from the runtime rate 16000
[  115.734399] usb 1-14.1: 3:1: cannot get min/max values for control 2 (id 3)
[  300.233368] mce: [Hardware Error]: Machine check events logged
--QEMU started here--
[  464.991218] VFIO - User Level meta-driver version: 0.3
[  465.261701] xhci_hcd 0000:04:00.0: remove, state 4
[  465.261710] usb usb4: USB disconnect, device number 1
[  465.261891] xhci_hcd 0000:04:00.0: USB bus 4 deregistered
[  465.261897] xhci_hcd 0000:04:00.0: remove, state 1
[  465.261901] usb usb3: USB disconnect, device number 1
[  465.261903] usb 3-1: USB disconnect, device number 2
[  465.323138] xhci_hcd 0000:04:00.0: USB bus 3 deregistered
[  465.400432] usbcore: deregistering interface driver ov534
[  465.499432] tun: Universal TUN/TAP device driver, 1.6
[  465.499437] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[  465.500357] device tap0 entered promiscuous mode
[  465.500534] hostbr0: port 1(tap0) entered listening state
[  465.500543] hostbr0: port 1(tap0) entered listening state
[  465.501220] IPv6: ADDRCONF(NETDEV_UP): hostbr0: link is not ready
[  465.543875] kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable
[  467.240268] vfio_ecap_init: 0000:02:00.0 hiding ecap 0x1e@0x258
[  467.240281] vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x900
[  480.529995] hostbr0: port 1(tap0) entered learning state
[  487.466676] kvm: zapping shadow pages for mmio generation wraparound
[  495.569983] hostbr0: topology change detected, propagating
[  495.569991] hostbr0: port 1(tap0) entered forwarding state
[  495.570023] IPv6: ADDRCONF(NETDEV_CHANGE): hostbr0: link becomes ready
--Shutdown of QEMU--
[  598.207298] hostbr0: port 1(tap0) entered disabled state
[  598.207420] device tap0 left promiscuous mode
[  598.207440] hostbr0: port 1(tap0) entered disabled state
--Nouveau starts spilling--
[  601.701150] nouveau 0000:02:00.0: fb1: nouveaufb frame buffer device
[  601.701157] [drm] Initialized nouveau 1.2.1 20120801 for 0000:02:00.0 on minor 1
[  601.715319] snd_hda_intel 0000:02:00.1: Disabling MSI
[  601.715336] snd_hda_intel 0000:02:00.1: Handle VGA-switcheroo audio client
[  601.725718] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[  601.725725] r8169 0000:05:00.0: can't disable ASPM; OS doesn't have ASPM control
[  601.725731] r8169 0000:05:00.0: enabling device (0400 -> 0403)
[  601.728245] r8169 0000:05:00.0 enp5s0: renamed from eth0
[  601.758725] xhci_hcd 0000:04:00.0: enabling device (0400 -> 0402)
[  601.758804] xhci_hcd 0000:04:00.0: xHCI Host Controller
[  601.758811] xhci_hcd 0000:04:00.0: new USB bus registered, assigned bus number 3
--Usbs connected to controller starts popping--

hotfunction

Member

Registered: 2015-02-05

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes. Same issue here with an R9 280. Windows 7 VM works perfectly fine on 1st run, 2nd run locks up the Host system to the point where it needs a hard reboot.

I believe it to be a problem with AMD cards PCI bus reset on guest reboot. Please let me know if you find a solution.

What cpus are you guys are runinng this on ? Intel or amd? I have A88X-PRO with amd cpu and it all works fine. The other poster has nvidia card so I dont think nvidia or ati are responsible.
I just shutdown and restarted both windows 10 and win 8.1 10 times no issues. I am using ovmf also. I have cople amd vidoe cards.

Intel i7 4790. Can replicate the issue every time on 2nd boot of Win7 guest.

Intel i5-3470. I have tried GTX-560 and GTX-295 with Ubuntu 14.04 guest, but both fail to boot on 2nd start.

Kingd

Member

Registered: 2014-04-04

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I followed post 1 but I am stuck with the following error message, after testing if the system works:

qemu-system-x86_64: -device vfio-pci,host=01:00.0: vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
qemu-system-x86_64: -device vfio-pci,host=01:00.0: vfio: failed to get group 1
qemu-system-x86_64: -device vfio-pci,host=01:00.0: Device initialization failed.
qemu-system-x86_64: -device vfio-pci,host=01:00.0: Device 'vfio-pci' could not be initialized

My primary VGA is the build in Intel HD Graphics 4000 and I want to passthrough my PowerColor HD 7970 (UEFI).

Any suggestions? I am thankful for every hint

Last edited by Kingd (2015-03-25 08:00:07)

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Guys, you with 2nd boot problem, dump bios from your card with gpuz, and use it in qemu/libvirt with romfile= option.

@Kingd your iommu doesn't seem to be enabled? cat /proc/cmdline? intel_iommu=on?
or you need ACS patch, you need to bind all devices in same group aka gpu and gpu audio= same group....

Last edited by slis (2015-03-25 09:02:01)

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

NVIDIA's drivers refuse to initialize on 2nd boot for me . Anyone having this issue ?

The card reboots fine as long as I don't install NVIDIA's drivers . As soon as I install them , rebooting results in black screen when Windows is about to show the desktop .

This happens with GT610 only , GTX770 behaves perfectly during reboots .

Microsoft Basic Display Driver is AWESOME , except for the resoluion .

asd651651

Member

Registered: 2015-03-23

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

NVIDIA's drivers refuse to initialize on 2nd boot for me . Anyone having this issue ?

The card reboots fine as long as I don't install NVIDIA's drivers . As soon as I install them , rebooting results in black screen when Windows is about to show the desktop .

This happens with GT610 only , GTX770 behaves perfectly during reboots .

Microsoft Basic Display Driver is AWESOME , except for the resoluion .

I have that issue too. It works fine when guest not installing the nvidia driver on ubuntu 14.04, but after the guest installing the nvidia driver, i can't boot the guest again after shutting down it.

No error message output in terminal and dmesg.

Last edited by asd651651 (2015-03-25 10:51:47)

Tyrewt

Member

Registered: 2014-09-13

Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Anyone notice the time is always out of sync on their Windows 7 guests?  Though configured correctly and using NTP, the time is always off by 3 hours.

kraghot

Member

Registered: 2015-03-25

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm interested in configuring this type of setup, and I hope I'll be building a new computer soon. What parts would you recommend for this type of build?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Anyone notice the time is always out of sync on their Windows 7 guests?  Though configured correctly and using NTP, the time is always off by 3 hours.

This sounds like you're setting the rtc to utc (default) instead of local time

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm interested in configuring this type of setup, and I hope I'll be building a new computer soon. What parts would you recommend for this type of build?

https://bbs.archlinux.org/viewtopic.php … 6#p1495466

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users