Arch Linux

Lauer

Member

Registered: 2014-09-11

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Use a bridge

I'm able to connect to the VNC server on the guest now!

Device manager does not list any display devices, and generic vga-compatible device is listed in the taskbar.

runwin-vfio is:

qemu-system-x86_64 -net nic -net bridge,br=br0 -enable-kvm -M q35 -m 1024 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none -nographic \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device piix4-ide,bus=pcie.0,id=piix4-ide \
-drive file=/dev/md0,id=disk,format=raw -device ide-hd,bus=piix4-ide.0,drive=disk

3000

Banned

Registered: 2013-01-18

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, is there a good how-to? I think the how-to on the first page in this thread is way too advanced and the how-to from the arch wiki is pure crap.

I have used XEN for two years passing through AMD, but now I have Nvidia and need to make KVM work and I have absolutely no clue how.

Can someone please tell me where I can find a step-by-step guide?

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, is there a good how-to? I think the how-to on the first page in this thread is way too advanced and the how-to from the arch wiki is pure crap.

I have used XEN for two years passing through AMD, but now I have Nvidia and need to make KVM work and I have absolutely no clue how.

Can someone please tell me where I can find a step-by-step guide?

places:
- in this topic (yes, reading most of it)
- web (have seen at least few situational tutorials)
- try & explore

if op post is too advanced for you - should you really bother with experimental stuff? i mean what will you do when some problems arise? go ask for more "ohhh, please do waste more of your time just to solve some trivial stuff for me" (note: not every problem arising is trivial, but judging by this thread alone, i'd say 90% of problems are trivial - already solved and discussed in this thread)

3000

Banned

Registered: 2013-01-18

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

thanks for nothing.

3000

Banned

Registered: 2013-01-18

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

oh and btw: you have of course no time to help, but time to write all that CRAP. Congratulations.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

oh and btw: you have of course no time to help, but time to write all that CRAP. Congratulations.

You're not improving your chances of getting help.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

3000

Banned

Registered: 2013-01-18

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Again an answer. And AGAIN thanks for nothing.

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Again an answer. And AGAIN thanks for nothing.

well, i am truly noone in this thread. but replying "thanks for no help" to aw is like... ridiculous. sure, he did no help as far as vga passthrough is concerned, you are absolutely right )

oh, btw. why do you think someone should spend their own precious time to recompile tons of info in this very thread and his own experience to enlighten some ignorant child coming with nothing but "i did this for quite some time, but now i just dont have any clue. someone go bring me step-by-step guide"?

not to mention me not being able to grasp at all how "how-to" and "experimental stuff" can coexist in someone's head unless that very head is completely broken in some way.

p.s. yup, one more stupid reply with absolutely no help. bad bad me
p.p.s. in case you are not completely hopeless, i'll just repeat myself: this very thread have really TONS of info. period.

3000

Banned

Registered: 2013-01-18

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

you don't want to <redacted> help? Guess what? DON'T <redacted> HELP. That simple.

moderator edit [ewaller] sanitized somewhat after a burst of frustration

Last edited by ewaller (2014-10-05 18:13:16)

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hey .

Alex , I narrowed my crashing problems to reboots only .

When I shutdown the VM cleanly then reexecute the shell file to fire it up , it works nicely , no crashes to either the host or other VMs .

It only happens when I reboot the VM .

Journalctl doesn't show any errors related to the issue .

Again , it started happening as soon as I *upgraded* to X99 platform .  : (

EDIT :

One more thing , when I reboot a VM with no VGA passthroughed to it , it reboots fine . So its related to VGA . as it is basically the same shell , the main difference between these VMs is the VGA .

EDIT 2 :

Kernel is stock ARCH , 3.16.3 .

Qemu-git , compiled two days ago .

Last edited by Denso (2014-10-05 18:31:37)

ewaller

Administrator

From: Pasadena, CA

Registered: 2009-07-13

Posts: 20,212

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

While 3000 takes a day off to cool down, I remind everyone to respect each other, please do not flame.  Read the link in my signature, and remember the Arch way.. 

Edit: Oh, I forgot, remember that life is a two way street

Last edited by ewaller (2014-10-05 18:18:47)

---

Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

zsph

Member

Registered: 2014-09-04

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I managed to narrow down the problem. The host hungs on suspend/resume/reboot/shutdown only if I the video card isn't released properly and as a result I cannot unbind it. Sometimes it gets released and everything works great but I cannot predict or spot the pattern why and when. I get this in dmesg when I fail to unbind the card:

[39245.912644] BUG: unable to handle kernel NULL pointer dereference at 0000000000000411
[39245.912708] IP: [<0000000000000411>] 0x411
[39245.912757] PGD 7c511d067 PUD 80d112067 PMD 0 
[39245.912801] Oops: 0010 [#1] SMP 
[39245.912831] Modules linked in: vfio_pci vfio_iommu_type1 vfio nfnetlink_queue nfnetlink_log nfnetlink bluetooth pci_stub vboxpci(O) vboxnetadp(O) vboxnetflt(O) cfg80211 vboxdrv(O) cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_conservative binfmt_misc nls_utf8 nls_cp437 vfat fat snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device joydev mxm_wmi snd_hda_codec_hdmi eeepc_wmi asus_wmi sparse_keymap rfkill iTCO_wdt iTCO_vendor_support evdev snd_hda_codec_realtek x86_pkg_temp_thermal snd_hda_codec_generic intel_powerclamp intel_rapl coretemp kvm_intel i915 snd_hda_intel efi_pstore snd_hda_controller snd_hda_codec psmouse kvm snd_hwdep drm_kms_helper snd_pcm drm serio_raw pcspkr efivars snd_timer i2c_i801 snd i2c_algo_bit soundcore i2c_core mei_me wmi lpc_ich mei mfd_core shpchp battery video button processor fuse autofs4 ext4 crc16 mbcache jbd2 algif_skcipher af_alg dm_crypt dm_mod hid_logitech_dj sg usbhid hid sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper ahci cryptd libahci libata ehci_pci xhci_hcd scsi_mod ehci_hcd r8169 mii usbcore usb_common thermal fan thermal_sys
[39245.913917] CPU: 4 PID: 5417 Comm: tee Tainted: G        W  O  3.16.0-i915.arb.only #1
[39245.913975] Hardware name: System manufacturer System Product Name/P8Z77-V LK, BIOS 1402 03/21/2014
[39245.914039] task: ffff88082bac82d0 ti: ffff88076652c000 task.ti: ffff88076652c000
[39245.914092] RIP: 0010:[<0000000000000411>]  [<0000000000000411>] 0x411
[39245.914148] RSP: 0018:ffff88076652fde8  EFLAGS: 00010206
[39245.914187] RAX: 0000000000000008 RBX: ffff88080f730098 RCX: 0000000000000000
[39245.914239] RDX: 0000000000000411 RSI: ffff88080f730098 RDI: ffff88080f730098
[39245.914291] RBP: ffff88080f730098 R08: 0000000000000246 R09: ffffffff8129c240
[39245.914343] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88080f730148
[39245.914397] R13: 0000000000000000 R14: ffff88076652ff58 R15: ffff88080d1c2758
[39245.914449] FS:  00007fd3ea278700(0000) GS:ffff88082fb00000(0000) knlGS:0000000000000000
[39245.914506] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[39245.914549] CR2: 0000000000000411 CR3: 000000080ea97000 CR4: 00000000001407e0
[39245.914600] Stack:
[39245.914618]  ffffffff813951c7 0000000000000004 ffff88080f730098 0000000000000004
[39245.914683]  ffffffff813960f8 0000000000000004 ffff88080f730098 ffff88080f730148
[39245.914748]  0000000000000246 ffffffff81396244 ffff88080f730098 ffffffffa078d000
[39245.914816] Call Trace:
[39245.914843]  [<ffffffff813951c7>] ? __rpm_callback+0x27/0x70
[39245.914887]  [<ffffffff813960f8>] ? rpm_idle+0x198/0x290
[39245.914933]  [<ffffffff81396244>] ? __pm_runtime_idle+0x54/0x80
[39245.914980]  [<ffffffff812d2467>] ? pci_device_remove+0x67/0xb0
[39245.915029]  [<ffffffff8138d605>] ? __device_release_driver+0x75/0xf0
[39245.915079]  [<ffffffff8138d699>] ? device_release_driver+0x19/0x30
[39245.915131]  [<ffffffff8138ca2d>] ? unbind_store+0xed/0x150
[39245.915176]  [<ffffffff812081bb>] ? kernfs_fop_write+0xdb/0x160
[39245.915225]  [<ffffffff8119903d>] ? vfs_write+0xad/0x1e0
[39245.915269]  [<ffffffff81198f3c>] ? vfs_read+0x11c/0x170
[39245.915313]  [<ffffffff81199b4d>] ? SyS_write+0x3d/0xa0
[39245.915355]  [<ffffffff814f596d>] ? system_call_fast_compare_end+0x10/0x15
[39245.915406] Code:  Bad RIP value.
[39245.915439] RIP  [<0000000000000411>] 0x411
[39245.915481]  RSP <ffff88076652fde8>
[39245.915507] CR2: 0000000000000411
[39245.923907] ---[ end trace 42dbcaccec8b4e5d ]---

This is my unbind script

#!/bin/bash

for dev in "$@"; do
    vendor=$(cat /sys/bus/pci/devices/${dev}/vendor)
    device=$(cat /sys/bus/pci/devices/${dev}/device)
    if [ -e /sys/bus/pci/devices/${dev}/driver ]; then
        echo "${dev}" | sudo tee /sys/bus/pci/devices/${dev}/driver/unbind > /dev/null
        while [ -e /sys/bus/pci/devices/${dev}/driver ]; do
                sleep 1
        done
        echo "${dev}" | sudo tee /sys/bus/pci/drivers_probe > /dev/null
    fi
done

I tried sp_off and disable_hugepages=1 as suggested here but with no luck.

Last edited by zsph (2014-10-05 20:05:54)

---

Asus P8Z77-V LK, Core i7 3770, Asus NVIDIA GTX 770 DirectCU II OC, Debian x64, EFI, Kernel 3.16 + i915 arbiter patch.
3DMark11: bare metal P10231 vs vfio P10039: 98% of performance!

kolobok

Member

Registered: 2014-10-06

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi!
I have AMD RS880M chipset with integrated and discrete radeons, it seems there is only GART IOMMU. (there is no option in bios)

[    0.000000] AGP: Please enable the IOMMU option in the BIOS setup
[    1.428469] PCI-DMA: using GART IOMMU.
[    1.428471] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture

[    1.428469] PCI-DMA: using GART IOMMU.
[    2.080672] [drm] GART: num cpu pages 131072, num gpu pages 131072
[    2.098628] [drm] PCIE GART of 512M enabled (table at 0x00000000C0040000).

When I am trying to bind vfio it fails.

[   57.776135] VFIO - User Level meta-driver version: 0.3
[   57.786888] vfio-pci: probe of 0000:02:00.0 failed with error -22

I have disabled radeon driver on second gpu by pci-stub

[    2.044704] pci-stub: add 1002:68E0 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    2.044719] pci-stub 0000:02:00.0: claimed by stub

Can you help me with this? What is 22 error code means?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi!
I have AMD RS880M chipset with integrated and discrete radeons, it seems there is only GART IOMMU. (there is no option in bios)

A GART IOMMU is not sufficient for doing device assignment.  http://vfio.blogspot.com/2014/08/iommu- … d-out.html

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

roadrunna

Member

Registered: 2014-10-07

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just like to thank you for this guide. I got it working on gentoo. nvidia  550 TI with intel integrated graphics. using virtio drivers from redhat. So far the only issue is when I close down the vm ,but this maybe resolved with dev eject I have yet to try. Is there a way to convert the qemu arguments to xml to be able to manage the vm using libvirt + virt manager?

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just like to thank you for this guide. I got it working on gentoo. nvidia  550 TI with intel integrated graphics. using virtio drivers from redhat. So far the only issue is when I close down the vm ,but this maybe resolved with dev eject I have yet to try. Is there a way to convert the qemu arguments to xml to be able to manage the vm using libvirt + virt manager?

yes, there is a way - you will have to lurk the thread few dozens of pages back though

the main issue you may probably run into is audio. i resolved it by passing through dedicated pci sound card to vm while physically connecting its' front-out to host's integrated audio line-in (this way host outputs both "native" sound and sound from vm). yes, it required additional hardware (i had a spare one), but at the same time in-game fps drastically improved (it may be side-effect of libvirt's inner stuff though, like more accurate/advanced cpu pinning; but i still think it was mainly the result of escaping from emulated audio - no solid proof though)

p.s. if anyone has a way of preserving emulated audio while using libvirt/virt-manager (e.g. QEMU_AUDIO_DRV=pa) - detailed report would be highly appreciated. in my case it was always QEMU_AUDIO_DRV=none - did not bother digging too much into this though

Last edited by sinny (2014-10-07 04:23:51)

kaeptnb

Member

Registered: 2013-09-04

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

p.s. if anyone has a way of preserving emulated audio while using libvirt/virt-manager (e.g. QEMU_AUDIO_DRV=pa) - detailed report would be highly appreciated. in my case it was always QEMU_AUDIO_DRV=none - did not bother digging too much into this though

Hi,

i successfully use libvirt and a emulated Soundcard via pulseaudio with this:

    <qemu:env name='QEMU_AUDIO_DAC_FIXED_FREQ' value='48000'/>
    <qemu:env name='QEMU_AUDIO_ADC_FIXED_FREQ' value='48000'/>
    <qemu:env name='QEMU_PA_SAMPLES' value='1024'/>
    <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
    <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>

to save yourself the hassle of localhostsecurity regarding pulseaudio, add "load-module module-native-protocol-unix auth-anonymous=1" to $HOME/.pulse/default.pa, this way you dont need to copy around the cookiefile of pulseaudio.

you might need to play around with the "default-fragments" and "default-fragment-size-msec" in $HOME/.pulse/daemon.conf - i just left them as default and its working - but this depends on buffersize within your soundcards and you might need lower the defaults to get correct a/v sync.

QEMU_PA_SERVER can also be a networkserver - "localhost" works also, but on the other end (pulseaudio network server) you need to disable authentication (at least i had to).

It works - its in sync. The main requirements i had. Downside is relative high cpu usage of the pulseaudio process, as it seems qemu never "closes"(suspends?) the device, so pulseaudio is always on about 10%cpu (but only on one cpu) on my machine.

Greetings
K

Last edited by kaeptnb (2014-10-07 12:07:06)

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

p.s. if anyone has a way of preserving emulated audio while using libvirt/virt-manager (e.g. QEMU_AUDIO_DRV=pa) - detailed report would be highly appreciated. in my case it was always QEMU_AUDIO_DRV=none - did not bother digging too much into this though

Hi,

i successfully use libvirt and a emulated Soundcard via pulseaudio with this:

    <qemu:env name='QEMU_AUDIO_DAC_FIXED_FREQ' value='48000'/>
    <qemu:env name='QEMU_AUDIO_ADC_FIXED_FREQ' value='48000'/>
    <qemu:env name='QEMU_PA_SAMPLES' value='1024'/>
    <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
    <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>

to save yourself the hassle of localhostsecurity regarding pulseaudio, add "load-module module-native-protocol-unix auth-anonymous=1" to $HOME/.pulse/default.pa, this way you dont need to copy around the cookiefile of pulseaudio.

you might need to play around with the "default-fragments" and "default-fragment-size-msec" in $HOME/.pulse/daemon.conf - i just left them as default and its working - but this depends on buffersize within your soundcards and you might need lower the defaults to get correct a/v sync.

QEMU_PA_SERVER can also be a networkserver - "localhost" works also, but on the other end (pulseaudio network server) you need to disable authentication (at least i had to).

It works - its in sync. The main requirements i had. Downside is relative high cpu usage of the pulseaudio process, as it seems qemu never "closes"(suspends?) the device, so pulseaudio is always on about 10%cpu (but only on one cpu) on my machine.

Greetings
K

whoa, thank you very much
this kinda makes my dual vm configuration more or less complete

p.s. would be great to have links to posts related to virt-manager setup (the one on page 94 and this one) at opening post

dwightjl

Member

Registered: 2013-10-23

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, is there a good how-to? I think the how-to on the first page in this thread is way too advanced

Here are some of the guides that I used:
http://www.pugetsystems.com/labs/articl … 4-KVM-585/
http://www.firewing1.com/howtos/fedora- … hrough-kvm
http://www.linux-kvm.org/page/VGA_device_assignment

This technology and process are very new, so guides are usually system-specific and are often incomplete. I had to read several guides and many pages on this forum to eventually get something that works on my system.

Speaking of which, I can confirm that the following configuration works:

Motherboard: ASRock Z77 Extreme 4-M
CPU: Intel i5 3570
Memory: 16GB DDR3 1600
Host GPU: Integrated GPU from the i5 3570
Guest GPU: EVGA 550Ti (GPU BIOS updated to the latest version)
Host OS: Ubuntu 14.04 Server updated to the 14.10 Utopic sources.
Guest OS: Windows 10 Preview
Kernel: 3.16.3 (No patches)
QEMU: v2.1 (No patches)
SeaBIOS: v1.7.5
Blacklisted drivers: nouveau, i915, intel_agp (Even though I'm using the intel integrated GPU for the host, blacklisting the drivers seems to fix GPU arbitration issues)

I tried using a spare ATI card as the host GPU but ran into the problem where both cards are included the the iommu_group and I decided to give up on that path for now. The integrated GPU doesn't have the same issue. I can use the integrated GPU without the kernel patches as long as I blacklist the intel drivers.

The reason I went with Ubuntu Server is because I plan to use my home server to stream Windows games with Steam in-home streaming, so I don't care to have a working desktop on that machine anyway. That way I can keep all my normal bare-bones server functions and use the Windows VM to stream to my Linux Gaming PC or Steam Machines around the house. I still need to convince nvidia to let me install drivers on the guest system, and that requires a QEMU patch that I'm too lazy to apply right now.

The Arch forums can be a little harsh sometimes. Please be patient with the people here. They have put a lot of time into getting this stuff to work and rightfully expect you to put in effort as well. Eventually someone will show up and help you out. Game is hard.

Last edited by dwightjl (2014-10-07 20:27:43)

roadrunna

Member

Registered: 2014-10-07

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If anybody is using the 3.17 kernel using i915 ,and patching manually here are both acs override patch + i915 vga arbiter fixes into 1. http://www.mediafire.com/download/02ri1 … -kvm.patch.  I'm using gentoo and I patch manually. Thought Id make it easier for someone. .

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I managed to narrow down the problem. The host hungs on suspend/resume/reboot/shutdown only if I the video card isn't released properly and as a result I cannot unbind it. Sometimes it gets released and everything works great but I cannot predict or spot the pattern why and when. I get this in dmesg when I fail to unbind the card:

[39245.912644] BUG: unable to handle kernel NULL pointer dereference at 0000000000000411
...
[39245.914843]  [<ffffffff813951c7>] ? __rpm_callback+0x27/0x70
[39245.914887]  [<ffffffff813960f8>] ? rpm_idle+0x198/0x290
[39245.914933]  [<ffffffff81396244>] ? __pm_runtime_idle+0x54/0x80
[39245.914980]  [<ffffffff812d2467>] ? pci_device_remove+0x67/0xb0
[39245.915029]  [<ffffffff8138d605>] ? __device_release_driver+0x75/0xf0
[39245.915079]  [<ffffffff8138d699>] ? device_release_driver+0x19/0x30
[39245.915131]  [<ffffffff8138ca2d>] ? unbind_store+0xed/0x150
[39245.915176]  [<ffffffff812081bb>] ? kernfs_fop_write+0xdb/0x160
[39245.915225]  [<ffffffff8119903d>] ? vfs_write+0xad/0x1e0
[39245.915269]  [<ffffffff81198f3c>] ? vfs_read+0x11c/0x170
[39245.915313]  [<ffffffff81199b4d>] ? SyS_write+0x3d/0xa0
[39245.915355]  [<ffffffff814f596d>] ? system_call_fast_compare_end+0x10/0x15
...

I actually get something similar when I stop or start (I forgot which) a VM, if I have the AMD High-Definition Audio Controller bound to the VM.  Blacklisting snd-hda-intel on the host prevents the panic, but also prevents the integrated Realtek sound card from working on the host.

EDIT: By "similar", I mean a very similar, if not the same, backtrace.

Last edited by DanaGoyette (2014-10-08 04:40:10)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I managed to narrow down the problem. The host hungs on suspend/resume/reboot/shutdown only if I the video card isn't released properly and as a result I cannot unbind it. Sometimes it gets released and everything works great but I cannot predict or spot the pattern why and when. I get this in dmesg when I fail to unbind the card:

[39245.912644] BUG: unable to handle kernel NULL pointer dereference at 0000000000000411
...
[39245.914843]  [<ffffffff813951c7>] ? __rpm_callback+0x27/0x70
[39245.914887]  [<ffffffff813960f8>] ? rpm_idle+0x198/0x290
[39245.914933]  [<ffffffff81396244>] ? __pm_runtime_idle+0x54/0x80
[39245.914980]  [<ffffffff812d2467>] ? pci_device_remove+0x67/0xb0
[39245.915029]  [<ffffffff8138d605>] ? __device_release_driver+0x75/0xf0
[39245.915079]  [<ffffffff8138d699>] ? device_release_driver+0x19/0x30
[39245.915131]  [<ffffffff8138ca2d>] ? unbind_store+0xed/0x150
[39245.915176]  [<ffffffff812081bb>] ? kernfs_fop_write+0xdb/0x160
[39245.915225]  [<ffffffff8119903d>] ? vfs_write+0xad/0x1e0
[39245.915269]  [<ffffffff81198f3c>] ? vfs_read+0x11c/0x170
[39245.915313]  [<ffffffff81199b4d>] ? SyS_write+0x3d/0xa0
[39245.915355]  [<ffffffff814f596d>] ? system_call_fast_compare_end+0x10/0x15
...

I actually get something similar when I stop or start (I forgot which) a VM, if I have the AMD High-Definition Audio Controller bound to the VM.  Blacklisting snd-hda-intel on the host prevents the panic, but also prevents the integrated Realtek sound card from working on the host.

EDIT: By "similar", I mean a very similar, if not the same, backtrace.

Try using pci-stub.ids instead of blacklisting the driver

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone .

Can anyone who uses OVMF successfully post his entire qemu launch command ?

OVMF might be the answer to all my hangs issues on VMs' reboots , but it still doesn't detect any of my drives .

I don't use libvirt , just plain shell scripts .

Much appreciated .

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone .

Can anyone who uses OVMF successfully post his entire qemu launch command ?

OVMF might be the answer to all my hangs issues on VMs' reboots , but it still doesn't detect any of my drives .

I don't use libvirt , just plain shell scripts .

Much appreciated .

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name windows -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu host,hv_time,-kvm_pv_eoi,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -drive file=/usr/share/ovmf/x64/ovmf_code_x64.bin,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/windows_VARS.fd,if=pflash,format=raw,unit=1 -m 8192 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu -realtime mlock=off -smp 6,sockets=1,cores=3,threads=2 -uuid 09d81f35-51a4-4c7d-8db0-46597c7233ba -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/windows.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -drive file=/dev/storage/windows-ovmf,if=none,id=drive-virtio-disk0,format=raw,aio=native -device virtio-blk-pci,ioeventfd=on,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/dev/storage/steam,if=none,id=drive-virtio-disk1,format=raw,aio=native -device virtio-blk-pci,ioeventfd=on,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=21,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:c6:b5:45,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device vfio-pci,host=00:1b.0,id=hostdev0,bus=pci.0,addr=0x2 -device vfio-pci,host=01:00.0,id=hostdev1,bus=pci.0,multifunction=on,addr=0x7,rombar=0,romfile=/var/lib/libvirt/firmware/HD7870_EFI.rom -device vfio-pci,host=01:00.1,id=hostdev2,bus=pci.0,addr=0x7.0x1 -device usb-host,hostbus=2,hostaddr=17,id=hostdev3 -device usb-host,hostbus=2,hostaddr=3,id=hostdev4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -set device.virtio-disk0.scsi=off -set device.virtio-disk0.x-data-plane=on -set device.virtio-disk1.scsi=off -set device.virtio-disk1.x-data-plane=on -msg timestamp=on

It's how libvirt launches my vm, but should work anyway with few adjustments.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone .

Can anyone who uses OVMF successfully post his entire qemu launch command ?

OVMF might be the answer to all my hangs issues on VMs' reboots , but it still doesn't detect any of my drives .

I don't use libvirt , just plain shell scripts .

Much appreciated .

LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name windows -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu host,hv_time,-kvm_pv_eoi,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff -drive file=/usr/share/ovmf/x64/ovmf_code_x64.bin,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/windows_VARS.fd,if=pflash,format=raw,unit=1 -m 8192 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu -realtime mlock=off -smp 6,sockets=1,cores=3,threads=2 -uuid 09d81f35-51a4-4c7d-8db0-46597c7233ba -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/windows.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,clock=vm,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 -drive file=/dev/storage/windows-ovmf,if=none,id=drive-virtio-disk0,format=raw,aio=native -device virtio-blk-pci,ioeventfd=on,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/dev/storage/steam,if=none,id=drive-virtio-disk1,format=raw,aio=native -device virtio-blk-pci,ioeventfd=on,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=21,id=hostnet0,vhost=on,vhostfd=22 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:c6:b5:45,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -device vfio-pci,host=00:1b.0,id=hostdev0,bus=pci.0,addr=0x2 -device vfio-pci,host=01:00.0,id=hostdev1,bus=pci.0,multifunction=on,addr=0x7,rombar=0,romfile=/var/lib/libvirt/firmware/HD7870_EFI.rom -device vfio-pci,host=01:00.1,id=hostdev2,bus=pci.0,addr=0x7.0x1 -device usb-host,hostbus=2,hostaddr=17,id=hostdev3 -device usb-host,hostbus=2,hostaddr=3,id=hostdev4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -set device.virtio-disk0.scsi=off -set device.virtio-disk0.x-data-plane=on -set device.virtio-disk1.scsi=off -set device.virtio-disk1.x-data-plane=on -msg timestamp=on

It's how libvirt launches my vm, but should work anyway with few adjustments.

Thank you very much !

I'm going to adapt these settings to my existing WIndows VM , however I couldn't find the file "windows_VARS.fd" , and Google points me to your post , if it isn't any trouble can you tell me what is it for ? is it important ? if so , how can I create one ?

Sorry for this barge of questions , I want to learn .