Arch Linux

guiper

Member

Registered: 2014-09-12

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

During a rainy weekend, played with getting OSX (Maverick) up and running with VGA Passthrough under KVM. I own a MacBook so in my case it should be legal, in case any moderator doesn't not agree feel free to delete this post.

Machine
Arch with 3.17 kernel and patches linked a couple of posts above
MB: Asrock Z77 Extreme 4-M
CPU Some  Intel I5 with VT-d enabled
AMD HD6450 being passthrough
qemu-git installed

Mavericks.iso I made on my Macbook Pro. Will not help here.
bios-mac.bin and chameleon_svn2360_boot you can find here http://blog.ostanin.org/2014/02/11/play … -x-on-kvm/

Installed OSX without using any passthrough, just using VNC, then for booting normally:

sudo qemu-system-x86_64 -enable-kvm -m 4096 -cpu core2duo -machine q35 \
-smp 4,sockets=1,cores=4,threads=1 \
-vga none -boot menu=on \
-device isa-applesmc,osk="GOOGLEIT!" \
-kernel /home/guilherme/OSX/chameleon_svn2360_boot \
-smbios type=2 \
-bios /home/guilherme/OSX/bios-mac.bin \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=00:1d.0,bus=root.1,addr=00.1 \
-device vfio-pci,host=00:1b.0,bus=root.1,addr=00.2 \
-device vfio-pci,host=00:1a.0,bus=root.1,addr=00.3 \
-drive file=/dev/samsung128/macosx,id=MacHDD,format=raw -device ide-hd,bus=ide.2,drive=MacHDD \
-device ide-drive,bus=ide.0,drive=MacDVD \
-drive id=MacDVD,if=none,snapshot=on,file=./Mavericks.iso
-netdev user,id=hub0port0 \
-device e1000-82545em,netdev=hub0port0,id=eth0,mac=52:54:00:8e:3a:fc \
-nographic

USB devices don't work in Chameleon boot selector, using VNC I created a Org.chameleon.Boot.plist file in /Extra settings the boot device timeout for 5 seconds, so it boots automatically the MacHDD device (hd0,2 in my case).

Current Status:
USB is flaky upon boot, disconnecting and reconnecting the devices get them to work fine.
Sleep / Resume works fine!!!
Couldn't get Audio to work but didn't test it enough. Tried to play with loading KEXTs but no success.
Graphics on the AMD 6450 work perfectly out of the box without any issues and acceleration is fine
Performance seems perfectly fine and on par with my MacBook pros, but didn't test too much.

Finally part of the experience using OSX is due to the nice mouse / gestures experience. Unless you have a proper mighty mouse an Apple keyboard maybe it doesn't worth it. For me, since I can fix Audio for the moment I'm back on Windows 8.1

Last edited by guiper (2014-09-15 08:23:56)

mbroemme

Member

From: Cologne

Registered: 2014-02-05

Posts: 40

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@aw:
is there any way to track any kernel patches related to vga passthrough to know what (for people living far from kernel contribution realms)? some kind of newsgroup or any kind of filter on some website?
i mean, you mostly post about it on  your blog, but just in case you "miss" something

http://vfio.blogspot.de/

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks sinny, much appreciated

@aw:
is there any way to track any kernel patches related to vga passthrough to know what (for people living far from kernel contribution realms)? some kind of newsgroup or any kind of filter on some website?
i mean, you mostly post about it on  your blog, but just in case you "miss" something

The short answer is no, it's unfortunately an involved process to track things going upstream.  You can follow along on mailing lists or by watching project commits, but it's an involved process, which is why distributions offering commercial support hire so many people.  I'll try to continue to note relevant updates on the blog as I become aware of them.

It seems like we've reached a point where everything we expect to go upstream is there, now we're just waiting for releases of these versions.  The only outstanding patches are the i915 vga arbiter patch and the acs override patch.  Am I forgetting anything?  Neither of these are expected to go upstream.  OVMF negates the need for the i915 patch and the acs problem needs to be solved by creating quirks for specific devices.  I think for the application here it is generally required to separate processor-based root ports.  Unfortunately Intel has declined to provide a guarantee of isolation of those ports on client processors (i5/i7) but does provide native acs on xeon.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

gneville

Member

Registered: 2014-04-02

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've been able to boot my Windows8 Guest up using the native libvirt OVMF support and can ping it, just unable to check if the VGA passthrough with my AMD R9 290 is working as I'm doing it remotely.

I have also managed to boot it up using a q35 machine type with the new OVMF from SVN, Windows8 is pinging again can't check VGA passthrough but will do later.

My command line entry for qemu looks like so:

/usr/local/qemu-xen-4-4-git-290814/bin/qemu-system-x86_64 -name windows8 -S -machine pc-q35-2.2,accel=kvm,usb=off -cpu host,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,kvm=off -drive file=/usr/share/ovmf/x64/ovmf_x64.bin,if=pflash,format=raw,unit=0 -m 5168 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 7b222825-fc7d-4a66-a72c-4875063752d2 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/windows8.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device i82801b11-bridge,id=pci.1,bus=pcie.0,addr=0x1e -device pci-bridge,chassis_nr=2,id=pci.2,bus=pci.1,addr=0x1 -device ich9-usb-ehci1,id=usb,bus=pci.2,addr=0x1.0x7 -drive file=/dev/mapper/vg1-ovmfwinv1data,if=none,id=drive-virtio-disk1,format=raw -device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x7,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1 -drive file=/dev/mapper/vg1-winv1data,if=none,id=drive-virtio-disk2,format=raw,cache=none -device virtio-blk-pci,scsi=off,bus=pci.2,addr=0x3,drive=drive-virtio-disk2,id=virtio-disk2 -drive file=/home/xen/iso/windows/en-gb_windows_8_1_enterprise_x64_dvd_2971910.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/root/virtio-win-0.1-74.iso,if=none,id=drive-ide0-2-0,readonly=on,format=raw -device ide-cd,bus=ide.2,unit=0,drive=drive-ide0-2-0,id=ide0-2-0 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:12:34:56,bus=pci.2,addr=0x2,rombar=0 -device usb-host,hostbus=1,hostaddr=4,id=hostdev0 -device usb-host,hostbus=1,hostaddr=5,id=hostdev1 -device vfio-pci,host=01:00.0,id=hostdev2,bus=pci.2,multifunction=on,addr=0x8 -device vfio-pci,host=01:00.1,id=hostdev3,bus=pci.2,addr=0x9 -msg timestamp=on

My XML looks like so:

<domain type='kvm'>
  <name>windows8</name>
  <uuid>7b222825-fc7d-4a66-a72c-4875063752d2</uuid>
  <memory unit='KiB'>5291456</memory>
  <currentMemory unit='KiB'>5291456</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='q35'>hvm</type>
    <loader type='pflash'>/usr/share/ovmf/x64/ovmf_x64.bin</loader>
    <boot dev='hd'/>
  </os>
 <features>
    <acpi/>
    <apic/>
    <pae/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  <cpu mode='host-passthrough'/>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/local/qemu-xen-4-4-git-290814/bin/qemu-system-x86_64</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' />
      <source dev='/dev/mapper/vg1-ovmfwinv1data'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x07' function='0x0'/>
    </disk>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <source dev='/dev/mapper/vg1-winv1data'/>
      <target dev='vdc' bus='virtio'/>
    </disk>
    <disk type='file' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <source file='/home/xen/iso/windows/en-gb_windows_8_1_enterprise_x64_dvd_2971910.iso'/>
     <target dev='hdc' bus='ide'/>
     <readonly/>
     <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
     <driver name='qemu' type='raw'/>
     <source file='/root/virtio-win-0.1-74.iso'/>
     <target dev='hdd' bus='ide'/>
     <readonly/>
     <address type='drive' controller='0' bus='2' unit='0'/>
    </disk>
    <interface type='network'>
      <mac address='52:54:00:12:34:56'/>
      <source network='default'/>
      <target dev='tap4'/>
      <model type='virtio'/>
      <alias name='windows-nic-0'/>
      <rom bar='off'/>
    </interface>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x7'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'/>
    <controller type='pci' index='2' model='pci-bridge'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x046d'/>
        <product id='0xc52e'/>
      </source>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source>
        <vendor id='0x045e'/>
        <product id='0x0719'/>
      </source>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0' />
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x08' function='0x0' multifunction='on'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x09' function='0x0'/>
    </hostdev>
   <memballoon model='none'/>
  </devices>
</domain>

noobman

Member

Registered: 2013-12-13

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I saw this i915 vgaarb patch may be in the 3.17 RC3 kernel. Is it true?

No, the other VGA arbiter patch is in 3.17, the i915 patch is not currently scheduled for any upstream kernel and is a blocking issue for using an unmodified kernel for users with IGD.  Those reporting success with an unmodified kernel probably don't have IGD or are not following the directions and using their GPU as a secondary card for the VM.  Please see the articles at the URL below, especially the FAQ for more info.

I can report that without IGD (disabled in bios) and with 2 discrete video cards instead, i was able to passthrough with an unmodified kernel starting with kernel 3.13. On machines with X79 and Z97 chipsets. Quite easy, mostly doing it like this:
-better investigation of how mobo sets up iommu groups to find out which slot(s) allows for the wanted config passthrough (this is to find the slot configs which work without patch).
-using /etc/modules-load.d/... conf file to load the kernel modules ( kvm \ kvm_intel \ vfio \ vfio_pci \ vfio_iommu_type1)
-using /etc/modprobe.d/... conf file to set modules options, like options kvm ignore_msrs=1 and options vfio_iommu_type1 allow_unsafe_interrupts=1
I think the above folders and methos can be safely used to load modules and hence avoid the need to compile entire kernel just for switches on those modules. I am not aware of any disadvantage using this method of module loading, maybe somebody else knows some disadvantage? If anyone else wants to try, both conf files are to be edited with one module or one option per each text line, e.g. separated by new lines, not just by spaces. Maybe not all modules really need to be there, but having them wont hurt.

Currently using 3.17-rc last, but in spite of all the patches going on, with this config i have not noticed any improvements since 3.13, which (very subjectively here) seems to work best for me. Only on Z97 machine i guess due to its bios, the 3.17-rc1 vanished an dmesg err like "could not find upstream pcie", although things were working either way. My personal conclusion, for desktop use, i would rather buy a 20-30 eur cheap video card for host, and disable the intel graphics entirely, and find the 2 slot combination it works without patches. The laptop users might not be so lucky, but for desktop i find this approach to be ideal atm. Just to avoid dealing with intel graphics crap.

I cant shake the feeling that igd holding its teeth to vga has smth to do with mei along with its vpro backdoor. Either this smells bad or i do (or both). This even more underlined when a logical disable mechanism no longer works and more when the "maintainer is not willing to accept a driver mode switch option". Why on earth would not want to accept a driver mode switch, srsly what could be wrong with that, can anybody elaborate. I sincerely dont understand. But i think these back each other, meaning then perhaps the disabling mechanism no longer working is probably "working as intended". Anyway me personally, the last thing i want is stuff i am not allowed to disable. Even just as a matter of principle, but i suspect this might be more than just principles.

Ansa89

Member

Registered: 2014-08-30

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't really know what is OVMF, but reading something on the web I understood it has something to do with UEFI.
Consider that I'm not using UEFI on the host, nor no the guest.

Look down, hint, hint

Ok, I have compiled OVMF and now I have a "OVMF.fd" file. I guess I have to pass that file as "-bios", right?

Moreover I tested the GTX 650 rom with your program, here is the output:

Valid ROM signature found @0h, PCIR offset 190h
        PCIR: type 0, vendor: 10de, device: 0fc6, class: 030000
        PCIR: revision 0, vendor revision: 1
Error, ran off the end

PS: just wondering how much important OVMF test is...I mean: is it a needed dependency for vga passthrough?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't really know what is OVMF, but reading something on the web I understood it has something to do with UEFI.
Consider that I'm not using UEFI on the host, nor no the guest.

Look down, hint, hint

Ok, I have compiled OVMF and now I have a "OVMF.fd" file. I guess I have to pass that file as "-bios", right?

Moreover I tested the GTX 650 rom with your program, here is the output:

Valid ROM signature found @0h, PCIR offset 190h
        PCIR: type 0, vendor: 10de, device: 0fc6, class: 030000
        PCIR: revision 0, vendor revision: 1
Error, ran off the end

PS: just wondering how much important OVMF test is...I mean: is it a needed dependency for vga passthrough?

Oh dear, you've gone to all this trouble to compile OVMF but haven't read this post to determine why you'd want it and how to use it?  Have I missed something in describing the motivation and requirements there?  OVMF isn't a "test", it's an alternative to VGA.  Since you're not getting any output from the monitor, it seems like you have an issue with VGA routing.  Avoiding VGA by using OVMF is one method of dealing with this.

Regarding the parse error, how recently did you grab that copy of rom-parser?  I fixed a similar issue in the code on the 4th.  Otherwise please send me a copy of the ROM binary so I can fix the code for it.  Since you already have the edk2 tree, there's also a parser there you can use, details in the comments here.

For the -bios option; you shouldn't ever need a -bios option.  The default bios that comes with QEMU is fine for seabios use and for OVMF you'd add an option like:

-drive if=pflash,format=raw,readonly,file=/path/to/OVMF.fd

Your previous posts indicate that you're using Windows 7, so please pay particular attention to the guest operating support for UEFI.  You'd need to upgrade your guest to Windows 8 for OVMF.

If that's not any option, we'll need to go back to debugging your VGA routing issue and 'dmesg | grep -i vga' might shed some light on what's wrong.  Please also pastebin the entire dmesg log from the host.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I saw this i915 vgaarb patch may be in the 3.17 RC3 kernel. Is it true?

No, the other VGA arbiter patch is in 3.17, the i915 patch is not currently scheduled for any upstream kernel and is a blocking issue for using an unmodified kernel for users with IGD.  Those reporting success with an unmodified kernel probably don't have IGD or are not following the directions and using their GPU as a secondary card for the VM.  Please see the articles at the URL below, especially the FAQ for more info.

I can report that without IGD (disabled in bios) and with 2 discrete video cards instead, i was able to passthrough with an unmodified kernel starting with kernel 3.13. On machines with X79 and Z97 chipsets. Quite easy, mostly doing it like this:
-better investigation of how mobo sets up iommu groups to find out which slot(s) allows for the wanted config passthrough (this is to find the slot configs which work without patch).
-using /etc/modules-load.d/... conf file to load the kernel modules ( kvm \ kvm_intel \ vfio \ vfio_pci \ vfio_iommu_type1)
-using /etc/modprobe.d/... conf file to set modules options, like options kvm ignore_msrs=1 and options vfio_iommu_type1 allow_unsafe_interrupts=1

You should not need vfio_iommu_type1 allow_unsafe_interrupts=1 on either of these systems unless interrupt remapping is disabled in the BIOS.

I think the above folders and methos can be safely used to load modules and hence avoid the need to compile entire kernel just for switches on those modules. I am not aware of any disadvantage using this method of module loading, maybe somebody else knows some disadvantage? If anyone else wants to try, both conf files are to be edited with one module or one option per each text line, e.g. separated by new lines, not just by spaces. Maybe not all modules really need to be there, but having them wont hurt.

Currently using 3.17-rc last, but in spite of all the patches going on, with this config i have not noticed any improvements since 3.13, which (very subjectively here) seems to work best for me. Only on Z97 machine i guess due to its bios, the 3.17-rc1 vanished an dmesg err like "could not find upstream pcie", although things were working either way. My personal conclusion, for desktop use, i would rather buy a 20-30 eur cheap video card for host, and disable the intel graphics entirely, and find the 2 slot combination it works without patches. The laptop users might not be so lucky, but for desktop i find this approach to be ideal atm. Just to avoid dealing with intel graphics crap.

I cant shake the feeling that igd holding its teeth to vga has smth to do with mei along with its vpro backdoor. Either this smells bad or i do (or both). This even more underlined when a logical disable mechanism no longer works and more when the "maintainer is not willing to accept a driver mode switch option". Why on earth would not want to accept a driver mode switch, srsly what could be wrong with that, can anybody elaborate. I sincerely dont understand. But i think these back each other, meaning then perhaps the disabling mechanism no longer working is probably "working as intended". Anyway me personally, the last thing i want is stuff i am not allowed to disable. Even just as a matter of principle, but i suspect this might be more than just principles.

The desire to reduce options and make things work out of the box for all users is a good one by the i915 maintainer.  Even users in this thread who patch the kernel themselves and add the necessary options are often surprised and complain at the loss of DRI support on the host.  Unfortunately it doesn't seem like we have many options that both work for everyone and maintain compatibility with the current versions of Xorg.  This is why I continue to push people towards OVMF as an alternative (but please read the posts I'm pointing Ansa89 to before you leap without understanding the motivation and restrictions).  If your intended guest OS supports UEFI and so does your hardware, you can use OVMF, forget about VGA routing, and you also don't need to go buy another VGA card just for the host.  Everything is already upstream to make this work, even if there aren't yet released package versions, there will be sooner than we'll have an i915 VGA arbiter solution.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Same identical problem.

My hardware

Motherboard: B75Pro3
Cpu:I5-3470S(host  gpu)
Secondary gpu: radeon r9 270.

I've gota Proxmox(Debian)  with kernel 3.14 with ACS and I915 arbiter patch. Included also vfio and vfio-vga in kernel.

Iommu is enabled I can passthrough devices like dvb pci express cards to other vm(ubuntu) but not secondary gpu with windows...

Can install using standard adapter, install drivers and infinite atikmpag.sys 0x16 bsod.

Tried detaching gpu, disabling card + install driver and also not passing audio to no avail.

I've got signal on my secondary monitor and windows logo is on screen.

Then bsod.

Tried several catalyst version from 13.4 to 14.7 RC3...

GRUB:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pci-stub.ids=1002:6811,1002:aab0  i915.enable_hd_vgaarb=1”

Any idea?

Another question:
I'm running this machine headless accessing via web or ssh.
Windows VM would be a Steam in Home streaming server...

Is it possible to disable in bios intel cpu and passing ati card to vm(pci stubbing card) as primary passthrough?
So I would be free to not use the i915 patch and, reading other posts, maybe results are better than mixing IGPU and discrete card...
P.S. I'm also curious about What means "regular installations"?

Thanks in advance...

@ Arakatak,

try to go to device manager under windows, navigate to display adapter, click on it, now disable it ( Don't un install ), after displaying the adapter, windows should blink, now install the ATI drivers.

This problem comes from regular installations, not KVM/QEMU

It didn't help. Why do you think it's not qemu/kvm? What means "regular installations"? Is it a Windows 8.1 Setup? If so, i also experimented with Windows 7.

At first Win7 showed some light on possible problems. I installed Catalyst drivers and on next reboot received BSOD (code 0x00000116 on first boot and 0x0000050 on sequential reboots). I followed the advice in this post and removed 01:00.1 audio device from passthrough, then reinstalled the drivers. Although tray icon device detach still worked wrong (resulting in still picture from the last moment before VGA detach), graphics device have functioned well with ATi driver.

Unfortunately, i wasn't able to make a fully functional fresh install of Win7 VM without audio 01:00.1 device passthrough. The problems were the same as i mentioned earlier (black screen after driver setup and reboot, but no "No Signal" error).

For the reference i provide my start scripts:

cat qemu-win7-setup qemu-win7

#!/bin/bash

QEMU_AUDIO_DRV=alsa QEMU_ALSA_DAC_SIZE_IN_USEC=0 QEMU_ALSA_DAC_PERIOD_SIZE=256 QEMU_ALSA_DAC_BUFFER_SIZE=512 QEMU_ALSA_DAC_DEV="hw:1,0" QEMU_ALSA_ADC_DEV="hw:1,0" qemu-system-x86_64 -enable-kvm \
 -M q35 \
 -m 8G \
 -cpu host \
 -smp 6,sockets=1,cores=3,threads=2 \
 -bios /usr/share/qemu/bios.bin \
 -vga none \
 -device virtio-scsi-pci,id=scsi \
 -drive file=/dev/System/Windows7Guest,id=disk,format=raw,discard=on -device scsi-hd,drive=disk \
 -drive file="/mnt/data/win.iso",id=isocd -device scsi-cd,drive=isocd \
 -drive file=/mnt/data/virtio-win-0.1-81.iso,id=virtiocd -device ide-cd,bus=ide.1,drive=virtiocd \
 -boot once=d \
 -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,id=extvga \
 -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
 -usb -usbdevice host:046d:c328 \
 -usbdevice host:046d:c247

#!/bin/bash

QEMU_AUDIO_DRV=alsa QEMU_ALSA_DAC_SIZE_IN_USEC=0 QEMU_ALSA_DAC_PERIOD_SIZE=256 QEMU_ALSA_DAC_BUFFER_SIZE=512 QEMU_ALSA_DAC_DEV="hw:1,0" QEMU_ALSA_ADC_DEV="hw:1,0" qemu-system-x86_64 -enable-kvm \
 -M q35 \
 -m 8G \
 -cpu host \
 -smp 6,sockets=1,cores=3,threads=2 \
 -bios /usr/share/qemu/bios.bin \
 -vga none \
 -device virtio-scsi-pci,id=scsi \
 -drive file=/dev/System/Windows7Guest,id=disk,format=raw,discard=on -device scsi-hd,drive=disk \
 -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,id=extvga \
 -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
 -usb -usbdevice host:046d:c328 \
 -usbdevice host:046d:c247

I welcome discussions on these problems, as it may help solve them.

@EDIT:@

Q: I hate running sudo every time I want to boot a VM, is it not possible to assign the right groups to the Hardware access?

Every additional layer of API complicates troubleshooting. Why should i use libvirt when i didn't even have minimalistic, but functional VM with 100% VGA passthrough support? As for complexity, i need proper documentation on VGA passthrough with libvirt (and possibly OVMF VGA passthrough, as my VGA functions well on native UEFI). So far i found neither.

Last edited by evilsephiroth (2014-09-16 23:57:23)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

P.S. I'm also curious about What means "regular installations"?

I think it means trying to move a bare metal installation to a VM rather than doing a fresh install in the VM.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Speaking of R9 290, I haven't heard anything new about the inability to reset the device.  Has any progress been made on that?

If it would help, I can try out various kernel patches.

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Any hint to diagnose my problem?
My system seems compliant...

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

speaking of ovmf, i wonder if there is some kind of list of uefi-compliant gpus, or some kind of way to know if some distinct gpu would work or now

gigabyte's hd 6450 does NOT support uefi, palit gt 430 is pending for test (i doubt it will support it though)

so i'll probably have to go "buy new cheap(-est) one(-s)" route - would not be that great to blind-pick anything

Last edited by sinny (2014-09-17 07:48:27)

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

speaking of ovmf, i wonder if there is some kind of uefi-compliant gpus, or some kind of way to know if some distinct gpu would work or now

gigabyte's hd 6450 does NOT support uefi, palit gt 430 is pending for test (i doubt it will support it though)

so i'll probably have to go "buy new cheap(-est) one(-s)" route - would not be that great to blind-pick anything

Just to be clear...

OVMF is a replacement for VGA, right?

can it be a problem solver in case of ATI bsod?

My guess would be no because they're related mainly to passtrough itself and not to vga routing...

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

...

Just to be clear...

OVMF is a replacement for VGA, right?

can it be a problem solver in case of ATI bsod?

My guess would be no because they're related mainly to passtrough itself and not to vga routing...

that was NOT written in response to you, in case you care

and on your problem:
based on what you write, you did not even try to read all the stuff that was written in OP post and on THIS page (not to mention doing some "research" in the thread or web on your own)
r9 270/r9 270x was reported as working at least few times in the sheet, so i'd bet 95% that you are doing something wrong (probably because of lack of understanding of what you are doing at all)

please do not waste people's time on something trivial - first waste your own time instead: read, explore, play around with parameters or start from scratch, repeat until it works or you spent like TONS of time with no result (and read TONS of info)

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

...

Just to be clear...

OVMF is a replacement for VGA, right?

can it be a problem solver in case of ATI bsod?

My guess would be no because they're related mainly to passtrough itself and not to vga routing...

that was NOT written in response to you, in case you care

and on your problem:
based on what you write, you did not even try to read all the stuff that was written in OP post and on THIS page (not to mention doing some "research" in the thread or web on your own)
r9 270/r9 270x was reported as working at least few times in the sheet, so i'd bet 95% that you are doing something wrong (probably because of lack of understanding of what you are doing at all)

please do not waste people's time on something trivial - first waste your own time instead: read, explore, play around with parameters or start from scratch, repeat until it works or you spent like TONS of time with no result (and read TONS of info)

Well, this is awkyard...

I've used Xen and kvm with various gpus with success. Only this gpu is causing me problems...

I did read all pages and search other users having my problems... It's been 4 weeks since I start experimenting with the system.

Applied all the solution in threads with no avail...

Based on first page, kernel 3.16 is a requirement but I wanted to achieve same thing with 13.4... Patching has been done also modifying patches from patchwork to adapt to my kernel...

Read throughly also your blog pages and my system conforms to your description...

I've also helped kantras at the time I was using xen 4.3  on AUR... So I think that I have a minimum understanding of what It's happening...

P.S. Links are broken on your post...

Why are you assuming I did not read all infos and done research? Is something not clear from my forum post?

Last edited by evilsephiroth (2014-09-17 08:28:27)

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

...
Why are you assuming I did not read all infos and done research? Is something not clear from my forum post?

and that's why:
- first part of the reason:

Just to be clear...
OVMF is a replacement for VGA, right?

- second part of the reason(may be found on previous page; yes, i lied about it being on THIS page):

http://vfio.blogspot.ru/2014/08/primary … t-vga.html

i might be wrong here, but from my point of view, after reading (yes, reading - not just skimming through) the blog post, there may be many questions, but not the one you asked.

you may check others' reports here (there is even r9 270x on 3.14 kernels): report sheet

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

...
Why are you assuming I did not read all infos and done research? Is something not clear from my forum post?

and that's why:
- first part of the reason:

Just to be clear...
OVMF is a replacement for VGA, right?

- second part of the reason(may be found on previous page; yes, i lied about it being on THIS page):

http://vfio.blogspot.ru/2014/08/primary … t-vga.html

i might be wrong here, but from my point of view, after reading (yes, reading - not just skimming through) the blog post, there may be many questions, but not the one you asked.

you may check others' reports here (there is even r9 270x on 3.14 kernels): report sheet

Checking right now the google docs...

relating to OVMF, I was confirming this information  from aw "  OVMF isn't a "test", it's an alternative to VGA.  Since you're not getting any output from the monitor, it seems like you have an issue with VGA routing."

Nevertheless, will check the google doc and report results... I would love to use proxmox and using a more recent kernel but I don't know if It's a feasible way to do this....

noobman

Member

Registered: 2013-12-13

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You should not need vfio_iommu_type1 allow_unsafe_interrupts=1 on either of these systems unless interrupt remapping is disabled in the BIOS.

Dunno why, but i need it just for X79. Without it does not work, and i get the warning as per last line:

dmesg | grep -E -i 'dmar|rmr|iommu|drh|remap'
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=6d0f0340-4faa-4691-b3b6-c26d936404f2 rw radeon.dpm=1 intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:1d2d,8086:1d3a
[    0.000000] ACPI: DMAR 0x00000000CB7DF000 0000E8 (v01 INTEL  DX79TO   0000028A MSFT 0100000D)
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=6d0f0340-4faa-4691-b3b6-c26d936404f2 rw radeon.dpm=1 intel_iommu=on pci-stub.ids=10de:1184,10de:0e0a,8086:1d2d,8086:1d3a
[    0.000000] Intel-IOMMU: enabled
[    0.024733] dmar: Host address width 40
[    0.024734] dmar: DRHD base: 0x000000fe901000 flags: 0x0
[    0.024739] dmar: IOMMU 0: reg_base_addr fe901000 ver 1:0 cap d2008c10ef0462 ecap f0207a
[    0.024740] dmar: DRHD base: 0x000000fe900000 flags: 0x1
[    0.024743] dmar: IOMMU 1: reg_base_addr fe900000 ver 1:0 cap d2078c106f0462 ecap f020fe
[    0.024744] dmar: RMRR base: 0x000000c8dda000 end: 0x000000c8ddafff
[    0.024745] dmar: RMRR base: 0x000000c8ddc000 end: 0x000000c8ddcfff
[    0.378367] DMAR: No ATSR found
[    0.378402] IOMMU 0 0xfe901000: using Queued invalidation
[    0.378405] IOMMU 1 0xfe900000: using Queued invalidation
[    0.378407] IOMMU: Setting RMRR:
[    0.378417] IOMMU: Setting identity map for device 0000:00:1d.0 [0xc8ddc000 - 0xc8ddcfff]
[    0.378441] IOMMU: Setting identity map for device 0000:00:1a.0 [0xc8dda000 - 0xc8ddafff]
[    0.378455] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.378463] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[   27.591182] vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

I also get lots of errors but somehow i am used to them:

dmesg | grep -E -i 'fail|error|warn|panic| bug|backtrace'
[    0.000000] ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Pm1aEventBlock: 32/16 (20140724/tbfadt-618)
[    0.000000] ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/PmTimerBlock: 32/24 (20140724/tbfadt-618)
[    0.000000] ACPI BIOS Warning (bug): Invalid length for FADT/Pm1aEventBlock: 16, using default 32 (20140724/tbfadt-699)
[    0.000000] ACPI BIOS Warning (bug): Invalid length for FADT/PmTimerBlock: 24, using default 32 (20140724/tbfadt-699)
[    0.095166] tsc: Marking TSC unstable due to check_tsc_sync_source failed
[    0.217170] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.384545] ioapic: probe of 0000:00:05.4 failed with error -22
[    2.077427] ACPI Warning: SystemIO range 0x0000000000004000-0x000000000000401f conflicts with OpRegion 0x0000000000004000-0x000000000000400f (\_SB_.PCI0.SMBS.SMBI) (20140724/utaddress-258)
[    2.988725] radeon 0000:03:00.0: registered panic notifier

The fadt ones started to appear since 3.14. The ioapic which fails to be recognized is "00:05.4 PIC: Intel Corporation Xeon E5/Core i7 I/O APIC (rev 07) \ Subsystem: Intel Corporation Device 4953". The path to use pci=nocrs or alternatives was not leading anywhere. The radeon panic notifier is weird coz the xorg log has really no (ee) and only few not related (ww). I think the video card (R7 240) should of been recognized as r600 but its seen as r300.
Maybe i am doing something wrong. In general i used to blame this board, coz i had lots of issues with this particular board and its badly broken bios(es). I tried other bioses but from previous one back i had that bug which was loading the cpu and made it hit 90 degrees at idle, not even booted, just inside bios sensors screen could watch temp increasing like crazy with apparently no reason - so since then i keep blaming mei and related "features" for almost anything.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

speaking of ovmf, i wonder if there is some kind of list of uefi-compliant gpus, or some kind of way to know if some distinct gpu would work or now

gigabyte's hd 6450 does NOT support uefi, palit gt 430 is pending for test (i doubt it will support it though)

so i'll probably have to go "buy new cheap(-est) one(-s)" route - would not be that great to blind-pick anything

I would hope that a card advertised with Windows 8 support would include a UEFI ROM.  If it claims to support secure boot, then it must support UEFI.  Otherwise there's quite a selection of ROMs that can be tested prior to purchase in the techpowerup database http://www.techpowerup.com/vgabios/.  It's also possible that another vendor may ship a UEFI BIOS compatible with your card that can be loaded into QEMU via the romfile option.  I also found that EVGA didn't ship a UEFI ROM for my GTX660, but will send one via support ticket.  Neither of your cards are very new though, so you may indeed be out of luck with them.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You should not need vfio_iommu_type1 allow_unsafe_interrupts=1 on either of these systems unless interrupt remapping is disabled in the BIOS.

Dunno why, but i need it just for X79. Without it does not work, and i get the warning as per last line:

[    0.024739] dmar: IOMMU 0: reg_base_addr fe901000 ver 1:0 cap d2008c10ef0462 ecap f0207a
[    0.024743] dmar: IOMMU 1: reg_base_addr fe900000 ver 1:0 cap d2078c106f0462 ecap f020fe
...
[   27.591182] vfio_iommu_type1_attach_group: No interrupt remapping support.  Use the module param "allow_unsafe_interrupts" to enable VFIO IOMMU support on this platform

I suspect it's either disabled in the BIOS or your kernel doesn't support interrupt remapping.  The VT-d Extended Capability register (ecap) defines but 3 to indicate interrupt remapping support.  It's set on both the IOMMUs in your system according to the log, so the hardware is capable of interrupt remapping.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

speaking of ovmf, i wonder if there is some kind of uefi-compliant gpus, or some kind of way to know if some distinct gpu would work or now

gigabyte's hd 6450 does NOT support uefi, palit gt 430 is pending for test (i doubt it will support it though)

so i'll probably have to go "buy new cheap(-est) one(-s)" route - would not be that great to blind-pick anything

Just to be clear...

OVMF is a replacement for VGA, right?

can it be a problem solver in case of ATI bsod?

My guess would be no because they're related mainly to passtrough itself and not to vga routing...

OVMF is a replacement for VGA, the idea is to avoid legacy VGA access and thus all the arbitration issues that go along with it by using a legacy-free BIOS implementation.

It likely does not solve the ATI BSOD.

Have you tried switching to a 440FX machine type, with either VGA or OVMF?

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Arakatak

Member

Registered: 2014-09-12

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Same identical problem.

I don't think our problems are identical in nature. I suggest you should try to pass VGA without audio, then boot Windows VM in safe mode, uninstall ALL ATi drivers (better to revert the system using restore points, Catalyst creates one on installation). After that install only VGA driver without CCC and other AMD bloatware. Report if it works

My problem usually results in black screen, not in BSOD. Unfortunately, i still can't fix it.

P.S. I'm also curious about What means "regular installations"?

I think it means trying to move a bare metal installation to a VM rather than doing a fresh install in the VM.

You've lead me to the idea a problem may be in VGA/system in general. The manufacturer claims the videocard is Win8 compatible. I can't test it though. My HDD is fully mapped as lvm. Still, i think doing a fresh install in VM is the best decision.

Are there any users who were able to do a successful AMD R7 260x setup with Catalyst drivers? The report sheet contains some mentions about this GPU. How can i find that people?

evilsephiroth

Member

Registered: 2014-09-16

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did you install the i915 arbiter patch?

Because before patching kernel, I was having Windows logo ==> black screen ==> reboot...

Then after patch, I could see seabios + bsod...

I don't know if it's related but...

Currently, I'm redoing installation so to will try to pass only video without radeon audio...

Will make you know

Same identical problem.

I don't think our problems are identical in nature. I suggest you should try to pass VGA without audio, then boot Windows VM in safe mode, uninstall ALL ATi drivers (better to revert the system using restore points, Catalyst creates one on installation). After that install only VGA driver without CCC and other AMD bloatware. Report if it works

My problem usually results in black screen, not in BSOD. Unfortunately, i still can't fix it.

P.S. I'm also curious about What means "regular installations"?

I think it means trying to move a bare metal installation to a VM rather than doing a fresh install in the VM.

You've lead me to the idea a problem may be in VGA/system in general. The manufacturer claims the videocard is Win8 compatible. I can't test it though. My HDD is fully mapped as lvm. Still, i think doing a fresh install in VM is the best decision.

Are there any users who were able to do a successful AMD R7 260x setup with Catalyst drivers? The report sheet contains some mentions about this GPU. How can i find that people?

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Same identical problem.

I don't think our problems are identical in nature. I suggest you should try to pass VGA without audio, then boot Windows VM in safe mode, uninstall ALL ATi drivers (better to revert the system using restore points, Catalyst creates one on installation). After that install only VGA driver without CCC and other AMD bloatware. Report if it works

My problem usually results in black screen, not in BSOD. Unfortunately, i still can't fix it.

P.S. I'm also curious about What means "regular installations"?

I think it means trying to move a bare metal installation to a VM rather than doing a fresh install in the VM.

You've lead me to the idea a problem may be in VGA/system in general. The manufacturer claims the videocard is Win8 compatible. I can't test it though. My HDD is fully mapped as lvm. Still, i think doing a fresh install in VM is the best decision.

Are there any users who were able to do a successful AMD R7 260x setup with Catalyst drivers? The report sheet contains some mentions about this GPU. How can i find that people?

Try to pass only gpu without hd audio (but assign both to vfio-pci).