Arch Linux

kidchunks

Member

Registered: 2014-12-03

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

try without -device vfio-pci,host=85:00.1,bus=root.1,addr=00.1 \ ?
i don't have amd, but reading this thread i think you need to remove amd audio device

edit: and try 440fx chipset not q35

I removed the audio device:

qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=85:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

Still the same error:

vfio-pci 0000:85:00.0: irq 89 for MSI/MSI-X

I tried with pc (440fx) and got the same error.

qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=85:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

Here is pc with pci-assign (no vfio):
qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device pci-assign,host=85:00.0 \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

Same error but this pci-stub throwing it.

pci-stub 0000:85:00.0: irq 89 for MSI/MSI-X

Last edited by kidchunks (2014-12-03 16:20:23)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

try without -device vfio-pci,host=85:00.1,bus=root.1,addr=00.1 \ ?
i don't have amd, but reading this thread i think you need to remove amd audio device

edit: and try 440fx chipset not q35

I removed the audio device:

qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=85:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

Still the same error:

vfio-pci 0000:85:00.0: irq 89 for MSI/MSI-X

This is not an error.  Not everything the kernel prints is an error.  This is simply indicating that the driver vfio-pci allocated irq 89 for the device for MSI/X.  That's it.

I tried with pc (440fx) and got the same error.

qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device ioh3420,bus=pci.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=85:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

ioh3402 should not be mixed with 440FX, remove that line, remove the bus= on the vfio-pci line, and set addr to something like 9.0 since 0.0 won't be available on the host bridge.

Here is pc with pci-assign (no vfio):
qemu-system-x86_64 -enable-kvm -M pc -m 1024 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device pci-assign,host=85:00.0 \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize \
-no-reboot

Same error but this pci-stub throwing it.

pci-stub 0000:85:00.0: irq 89 for MSI/MSI-X

Again, not an error.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You do know that xeon lacks integrated video core.
He may be needing it to get host output. However, sometimes buying a cheap-as-cardboard GT610 can be more profitable than buying i7 for integrated video.

Whoa, just noted that this post is #3499.

Check ark, E3-1245v2 has HD P4000 video.

Whoa! That was a surprise. There's only four versions in whole E3 v2 line-up, 1225, 1245 and 1275. And one with 2000 graphics.
Well, E3-1240v2 is a more popular choice, and it lacks graphics. Attention must be paid when choosing the CPU, as it more likely will not have it.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

tatsu

Member

Registered: 2014-12-03

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all!

I'd like to know how to apply the kernel patches. the two links simply lead to a forum archive with text posts containing what seems to be some kernel files.

I have no idea how to use this. some help please?

winie

Member

Registered: 2014-03-01

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all!

I'd like to know how to apply the kernel patches. the two links simply lead to a forum archive with text posts containing what seems to be some kernel files.

I have no idea how to use this. some help please?

You don't need those; just the first file which is a .tar.gz file
Download and extract it a folder and go to that folder and run these lines:
sudo pacman -S base-devel
makepkg -s

After the kernel compile is finnished few .pkg.tar.xz files will be created. save them if you want for next time.
Install them by running:
sudo pacman -U *.pkg.tar.xz

winie

Member

Registered: 2014-03-01

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The kvm version of intel vgt is out:
http://www.phoronix.com/vr.php?view=MTg1MzQ

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The kvm version of intel vgt is out:
http://www.phoronix.com/vr.php?view=MTg1MzQ

Good news.

Now only we need aw or someone smart enough to make it use each output for each vm  (probably not possible )

winie

Member

Registered: 2014-03-01

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The kvm version of intel vgt is out:
http://www.phoronix.com/vr.php?view=MTg1MzQ

Good news.

Now only we need aw or someone smart enough to make it use each output for each vm  (probably not possible )

Actually that's what they're doing. Each vm gets a virtual monitor which can be connected to a physical output or not.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hmm... i tried xen, and i couldn't get 2 vms at same time to output to different outputs at same time. Am i missing something?

Imagine if we could output 3 windows vm's to 3 different outputs at same time, so we could have 3 separate vm's with 3d acceleration on single host with igpu...

Last edited by slis (2014-12-04 11:47:06)

tatsu

Member

Registered: 2014-12-03

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The kvm version of intel vgt is out:
http://www.phoronix.com/vr.php?view=MTg1MzQ

OMG wow thanks for sharing the love, man!

You don't need those; just the first file which is a .tar.gz file
Download and extract it a folder and go to that folder and run these lines:
sudo pacman -S base-devel
makepkg -s
After the kernel compile is finnished few .pkg.tar.xz files will be created. save them if you want for next time.
Install them by running:
sudo pacman -U *.pkg.tar.xz

I installed a newer kernel? Do I really have to use his?

winie

Member

Registered: 2014-03-01

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I installed a newer kernel? Do I really have to use his?

If using ovmf then probably not.
The patches on the first post are for vga arbitration, acs override and radeon related fixes.
Booting vm with ovmf  in efi mode solves the first one. 
acs isn't an issue if you have only one pcie graphic card or you passthrough all to the same vm

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The kvm version of intel vgt is out:
http://www.phoronix.com/vr.php?view=MTg1MzQ

Good news.

Now only we need aw or someone smart enough to make it use each output for each vm  (probably not possible )

Actually that's what they're doing. Each vm gets a virtual monitor which can be connected to a physical output or not.

Whoa, this is cool. I was just wondering exactly how this sharing of a GPU between different VMs would work, as there is only 1 physical screen attached. Being able to use the different outputs for different VMs seems so impossible Do you have any sources/pointers to more info on this topic?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Do you have any sources/pointers to more info on this topic?

http://lists.freedesktop.org/archives/i … 56652.html
Check out the ending - there is links to the presentations and there's links for sources.
Sadly, i am unable to check out these sources myself, so i can't help you anymore further.

whole github is banned in my country for awesome reasons, and i haven't got a VPN up yet

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

zsph

Member

Registered: 2014-09-04

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone succeeded to use qemu + vfio without sudo/root? I tried to chown /dev/vfio/1 and /dev/vfio/vfio and increased ulimit to 9000000 (8GB dedicated to qemu + ~1GB for safety) but I'm gettings such errors:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f3b7ad1ce60, 0x0, 0x80000000, 0x7f3944000000) = -12 (Cannot allocate memory)
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f3b7ad1ce60, 0x100000000, 0x180000000, 0x7f39c4000000) = -12 (Cannot allocate memory)
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: memory listener initialization failed for container

I don't use libvirt btw, just qemu.

---

Asus P8Z77-V LK, Core i7 3770, Asus NVIDIA GTX 770 DirectCU II OC, Debian x64, EFI, Kernel 3.16 + i915 arbiter patch.
3DMark11: bare metal P10231 vs vfio P10039: 98% of performance!

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone succeeded to use qemu + vfio without sudo/root? I tried to chown /dev/vfio/1 and /dev/vfio/vfio and increased ulimit to 9000000 (8GB dedicated to qemu + ~1GB for safety) but I'm gettings such errors:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f3b7ad1ce60, 0x0, 0x80000000, 0x7f3944000000) = -12 (Cannot allocate memory)
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio_dma_map(0x7f3b7ad1ce60, 0x100000000, 0x180000000, 0x7f39c4000000) = -12 (Cannot allocate memory)
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: memory listener initialization failed for container

I don't use libvirt btw, just qemu.

How are you setting ulimit -l?  It's rather tricky

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

zsph

Member

Registered: 2014-09-04

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

How are you setting ulimit -l?  It's rather tricky

First I calculated it as (8 * 1024 + 20) * 1024 = 8409088, but I was getting same errors. So I though that extra 20mb was not enough and went with 9000000.

I do it as

sudo -s
ulimit -l 9000000

---

Asus P8Z77-V LK, Core i7 3770, Asus NVIDIA GTX 770 DirectCU II OC, Debian x64, EFI, Kernel 3.16 + i915 arbiter patch.
3DMark11: bare metal P10231 vs vfio P10039: 98% of performance!

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

How are you setting ulimit -l?  It's rather tricky

First I calculated it as (8 * 1024 + 20) * 1024 = 8409088, but I was getting same errors. So I though that extra 20mb was not enough and went with 9000000.

I do it as

sudo -s
ulimit -l 9000000

And then what?  At this point you either run qemu directly, which means your running it as root and the ulimit was unnecessary, you ^d back to yourself and you've lost the ulimit, or you su back to yourself.  The last option is the only one that keeps the ulimit.  If you want to make a user that has elevated limits then look at /etc/security/limits.conf, or however your distro does it.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Norcoen

Member

From: Norway

Registered: 2013-11-02

Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did someone try getting Intels GVT-g / KVMGT to work with KVM yet?
Heard it's ready now after already working in Xen for a while now.

zsph

Member

Registered: 2014-09-04

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And then what?

That was my mistake perhaps, I was doing ^d. But then I also tried su - <myusername> and still received memory allocation error.

If you want to make a user that has elevated limits then look at /etc/security/limits.conf, or however your distro does it.

I will look, thanks.

---

Asus P8Z77-V LK, Core i7 3770, Asus NVIDIA GTX 770 DirectCU II OC, Debian x64, EFI, Kernel 3.16 + i915 arbiter patch.
3DMark11: bare metal P10231 vs vfio P10039: 98% of performance!

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And then what?

That was my mistake perhaps, I was doing ^d. But then I also tried su - <myusername> and still received memory allocation error.

That should have worked if you set the limit high enough:

$ sudo -s
# ulimit -l 1000
# ulimit -l
1000
# su - alwillia
$ ulimit -l
1000

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

kidchunks

Member

Registered: 2014-12-03

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

This is not an error.  Not everything the kernel prints is an error.  This is simply indicating that the driver vfio-pci allocated irq 89 for the device for MSI/X.  That's it.

Thanks for the clarification regarding irq!

ioh3402 should not be mixed with 440FX, remove that line, remove the bus= on the vfio-pci line, and set addr to something like 9.0 since 0.0 won't be available on the host bridge.

Removed ioh3402 and changed addr card:

qemu-system-x86_64 -enable-kvm -M pc -m 4096 -cpu host \
-smp 8,sockets=1,cores=4,threads=4 \
-bios /usr/share/seabios/bios.bin -vga none \
-device vfio-pci,host=85:00.0,addr=08.0,multifunction=on,x-vga=on \
-device vfio-pci,host=85:00.0,addr=08.1 \
-drive file=/root/qemu/win7.img,cache=none,if=virtio \
-drive file=/root/qemu/win7.iso,index=0,media=cdrom \
-drive file=/root/qemu/virtio-win-0.1-81.iso,index=1,media=cdrom \
-usb -usbdevice host:046d:c068 \
-vnc :1 \
-usbdevice tablet \
-daemonize

I was able to install the drivers but upon reboot, got stuck in a boot loop.

Figured it could be driver related so I tried the latest catalyst beta drivers (14.11.2) and it worked! Audio and video are passing through with no problems.

The above also fixed an issue I had with video card freezing (stuck image) on reboot/shutdown so no need for no-reboot.

The only problem I have now is that I can't see the bios(seabios) screen but that's minor given it still boots into whatever drive is first. Strangely enough, OVMF(UEFI) works fine. Maybe something with the card.

In any case, glad to get it working after a week!

Thanks AW and others for the help.

To those running an r9 290x...use the 14.11.2 drivers!

Last edited by kidchunks (2014-12-06 03:49:43)

mastersplinter777

Member

Registered: 2012-10-09

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Finally succeeded to passthrough my Zotac GT630 to OSX 10.9.5 with working 3D
https://imagizer.imageshack.us/v2/1152x … u1Qwht.png

winie

Member

Registered: 2014-03-01

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Finally succeeded to passthrough my Zotac GT630 to OSX 10.9.5 with working 3D
https://imagizer.imageshack.us/v2/1152x … u1Qwht.png

I didn't get os x 10.9.5 working in qemu. I get a fast bootloop from seabios.
Can you post your command line?

anthonym52

Member

Registered: 2014-12-07

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Finally succeeded to passthrough my Zotac GT630 to OSX 10.9.5 with working 3D
https://imagizer.imageshack.us/v2/1152x … u1Qwht.png

I didn't get os x 10.9.5 working in qemu. I get a fast bootloop from seabios.
Can you post your command line?

I second this -- also, has anyone written an Ubuntu tutorial or pre-compiled everything for 14.04?

risho

Member

Registered: 2011-09-06

Posts: 44

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

so i switched out my gtx 580 that was  running via bios and vga which was running fine to a radeon r9 290x in hopes of being able to get it set up using ovmf. I can get it to output a display by using

sudo vfio-bind 0000:01:00.0 0000:01:00.1 #0000:00:03.0

qemu-system-x86_64 -enable-kvm -M q35 -m 8196 -mem-path /dev/hugepages  -cpu host,kvm=off \
        -smp 6,sockets=1,cores=3,threads=2 \
        -boot menu=on \
        -bios /usr/share/qemu/bios.bin -vga none\
        -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
        -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on -nographic \
        -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \

on vga and seabios but on ovmf it won't work. it shows no output. i accidentially left the command running over night and it appears it output "3h" every once and a while after many many lines. this is the command i was running to try to get it running with ovmf

sudo vfio-bind 0000:01:00.0 0000:01:00.1 #0000:00:03.0

qemu-system-x86_64 -enable-kvm -m 8196  -cpu host,kvm=off \
        -smp 6,sockets=1,cores=3,threads=2 \
        -boot menu=on \
        -drive if=pflash,format=raw,readonly,file=/home/anonymous/kvm/OVMF_CODE.fd \
        -drive if=pflash,format=raw,file=/home/anonymous/kvm/OVMF_VARS.fd \
        -device vfio-pci,host=01:00.0 -nographic  -vga none \
        -device vfio-pci,host=01:00.1 \

i tried running aw's rom parser but it would output a 0 byte file even on a fresh restart. does that mean my card doesnt support efi?

when i ran with "multifunction=on,x-vga=on" it would show uefi interactive shell inside of my terminal but still wouldnt show anyhting on my graphics card screen.

UEFI Interactive Shell v2.1
EDK II
UEFI v2.40 (EDK II, 0x00010000)
Mapping table
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x1)/Ata(0x0)
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0)
     BLK1: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1)
Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell> 

Last edited by risho (2014-12-07 07:31:00)