KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

argon · 2013-07-11 20:00:35

Hello and thank you for your great tutorial.

I did everything I needed to do (what you mentioned), dmesg does not show any errors, however my second monitor
is not turning on.

I was trying two approaches: at first I had assigned my second GPU (nvidia 7600GT) to a machine, and then installed windows7.
After that I did everything to make this hardware work - installation of drivers, but in hardware manager in windows, there is
an info: "bla bla does not work, because it has problems" or something similar. I thought, maybe using command line I'd be more
lucky, and I tried with

qemu-system-x86_64 -enable-kvm -M q35 -m 4096 -cpu host -smp 2,sockets=1,cores=2,threads=1 \
 -bios /usr/share/qemu/bios.bin -vga none -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 -device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ 
 -nographic -boot c -drive file=/var/lib/libvirt/images/Windows7.img,if=virtio

Windows is starting (I hear how HDD works), but my screen is still dead. I haven't tried any qemu/kernel patching yet.
Do you have any ideas what should I do?

EDIT: I actually managed to make this work. I'll describe my case later.

Last edited by argon (2013-07-13 13:35:49)

argon · 2013-07-13 16:30:17

Ok, I'll try to be quick here. What helped me was in fact pointing out to qemu where seabios has its
roms. After I've compiled seabios given on the first page, I added

-L /path/to/seabios/out/ -L /usr/share/qemu/bios.bin

to the command line and my GPU responded and now I am able to use it without any problems.
However I'm not sure whether mobo BIOS update I did, was helpful - probably.

I also have to say, I am also using patched linux-mainline and qemu.
My hardware:
MB: ASUS M4A89TD PRO USB3.0
Host GPU: ASUS NVIDIA GTX460
Guest GPU: GIGABYTE NVIDIA 7600GT

Last edited by argon (2013-07-13 16:33:54)

myweb · 2013-07-13 17:38:37

apoapo wrote:

I finally got some progress.
After trying mainline kernel and with vbios, I went to bios/uefi and turned the Primary VGA card from onboard to pci-express ...
Now I see the seabios in passthroughed screen, but with blinking cursor in it, so i will try some things .

Hello,

Could you please describe how did you get vbios and how you specified vbios for qemu?

Thank you in advance!

nbhs · 2013-07-13 18:58:28

argon wrote:

Ok, I'll try to be quick here. What helped me was in fact pointing out to qemu where seabios has its
roms. After I've compiled seabios given on the first page, I added
-L /path/to/seabios/out/ -L /usr/share/qemu/bios.bin
to the command line and my GPU responded and now I am able to use it without any problems.
However I'm not sure whether mobo BIOS update I did, was helpful - probably.
I also have to say, I am also using patched linux-mainline and qemu.
My hardware:
MB: ASUS M4A89TD PRO USB3.0
Host GPU: ASUS NVIDIA GTX460
Guest GPU: GIGABYTE NVIDIA 7600GT

Sorry for the late reply i was kinda busy working, so you're saying it didnt work with the -bios command? my seabios package should install it to /usr/share/qemu/bios.bin by default

nbhs · 2013-07-13 19:00:41

naythuwin wrote:

[abcd@unknow ~]$ qemu-system-x86_64 -enable-kvm -m 2048 -cpu host -smp 2 -device vfio-pci,host=05:00.0,x-vga=on -bios /usr/share/qemu/bios.bin -vga none
KVM internal error. Suberror: 1
emulation failure
EAX=00000000 EBX=00000000 ECX=0000a19e EDX=000003d4
ESI=00009e48 EDI=0000823c EBP=0000c58a ESP=00008232
EIP=0009ffff EFL=00010006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 ffffffff 00809300
CS =0000 00000000 ffffffff 00809b00
SS =0000 00000000 ffffffff 00809300
DS =0000 00000000 ffffffff 00809300
FS =0000 00000000 ffffffff 00809300
GS =0000 00000000 ffffffff 00809300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT= 000fd808 00000037
IDT= 00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
cpu : intel core i7
mobo : foxconn h61ap
kernel : Linux abcdef 3.10.0-1-mainline #1 SMP PREEMPT Wed Jul 10 23:38:07 MMT 2013 x86_64 GNU/Linux
graphic : onboard intel and pny geforce 8400gs pci 512M low profile
lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
05:00.0 VGA compatible controller: NVIDIA Corporation G98 [GeForce 8400 GS Rev. 2] (rev a1)
secondery passthrough is work for me but primary is not
help me!

Does this only happens when using vfio-pci? You should also use the q35 model and put your gpu behind a root port like the redhat dev recommended

nbhs · 2013-07-13 19:03:02

myweb wrote:

apoapo wrote:
I finally got some progress.
After trying mainline kernel and with vbios, I went to bios/uefi and turned the Primary VGA card from onboard to pci-express ...
Now I see the seabios in passthroughed screen, but with blinking cursor in it, so i will try some things .
Hello,
Could you please describe how did you get vbios and how you specified vbios for qemu?
Thank you in advance!

myweb here's the discussion of some user that got it working with an intel board: http://lists.gnu.org/archive/html/qemu- … 01722.html, also like i mentioned before this patch might be of use http://lists.gnu.org/archive/html/qemu- … 01722.html you could also try doing a bug report on qemu-devel or kvm-devel

Last edited by nbhs (2013-07-13 19:08:17)

argon · 2013-07-13 20:22:30

nbhs wrote:

argon wrote:
Ok, I'll try to be quick here. What helped me was in fact pointing out to qemu where seabios has its
roms. After I've compiled seabios given on the first page, I added
-L /path/to/seabios/out/ -L /usr/share/qemu/bios.bin
to the command line and my GPU responded and now I am able to use it without any problems.
However I'm not sure whether mobo BIOS update I did, was helpful - probably.
I also have to say, I am also using patched linux-mainline and qemu.
My hardware:
MB: ASUS M4A89TD PRO USB3.0
Host GPU: ASUS NVIDIA GTX460
Guest GPU: GIGABYTE NVIDIA 7600GT
Sorry for the late reply i was kinda busy working, so you're saying it didnt work with the -bios command? my seabios package should install it to /usr/share/qemu/bios.bin by default

No problem, I really enjoyed digging in the issue by myself . And no, it didn't work.
But now, I realized that somehow md5 of those two bios.bin files were different. So maybe
I haven't in fact installed your package but I was only going to...

Now md5 sums are correct, and it should work.

Last edited by argon (2013-07-13 20:23:22)

nbhs · 2013-07-13 20:31:41

argon wrote:

nbhs wrote:
argon wrote:
Ok, I'll try to be quick here. What helped me was in fact pointing out to qemu where seabios has its
roms. After I've compiled seabios given on the first page, I added
-L /path/to/seabios/out/ -L /usr/share/qemu/bios.bin
to the command line and my GPU responded and now I am able to use it without any problems.
However I'm not sure whether mobo BIOS update I did, was helpful - probably.
I also have to say, I am also using patched linux-mainline and qemu.
My hardware:
MB: ASUS M4A89TD PRO USB3.0
Host GPU: ASUS NVIDIA GTX460
Guest GPU: GIGABYTE NVIDIA 7600GT
Sorry for the late reply i was kinda busy working, so you're saying it didnt work with the -bios command? my seabios package should install it to /usr/share/qemu/bios.bin by default
No problem, I really enjoyed digging in the issue by myself . And no, it didn't work.
But now, I realized that somehow md5 of those two bios.bin files were different. So maybe
I haven't in fact installed your package but I was only going to...
Now md5 sums are correct, and it should work.

Im actually surprised you got that card working gratz.

reintseri · 2013-07-16 06:29:59

Hi everyone and thanks for this tutorial, exiting stuff.

Do you guys know if there's any chance on getting this to work on a Mac Pro (2009). I guess
after a firmware update VT-d support should be present. I have OS X and Arch dual boot setup
but for some reason the /sys/bus/pci/drivers/vfio-pci folder doesn't exist. No iommu_group links
in the graphics card folders either.

I have a GeForce 9500 GT as a primary and I was hoping to passthrough a Radeon 5400 series
device.

uname -r gives me 3.9.9-1-ARCH which I guess should have the necessary options on.

I can post whatever additional info you need, thanks.

resr · 2013-07-16 09:33:54

reintseri

I had a similar experience which I'm presently working through. I can't speak for mac, but I had the same symptoms with my new arch system.

I added

intel_iommu=on

to my kernel parameters, which gives me "[ 0.000000] Intel-IOMMU: enabled" from a

dmesg | grep IOMMU

.

But I wasn't getting any iommu_group links under /sys/bus/pci/devices/<device>, the same as yourself.

My /sys/kernel/iommu_groups/ folder is empty too.

I'm about to rebuild the kernel, making sure that the

CONFIG_VFIO_IOMMU_TYPE1=y
CONFIG_VFIO=y
CONFIG_VFIO_PCI=y
CONFIG_VFIO_PCI_VGA=y

options are there.

And as per rabcor's experience halfway through the thread I'm going to try CONFIG_INTEL_IOMMU_DEFAULT_ON=y as well.

I'm a little bit different from the thread instructions as I'm only trying to pass a firewire card through, so I've skipped the patched seabios step.

I have things in my /sys/bus/pci/drivers/vfio-pci folder, but I've recompiled once.

resr · 2013-07-16 10:22:05

Don't mind me. I've found that VT-d isn't a feature of my 3770K cpu.

Bit silly paying the extra for the K as I haven't overclocked it yet.

reintseri · 2013-07-16 11:01:47

Ok I've made some progress. I set the intel_iommu=on and vfio_iommu_type1.allow_unsafe_interrupts=1 in grub and now I have the iommu_group folders. I also modprobed vfio-pci and the drivers are present also.

I can also bind a device without any (visible) error messages using the script. Testing with qemu starts a black screen (as it should) but no picture from the other graphics card.

I'd appreciate if someone can tell me how to troubleshoot the problem. I had the proprietary nvidia drivers for my host installed first, but I removed them and installed nouveau instead. No change I guess.

Qemu version: QEMU emulator version 1.5.1

argon · 2013-07-16 15:43:57

reintseri:
Hm, I don't know, whether removing nvidia drivers was helpful. However, you need
kernel and qemu with pci-resetting support. You can find linux-mainline (3.10), qemu with
mentioned patches in this thread and you'll have to compile them and install. Also you'll need
to use patched seabios - again, link available in this thread, few pages back.

This is exactly what I did and what seem to work for me.

Last edited by argon (2013-07-16 15:45:30)

perfectsine · 2013-07-16 19:26:27

nbhs:

Do you know if it would be possible to run a headless setup with one video card as "secondary" passthrough? I have a server that I want to test this on but, it doesn't have a built-in graphics adapter.

nbhs · 2013-07-16 19:32:46

perfectsine wrote:

nbhs:
Do you know if it would be possible to run a headless setup with one video card as "secondary" passthrough? I have a server that I want to test this on but, it doesn't have a built-in graphics adapter.

Using this guide probably not, though if you have a radeon card you could pass it through as a secondary device to qemu that means both the emulated card and the radeon card will be present on the vm, note that im not sure if this is working anymore since ive been using vfio instead of "pci-assing" for a long time ( last time i tried it worked )

nbhs · 2013-07-16 23:51:15

reintseri wrote:

Ok I've made some progress. I set the intel_iommu=on and vfio_iommu_type1.allow_unsafe_interrupts=1 in grub and now I have the iommu_group folders. I also modprobed vfio-pci and the drivers are present also.
I can also bind a device without any (visible) error messages using the script. Testing with qemu starts a black screen (as it should) but no picture from the other graphics card.
I'd appreciate if someone can tell me how to troubleshoot the problem. I had the proprietary nvidia drivers for my host installed first, but I removed them and installed nouveau instead. No change I guess.
Qemu version: QEMU emulator version 1.5.1

Could you try giving a shot using the patched builds?

reintseri · 2013-07-17 10:40:59

Ok I have now the patched mainline kernel, patched QEMU, patched Seabios and it .... works!

Thanks so much. I can give out my system specs later if someone else is trying to do this on a Mac Pro or with a similar setup.

Diggo · 2013-07-17 13:53:16

Congrats I´d like to see the specs, too. ´Trying to build a small list of components that work.

argon · 2013-07-17 18:16:26

Low-end NVIDIA 210 also works.

nbhs · 2013-07-17 18:58:36

reintseri wrote:

Ok I have now the patched mainline kernel, patched QEMU, patched Seabios and it .... works!
Thanks so much. I can give out my system specs later if someone else is trying to do this on a Mac Pro or with a similar setup.

Awesome please post you specs! i assume you got it working on intel right (well duh its a mac!)? I also updated the qemu patch yesterday, it seems i forgot to include some late patches that contain some quirks for the vga's, so i recommend people who couldnt get it to work to try the lastest builds

Also you can try doing this with mac osx, here's a guide on how to get mac running on qemu: http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/, unfortunately i couldnt get it to run on my amd system but it should work on intel, it would be sweet if you can get vga passthrough working with a mac install!

Last edited by nbhs (2013-07-17 19:19:43)

teekay · 2013-07-17 19:31:45

nbhs wrote:

Also you can try doing this with mac osx, here's a guide on how to get mac running on qemu: http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/, unfortunately i couldnt get it to run on my amd system but it should work on intel, it would be sweet if you can get vga passthrough working with a mac install!

I do have a snow leopard VM with some AMD patched kernel which works fine in VMware also in VirtualBox (no guest tools, though). But I failed to get it running in Qemu - with VGA passthrough that would be so cool, indeed.

myweb · 2013-07-18 10:25:55

Dear All,

I would like to install Windows 8 without any VGA-Passthrough and faced with terrible performance issue I suggest with storage: in takes more than half a hour to pass from partition selection dialog to coping files. A long time coping percentage stay at 0%. Is it OK? Is it any way to improve performance? Live Ubuntu works well.

I use the following lines for storage:
-device ahci,bus=pcie.0,id=ahci
-drive file=/home/nbhs/windows.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk

nbhs · 2013-07-18 13:40:50

myweb wrote:

Dear All,
I would like to install Windows 8 without any VGA-Passthrough and faced with terrible performance issue I suggest with storage: in takes more than half a hour to pass from partition selection dialog to coping files. A long time coping percentage stay at 0%. Is it OK? Is it any way to improve performance? Live Ubuntu works well.
I use the following lines for storage:
-device ahci,bus=pcie.0,id=ahci
-drive file=/home/nbhs/windows.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk

Could you post the entire qemu cmdline? You could also try using the faster virtio blk

Last edited by nbhs (2013-07-18 13:41:13)

myweb · 2013-07-18 20:07:33

qemu-system-x86_64 -M q35 -nodefconfig -readconfig ~/new/qemu/src/qemu-1.5.1/docs/q35-chipset.cfg -m 2048 -enable-kvm -cpu host \
-smp 2,sockets=1,cores=2,threads=1 -vga std \
-L ~/tmp/src/seabios-1.7.3/out/ \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/run/media/user/CopiedData/Soft/9200.16384.120725-1247_x64frev_Enterprise_VL_HRM_CENA_X64FREV_EN-US_DVD.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-drive file=/home/user/windows.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk

nbhs · 2013-07-18 20:45:16

myweb wrote:

qemu-system-x86_64 -M q35 -nodefconfig -readconfig ~/new/qemu/src/qemu-1.5.1/docs/q35-chipset.cfg -m 2048 -enable-kvm -cpu host \
-smp 2,sockets=1,cores=2,threads=1 -vga std \
-L ~/tmp/src/seabios-1.7.3/out/ \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/run/media/user/CopiedData/Soft/9200.16384.120725-1247_x64frev_Enterprise_VL_HRM_CENA_X64FREV_EN-US_DVD.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-drive file=/home/user/windows.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk

Have you tried to copy the iso image to your hard drive and install it from there?

Last edited by nbhs (2013-07-18 20:45:51)

Arch Linux

#226 2013-07-11 20:00:35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#227 2013-07-13 16:30:17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#228 2013-07-13 17:38:37

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#229 2013-07-13 18:58:28

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#230 2013-07-13 19:00:41

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#231 2013-07-13 19:03:02

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#232 2013-07-13 20:22:30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#233 2013-07-13 20:31:41

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#234 2013-07-16 06:29:59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#235 2013-07-16 09:33:54

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#236 2013-07-16 10:22:05

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#237 2013-07-16 11:01:47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#238 2013-07-16 15:43:57

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#239 2013-07-16 19:26:27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#240 2013-07-16 19:32:46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#241 2013-07-16 23:51:15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#242 2013-07-17 10:40:59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#243 2013-07-17 13:53:16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#244 2013-07-17 18:16:26

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#245 2013-07-17 18:58:36

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#246 2013-07-17 19:31:45

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#247 2013-07-18 10:25:55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#248 2013-07-18 13:40:50

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#249 2013-07-18 20:07:33

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#250 2013-07-18 20:45:16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Board footer