Arch Linux

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'll give the VMT option a try, but Windows isn't complaining about lack of virtualization ability.  It has a DCOM error in the Event viewer similar to http://superuser.com/questions/282994/w … ion-failed Bear in mind that the same installation worked under qemu script.

You can find the resulting qemu command line that libvirt uses in /var/log/libvirt (or simply using ps while the VM is running) to compare to your script

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

lefta

Member

Registered: 2015-05-18

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello there,

Some months ago, I managed to make a virtual machine with Qemu/KVM/VGA passthrough. Everything went fine.
One or two months ago, I started my VM, but Windows 7 froze right after openning a session. I couldn't find a way to get past this freeze.
On freeze, Qemu outputs "main-loop: WARNING: I/O thread spun for 1000 iterations".
Waiting past the loop is not an option, as Windows crashes complaining for missed clock interrupts.
However, the installation CD does not crash, but freezes every few seconds (without redisplaying the error message though).

I stripped down the command line to:

qemu-system-x86_64 \ 
-m 1G \ 
-vga none \ 
-boot d \ 
-drive file=/home/ced/Documents/iso/windows7.iso,id=virtdrv,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtdrv \ 
-enable-kvm \ 
-cpu host -smp 1,sockets=1,cores=1,threads=1

dmesg:

[ 1684.510717] kvm: zapping shadow pages for mmio generation wraparound

Prentend installing QEmu (-x means disabled, it is a gentoo)

app-emulation/qemu-2.3.0::gentoo  USE="aio alsa bluetooth caps curl fdt jpeg ncurses nls opengl pin-upstream-blobs png python seccomp threads usb uuid vhost-net vnc -accessibility -debug -filecaps -glusterfs -gtk -gtk2 -infiniband -iscsi -lzo -nfs -numa -pulseaudio -rbd -sasl -sdl (-selinux) -smartcard -snappy -spice -ssh -static -static-softmmu -static-user -systemtap -tci {-test} -tls -usbredir -vde -virtfs -xattr -xen -xfs" PYTHON_TARGETS="python2_7" QEMU_SOFTMMU_TARGETS="x86_64 -aarch64 -alpha -arm -cris -i386 -lm32 -m68k -microblaze -microblazeel -mips -mips64 -mips64el -mipsel -moxie -or32 -ppc -ppc64 -ppcemb -s390x -sh4 -sh4eb -sparc -sparc64 -unicore32 -xtensa -xtensaeb" QEMU_USER_TARGETS="-aarch64 -alpha -arm -armeb -cris -i386 -m68k -microblaze -microblazeel -mips -mips64 -mips64el -mipsel -mipsn32 -mipsn32el -or32 -ppc -ppc64 -ppc64abi32 -s390x -sh4 -sh4eb -sparc -sparc32plus -sparc64 -unicore32 -x86_64"

Before this problem, I updated my system. As I don't update it so often, I had about 200 updates (1/3 of my system), so I suspect an upgrade to be the culprit. But if I'm right, which one?
Google did not give any usefull result.

Hints :
- SystemRescueCD (x86) and Gentoo live CD (amd64) do not have the problem.
- Windows 7 installation CD (x86 & amd64) freezes.
- Windows 7 installation CD (x86) without KVM freezes even faster.
- All 2.x QEmu versions in portage tree suffer from the same problem (so from 2.1 to 2.3).
- Seabios 1.7.5 and 1.7.4 don't work.
- Linux kernels from 3.18.2 to 4.0.3 have the problem too.

My guesses :
- It is not KVM related (obviously).
- KVM seems to delay the freeze (why?).
- It does not seem to be directly QEmu related (but I could be wrong).
- It should be QEmu dependency related (but I'm not sure at all).

What can make Windows freeze but not Linux? ( Trolls, keep away please... )

Specs :
Gentoo Linux
QEmu 2.3.0 (I can downgrade/upgrade if needed)
Seabios 1.7.5 (I know 1.8 is out, but I couldn't install it on my gentoo)
CPU Intel i5-3470

I know it is not VGA passthrough related, but I though you were the most able to help me.

ceri

Member

Registered: 2013-10-12

Posts: 58

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't really know/understand the architecture of the interaction between PCI/e ports and the CPU & MB, and so I'm confused about the ACS isolation stuff, but from what you've written here and on your blog, would I be correct in assuming that:

a) if i wanted to use discrete graphics (w/ a UEFI bios) on a single guest, and intel integrated graphics on the host (with no other PCI/PCIe devices), I wouldn't have to worry about using any patches or isolation, nor the type of CPU/motherboard I get (except for CPU/GFX compatibility with the MB and VT-d, etc)?

Yes, if the discrete graphics card for the guest is the only plugin card, then pretty much anything will work.  If your chipset is not quirked for PCH root port isolation, then you'd need to install it in a processor root port slot to keep it from getting grouped with onboard devices, if it is quirked, then any slot will work.

*snip*

Thanks for the clarifications!

corna

Member

Registered: 2015-05-13

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi again
First, thanks for your help! Now I can run a Windows 7 guest for gaming and everything works almost perfectly!
The only problem is due to the i915 vga arbitration patch: enabling it causes lots of animation problems on my GNOME host. I'm trying to understand if I can use OVMF
Currently I can't run QEMU with OVMF:
* if I don't specify any vga passthrough in my QEMU configuration it works fine and it outputs everything in its window
* if I specify a vga passthrough it does not produce any output
I have a Zotac GTX660 Ti and, using this tool, it seems that it doesn't support UEFI (but I can run Windows in UEFI mode without the virtualization, does it matter?). But according to a post on this thread (that I can't find now), there is a chance of making a non-UEFI card work with UEFI by using a romfile of another card. The problem is that, also in BIOS mode, if I specify ANY romfile the vga passthrough does not work anymore... Do you have any idea?
This is my system:

MB: Sapphire H61 Pure Platinum ITX
CPU: i7-3770
RAM: 8GB DDR3 1600MHz
GPU: Zotac Nvidia GTX660 Ti

and this is my QEMU configuration (in BIOS mode)

#!/bin/sh
QEMU_AUDIO_DRV=alsa
DISKIMG="/media/nicola/data/qemu/WindowsVM.img"
qemu-system-x86_64 --enable-kvm -M q35 -smp 8,sockets=1,cores=4,threads=2 -drive file=${DISKIMG},if=virtio  -m 6144 \
-soundhw hda -cpu host,kvm=off \
-net nic,model=virtio -net user,hostname=windowsvm \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1 -vga none

Thanks!

Last edited by corna (2015-05-19 15:54:06)

Jimi

Member

From: Brooklyn, NY

Registered: 2009-09-25

Posts: 125

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone tried to pass two GPUs and use SLI on a VM?

---

github

Schlunze

Member

Registered: 2013-10-03

Posts: 53

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

can somebody share a startup.nsh script or link for swiching to pci-e graphic on boot?

Thanks in advance.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone tried to pass two GPUs and use SLI on a VM?

Not SLI, but XDMA Crossfire setup is running fine on my machine:
ASUS F2A55+Athlon X4 750K(trinity)+2xASUS HD7750 Silent. They do crossfire. Performance improvement is same as bare-metal. PCI-E 3.0 needed for improvement.

And i highly doubt SLI running - i've suggested trying HyperSLI to some guy with two GTX980 some pages back, and he failed doing the magic, so i think SLI won't work.

I don't know about regular crossfire, my crossfire setup doesn't require a separate bridge and relies solely on pci-e, so it's simpler to manage.

Last edited by Duelist (2015-05-19 21:30:29)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi again
First, thanks for your help! Now I can run a Windows 7 guest for gaming and everything works almost perfectly!
The only problem is due to the i915 vga arbitration patch: enabling it causes lots of animation problems on my GNOME host. I'm trying to understand if I can use OVMF
Currently I can't run QEMU with OVMF:
* if I don't specify any vga passthrough in my QEMU configuration it works fine and it outputs everything in its window
* if I specify a vga passthrough it does not produce any output
I have a Zotac GTX660 Ti and, using this tool, it seems that it doesn't support UEFI (but I can run Windows in UEFI mode without the virtualization, does it matter?). But according to a post on this thread (that I can't find now), there is a chance of making a non-UEFI card work with UEFI by using a romfile of another card. The problem is that, also in BIOS mode, if I specify ANY romfile the vga passthrough does not work anymore... Do you have any idea?
This is my system:

MB: Sapphire H61 Pure Platinum ITX
CPU: i7-3770
RAM: 8GB DDR3 1600MHz
GPU: Zotac Nvidia GTX660 Ti

and this is my QEMU configuration (in BIOS mode)

#!/bin/sh
QEMU_AUDIO_DRV=alsa
DISKIMG="/media/nicola/data/qemu/WindowsVM.img"
qemu-system-x86_64 --enable-kvm -M q35 -smp 8,sockets=1,cores=4,threads=2 -drive file=${DISKIMG},if=virtio  -m 6144 \
-soundhw hda -cpu host,kvm=off \
-net nic,model=virtio -net user,hostname=windowsvm \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1 -vga none

Thanks!

Well... Running windows UEFI on bare metal won't help, chances are you're just having CSM on and it continues to use VGA.

Try dumping your card's ROM using envytools(and recheck for UEFI-compatibility) and adding it as a romfile option to your non-OVMF setup. If it works - try searching(i'd ask manufacturer first since my GT610 has GOP(UEFI) in it's shipped ROM) for UEFI-compatible ROM somewhere there.
When you start up your OVMF-VM, it WON'T USE YOUR GRAPHICS UNTIL WINDOWS DRIVERS KICK IN.
So it boots, shows you all the animations of windows logos and stuff, and only then fires up your GPU and starts outputting.
Basically, Schlunze in his posts request a .nsh EFI shell script to make OVMF use PT-d GPU from the very beginning, not relying on QXL(i didn't even think it's possible!).

You won't get accelerated QEMU Windows Graphics when not doing secondary passthrough.

And if i recall correctly, i915-patch actually breaks DRI, so... yeah.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

corna

Member

Registered: 2015-05-13

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi again
First, thanks for your help! Now I can run a Windows 7 guest for gaming and everything works almost perfectly!
The only problem is due to the i915 vga arbitration patch: enabling it causes lots of animation problems on my GNOME host. I'm trying to understand if I can use OVMF
Currently I can't run QEMU with OVMF:
* if I don't specify any vga passthrough in my QEMU configuration it works fine and it outputs everything in its window
* if I specify a vga passthrough it does not produce any output
I have a Zotac GTX660 Ti and, using this tool, it seems that it doesn't support UEFI (but I can run Windows in UEFI mode without the virtualization, does it matter?). But according to a post on this thread (that I can't find now), there is a chance of making a non-UEFI card work with UEFI by using a romfile of another card. The problem is that, also in BIOS mode, if I specify ANY romfile the vga passthrough does not work anymore... Do you have any idea?
This is my system:

MB: Sapphire H61 Pure Platinum ITX
CPU: i7-3770
RAM: 8GB DDR3 1600MHz
GPU: Zotac Nvidia GTX660 Ti

and this is my QEMU configuration (in BIOS mode)

#!/bin/sh
QEMU_AUDIO_DRV=alsa
DISKIMG="/media/nicola/data/qemu/WindowsVM.img"
qemu-system-x86_64 --enable-kvm -M q35 -smp 8,sockets=1,cores=4,threads=2 -drive file=${DISKIMG},if=virtio  -m 6144 \
-soundhw hda -cpu host,kvm=off \
-net nic,model=virtio -net user,hostname=windowsvm \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1 -vga none

Thanks!

Well... Running windows UEFI on bare metal won't help, chances are you're just having CSM on and it continues to use VGA.

Try dumping your card's ROM using envytools(and recheck for UEFI-compatibility) and adding it as a romfile option to your non-OVMF setup. If it works - try searching(i'd ask manufacturer first since my GT610 has GOP(UEFI) in it's shipped ROM) for UEFI-compatible ROM somewhere there.
When you start up your OVMF-VM, it WON'T USE YOUR GRAPHICS UNTIL WINDOWS DRIVERS KICK IN.
So it boots, shows you all the animations of windows logos and stuff, and only then fires up your GPU and starts outputting.
Basically, Schlunze in his posts request a .nsh EFI shell script to make OVMF use PT-d GPU from the very beginning, not relying on QXL(i didn't even think it's possible!).

You won't get accelerated QEMU Windows Graphics when not doing secondary passthrough.

And if i recall correctly, i915-patch actually breaks DRI, so... yeah.

I've found 3 UEFI-compatible ROMs:

Asus.GTX660Ti.2048.130208.rom
Gigabyte.GTX660Ti.2048.130107.rom
MSI.GTX660Ti.2048.120828.rom

The system does not boot with any of these one (in non-OVMF mode)
I can't find any doc about envytools (I suppose I should use nvagetbios), can you please give me the doc's link or the syntax of the command?
Thanks

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've found 3 UEFI-compatible ROMs:

Asus.GTX660Ti.2048.130208.rom
Gigabyte.GTX660Ti.2048.130107.rom
MSI.GTX660Ti.2048.120828.rom

The system does not boot with any of these one (in non-OVMF mode)
I can't find any doc about envytools (I suppose I should use nvagetbios), can you please give me the doc's link or the syntax of the command?
Thanks

nvagetbios is a part of envytools. You git clone the source, compile it as noted in readme, and use nvagetbios to extract the bios. The instructions on how to use it are output by the program itself as usual.

Well.. You've tried to use regular SeaBios with those ROM and failed? Can you show me how you attach those files to your card?.. It should be something like "romfile=..."...

OT: yay, my next post is numbered 256!

Last edited by Duelist (2015-05-19 22:47:25)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

ghormoon

Member

Registered: 2015-05-20

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hi, did anyone encounter a situation, where passing in VGA into linux guest worked (including X), but windows one (with the same config, except things like disk path etc.) returned in the device manager "Code 12: This device cannot find enough free resources that it can use" ?

I'm trying with current Arch with libvirt, since I have only one gpu (laptop), it's possible to blindly write "virsh destroy vmname", but not to find the qemu, kill it and rebind it myself )

hw:
HP Elitebook 8560p, radeon 6470M

Last edited by ghormoon (2015-05-20 05:13:58)

corna

Member

Registered: 2015-05-13

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hi, did anyone encounter a situation, where passing in VGA into linux guest worked (including X), but windows one (with the same config, except things like disk path etc.) returned in the device manager "Code 12: This device cannot find enough free resources that it can use" ?

I'm trying with current Arch with libvirt, since I have only one gpu (laptop), it's possible to blindly write "virsh destroy vmname", but not to find the qemu, kill it and rebind it myself )

hw:
HP Elitebook 8560p, radeon 6470M

I have the same result if I pass multiple cards (discrete GPU + virtual GPU). I also had the same result when running Windows on bare metal (discrete + integrated), so I suppose is a limitation of Windows. Have you selected -vga none in the QEMU configuration?

ghormoon

Member

Registered: 2015-05-20

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

yes, even though -vga none doesn't actually make any difference. it only stops cirrus output from showing on vnc console, but will not remove the cirrus card itself.
If I remove the cirrus card, I end up in state, where the qemu will hang and not even try to boot anything. I can either have both, cirrus + passed in gpu, or none, but having only passed in one will hang seabios

corna

Member

Registered: 2015-05-13

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've found 3 UEFI-compatible ROMs:

Asus.GTX660Ti.2048.130208.rom
Gigabyte.GTX660Ti.2048.130107.rom
MSI.GTX660Ti.2048.120828.rom

The system does not boot with any of these one (in non-OVMF mode)
I can't find any doc about envytools (I suppose I should use nvagetbios), can you please give me the doc's link or the syntax of the command?
Thanks

nvagetbios is a part of envytools. You git clone the source, compile it as noted in readme, and use nvagetbios to extract the bios. The instructions on how to use it are output by the program itself as usual.

Well.. You've tried to use regular SeaBios with those ROM and failed? Can you show me how you attach those files to your card?.. It should be something like "romfile=..."...

OT: yay, my next post is numbered 256!

Yes, I should have read the README more carefully ;-)
I've extracted the BIOS of my GPU with envytools and the VM (in non-OVMF mode) works fine. But my BIOS is 1MB, while the techpowerup BIOSs are about 180KB... Are them in 2 different formats?
What do you mean with "it WON'T USE YOUR GRAPHICS UNTIL WINDOWS DRIVERS KICK IN."? Where does it output before the windows driver?
This is my QEMU script

#!/bin/sh
QEMU_AUDIO_DRV=pa
QEMU_PA_SAMPLES=128
DISKIMG="/winvm/winVM_C.img"
D_DRIVE="/home/nicola/Documents/winVM_D.img"
ROMFILE="/winvm/gtx660ti_bios/envytools_Zotac.rom"
OVMF_CODE="/winvm/ovmf-x64/OVMF_CODE-pure-efi.fd"
OVMF_VARS="/winvm/ovmf-x64/OVMF_VARS-pure-efi.fd"

ulimit -l 8409088

qemu-system-x86_64 --enable-kvm -M q35 -smp 8,sockets=1,cores=4,threads=2 -m 6144 \
-soundhw hda -cpu host,kvm=off -rtc base=localtime,clock=host \
-net nic,model=virtio -net user,hostname=windowsvm \
-drive file=${DISKIMG},if=virtio -drive file=${D_DRIVE},if=virtio \
-drive if=pflash,format=raw,readonly,file=${OVMF_CODE} \
-drive if=pflash,format=raw,file=${OVMF_VARS} \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,x-vga=on,multifunction=on,romfile=${ROMFILE} \
-device vfio-pci,host=01:00.1,bus=pcie.0 -vga none $@

The non-OVMF script is identical, except for the OVMF_CODE and OVMF_VARS lines

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I should have read the README more carefully ;-)
I've extracted the BIOS of my GPU with envytools and the VM (in non-OVMF mode) works fine. But my BIOS is 1MB, while the techpowerup BIOSs are about 180KB... Are them in 2 different formats?
What do you mean with "it WON'T USE YOUR GRAPHICS UNTIL WINDOWS DRIVERS KICK IN."? Where does it output before the windows driver?

A typical GPU ROM is 64 kilobytes, UEFI-enabled one is 128k. That's on AMD. On Nvidia cards there may be differences, try opening the downloaded files and your own VBIOS in hex editor and look. But actually, have you even bothered asking the manufacturer?
Anyway, i think both files should work, the one you've extracted and the one you've downloaded.

Where does it outputs? Well, depends on the number of GPUs you're assigning: when using windows7+OVMF, you'll have to use QXL GPU additionally. Originally OVMF did output it's text stuff(and tianocore logo) only to QXL, now it mirrors it to the assigned card. Then windows boot logo comes up and renders itself only on QXL, or, if there's no QXL present, nowhere. That's why i've warned you - there was some occasions where people thought that their VM hung, but it was loading silently with no output. It's only hanging if it eats 100% of one vCPU.

Last edited by Duelist (2015-05-20 13:39:27)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hi, did anyone encounter a situation, where passing in VGA into linux guest worked (including X), but windows one (with the same config, except things like disk path etc.) returned in the device manager "Code 12: This device cannot find enough free resources that it can use" ?

I'm trying with current Arch with libvirt, since I have only one gpu (laptop), it's possible to blindly write "virsh destroy vmname", but not to find the qemu, kill it and rebind it myself )

hw:
HP Elitebook 8560p, radeon 6470M

Surprise-surprise, you have two devices doing their shady VGA stuff, and only ONE VGA memory range! If i understand this correctly, VGA arbitration should manage this, but windows... just disables the second card, be it cirrus or assigned. Try doing OVMF-wa... you have 6470M and a notebook, OVMF will not work.
Actually, you should be able to bind one device to the VM and use it. But you have a notebook, i can't really remember anyone doing this with their laptop, and i do remember aw saying something about mobile GPUs being "not so dedicated" as their discrete counterparts. Or was it about intel integrated gpus?..
If it hangs - try looking into dmesg via SSH or into qemu logs or somewhere else, there should be something in dmesg.

Last edited by Duelist (2015-05-20 13:39:53)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

ghormoon

Member

Registered: 2015-05-20

Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

as for the range - yes, I've read a tutorial, where they said I have to disable cirrus in the device manager and reboot. but no change, there's down facing arrow (device disabled), but it still occupies the memory range
as for OVMF - I've dumped the rom some time ago and it was not OVMF, that's verified. No idea if there's any way to get OVMF one which will be compatible. Likely not.
This laptop doesn't even have the intel IGP wired in at all, just this Radeon as PCI device. it works in linux, so it is possible to pass in there are way more problems with laptops that actually can switch between those two.

for the hang - dmesg won't be much help, but If I make seabios output debug to console, it hangs at point where seabios tries to initialize the VGA. it looks like this when it boots http://pastebin.com/t7nRAdNj (with cirrus), and like this http://pastebin.com/hBQm9dme without cirrus (hangs).

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

as for the range - yes, I've read a tutorial, where they said I have to disable cirrus in the device manager and reboot. but no change, there's down facing arrow (device disabled), but it still occupies the memory range
as for OVMF - I've dumped the rom some time ago and it was not OVMF, that's verified. No idea if there's any way to get OVMF one which will be compatible. Likely not.
This laptop doesn't even have the intel IGP wired in at all, just this Radeon as PCI device. it works in linux, so it is possible to pass in there are way more problems with laptops that actually can switch between those two.

for the hang - dmesg won't be much help, but If I make seabios output debug to console, it hangs at point where seabios tries to initialize the VGA. it looks like this when it boots http://pastebin.com/t7nRAdNj (with cirrus), and like this http://pastebin.com/hBQm9dme without cirrus (hangs).

It hangs just after it tries to load VGA ROM. So i think it's integrated into some other weird place. Check your ROM.
Try extracting it via sysfs and attaching it as romfile=... option.

Last edited by Duelist (2015-05-20 16:35:03)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Schlunze

Member

Registered: 2013-10-03

Posts: 53

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

Basically, Schlunze in his posts request a .nsh EFI shell script to make OVMF use PT-d GPU from the very beginning, not relying on QXL(i didn't even think it's possible!).

The point is, now i have to switch onboard graphics or PCI-E graphics in main UEFI BIOS, then i can choose in the integratetd Bootloader(CSM) or gummiboot, wich OS shoud be started (Dual Boot).

It might be useful for isolating the "second" OS from scrach too, not fiddling around with wrong OS disk numbers and increases boot time^^.

The other way is to make fix settings for second OS in main BIOS, mount the EFI partion on two Systems linux nativ win virtualized. That´s a really bad idea i think. I don´t know if EFI stuff is ro by OSses.
And IGP graphics maybe not work during boot up linux in this case, thats a no go too. On the other hand is it possible to have two EFi partitons?

Maybe there is another way without startup.nsa?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

Basically, Schlunze in his posts request a .nsh EFI shell script to make OVMF use PT-d GPU from the very beginning, not relying on QXL(i didn't even think it's possible!).

The point is, now i have to switch onboard graphics or PCI-E graphics in main UEFI BIOS, then i can choose in the integratetd Bootloader(CSM) or gummiboot, wich OS shoud be started (Dual Boot).

It might be useful for isolating the "second" OS from scrach too, not fiddling around with wrong OS disk numbers and increases boot time^^.

The other way is to make fix settings for second OS in main BIOS, mount the EFI partion on two Systems linux nativ win virtualized. That´s a really bad idea i think. I don´t know if EFI stuff is ro by OSses.
And IGP graphics maybe not work during boot up linux in this case, thats a no go too. On the other hand is it possible to have two EFi partitons?

Maybe there is another way without startup.nsa?

Sorry, my mind is a bit blurry, so i'll answer only one question you've asked and i've understood:

On the other hand is it possible to have two EFi partitons?

Yes. Moreover, if i recall correctly, you can have two different boot loaders on the same EFI special partition(ESP).

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Schlunze

Member

Registered: 2013-10-03

Posts: 53

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks,

Yes. Moreover, if i recall correctly, you can have two different boot loaders on the same EFI special partition(ESP).

I look for a way to get rid of swiching primary graphic output in main bios for dual boot reasons.

Current standart configuration:
Bios setting for primary graphic output is onboard graphics.
Linux is bootet in CSM, i can use a win7 uefi vm with vga-passthrough. No need for vga arbitration due to qemu-ovmf on host. (Linux uefi not tested right now, maybe the EFI fat fs gets corrupt if its mountet on host and vm)

Dual Boot Windows7 UEFI:
Bios setting for primary graphic output is onboard graphics.
Windows is startet with standart vga-graphic, Intel driver is not installed right now. Pcie card shows up in windows device manger say everything is fine. But I can´ t use the nvidia card in this case.
For me it looks like a Windows VGA arbitration problem.

Dual Boot Windows7 UEFI: (most desired dual boot usecase)
Bios setting for primary graphic output is PCI-Express.
Pcie card is fully useable, the integrated graphic is turned off, not useable in this case.

I don´t know a solution for this switching problem.

Maybe a startup.nsh script to manage the inital switching depending on bootloader entry. Not possible?
Maybe boot Linux CSM with pcie as primary graphic output in main bios. Primary graphics is in pci-stub for qemu - i dont know what happens in this case but i will try it.

Thank you in advance again.

Last edited by Schlunze (2015-05-20 21:29:00)

paperinick

Member

Registered: 2015-04-14

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Weeeeeeird. I have libvirt and kraxel-org-based jenkins-git images, and it works fine there.

Can you please share that image somewhere? Thanks!!

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks,

Yes. Moreover, if i recall correctly, you can have two different boot loaders on the same EFI special partition(ESP).

I look for a way to get rid of swiching primary graphic output in main bios for dual boot reasons.

...snip...

Dual Boot Windows7 UEFI: (most desired dual boot usecase)
Bios setting for primary graphic output is PCI-Express.
Pcie card is fully useable, the integrated graphic is turned off, not useable in this case.

I don´t know a solution for this switching problem.

Maybe a startup.nsh script to manage the inital switching depending on bootloader entry. Not possible?
Maybe boot Linux CSM with pcie as primary graphic output in main bios. Primary graphics is in pci-stub for qemu - i dont know what happens in this case but i will try it.

Thank you in advance again.

Whaaat...
I didn't understand a thing again. Dual Boot - is a term of having two separate Operating Systems installed simultaneously. not running them. So you boot up your machine, choose, for example, linux, and use it. To get into windows, you shutdown your linux system, start the PC again and choose windows in your bootloader(usually grub). So dual booting doesn't involve virtualization at all.

If you want to pass-through a primary card into the VM - you just do it, considering you're using OVMF. No need to think about arbitration. Linux host boot loader(UEFI or legacy bios) doesn't matter at all.
So, you select pci-e graphics as your main GPU in your firmware setup(the "bios"), boot up bare metal windows and use it. Then you reboot to your linux host with a second GPU as host output, usually it is intel integrated GPU and having that "primary" discrete card drivers blacklisted and the device itself bound to pci-stub, futher ensuring that it's not used by anything in the host apart from vfio-pci.
Then, when the card is bound to vfio-pci, your start up your VM and live happily. That's how, i think, you want to live with your system.

@paperinick:

[user@crossfire ~]$ dnf info edk2.git-ovmf-x64.noarch
Using metadata from Sun May 17 15:15:34 2015 (4 days, 6:49:58 hours old)
Installed Packages
Name        : edk2.git-ovmf-x64
Arch        : noarch
Epoch       : 0
Version     : 0
Release     : 20150518.b1004.g54ae9c0
Size        : 11 M
Repo        : @System
From repo   : qemu-firmware-jenkins
Summary     : Open Virtual Machine Firmware
URL         : http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=EDK2
License     : BSD License (no advertising) with restrictions on use and redistribution
Description : EFI Development Kit II
            : Open Virtual Machine Firmware
            : 64bit version

I was too lazy to fix the paths, so i just did tar -cvz /usr/share/edk2.git/ovmf-x64/*.fd -f ovmf.tar.gz...

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

corna

Member

Registered: 2015-05-13

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

A typical GPU ROM is 64 kilobytes, UEFI-enabled one is 128k. That's on AMD. On Nvidia cards there may be differences, try opening the downloaded files and your own VBIOS in hex editor and look. But actually, have you even bothered asking the manufacturer?
Anyway, i think both files should work, the one you've extracted and the one you've downloaded.

Where does it outputs? Well, depends on the number of GPUs you're assigning: when using windows7+OVMF, you'll have to use QXL GPU additionally. Originally OVMF did output it's text stuff(and tianocore logo) only to QXL, now it mirrors it to the assigned card. Then windows boot logo comes up and renders itself only on QXL, or, if there's no QXL present, nowhere. That's why i've warned you - there was some occasions where people thought that their VM hung, but it was loading silently with no output. It's only hanging if it eats 100% of one vCPU.

Hi
I've installed Windows in UEFI mode and I'm trying to make it work with OVMF
When I have qxl activated it prints error 12, but if I remove it I don't have any output from the discrete card. I tried with the 3 UEFI-enabled ROMS found on techpowerup and with my Zotac ROM, but the result is always the same
How can I retrieve more debug informations?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

A typical GPU ROM is 64 kilobytes, UEFI-enabled one is 128k. That's on AMD. On Nvidia cards there may be differences, try opening the downloaded files and your own VBIOS in hex editor and look. But actually, have you even bothered asking the manufacturer?
Anyway, i think both files should work, the one you've extracted and the one you've downloaded.

Where does it outputs? Well, depends on the number of GPUs you're assigning: when using windows7+OVMF, you'll have to use QXL GPU additionally. Originally OVMF did output it's text stuff(and tianocore logo) only to QXL, now it mirrors it to the assigned card. Then windows boot logo comes up and renders itself only on QXL, or, if there's no QXL present, nowhere. That's why i've warned you - there was some occasions where people thought that their VM hung, but it was loading silently with no output. It's only hanging if it eats 100% of one vCPU.

Hi
I've installed Windows in UEFI mode and I'm trying to make it work with OVMF
When I have qxl activated it prints error 12, but if I remove it I don't have any output from the discrete card. I tried with the 3 UEFI-enabled ROMS found on techpowerup and with my Zotac ROM, but the result is always the same
How can I retrieve more debug informations?

1. What windows do you use? Win 7? Have you followed the instructions from OVMF whitepaper? Win 8? Win 8 won't need QXL at all.
2. ISA debug console, like

-debugcon file:debug.log -global isa-debugcon.iobase=0x402 

should produce debug.log with OVMF logging.
Have you tried installing windows with QXL and then installing the drivers despite seeing code 12? Maybe QXL will fall off(windows will disable it) and the GPU will fire up nicely(like it happened on my setup).

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.