Arch Linux

kaeptnb

Member

Registered: 2013-09-04

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The patch is an early implementation, the chunks that apply are no longer relevant.  What really confuses me is the implication that qemu.git works on 3.11 with this patch.  The problem is that the user API is different, which leads me to suspect that no bus reset is happening when you say it works.

Yes qemu.git works perfectly with 3.11(.6) AND the necessary kernel patches. I can restart the guest as often as i like.
Is there something i can test to confirm your suspicion?

Greetings

adr

Member

Registered: 2014-01-27

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Then I recompiled the kernel using patch found here https://lkml.org/lkml/2014/1/20/453 instead of the override_for_missing_acs_capabilities.patch provided in linux-mainline from first post, the result is better:

### Group 8 ###
    00:1c.4 PCI bridge: Intel Corporation 82801 PCI Bridge (rev c4)
    03:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03)
    04:00.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]
    04:01.0 Multimedia controller: Philips Semiconductors SAA7130 Video Broadcast Decoder (rev 01)

Do these devices work on the host?  Google "ASM1083 interrupt" and note the widespread problems people have with this PCIe-to-PCI bridge.  I have one on a non-VT-d system and I can't even get a NIC to work under it with a bare-metal OS.  It also does not provide a PCIe capability, in direct violation of the PCIe spec which makes us hiccup trying to figure out the topology, which is why it gets assigned to the same group as the root port.  You probably see a WARN_ON during boot because of this.  I've written off the PCI slot on my system with this bridge, IMHO it's junk.

The sound card (04:00.0) works on the host, I didn't notice any issue with it for the moment. The DVB tuner (04:01.0) is seen by the host as I have an adapter0 in /dev/dvb. However I did not manage to tune any channel yet, but this may be due to my antenna or poor reception.

It's entirely possible that one or both of the devices is not PCI2.3 compliant in supporting INTx disable and they share an interrupt.  I posted a script in the thread you reference below, does it say whether these devices support INTx disable?  What does lspci report for the interrupt line and what else is using it?

Using script:

#!/bin/sh
# Usage $0 <PCI device>, ex: 9:00.0

INTX=$(( 0x400 ))
ORIG=$(( 0x$(setpci -s $1 4.w) ))

if [ $(( $INTX & $ORIG )) -ne 0 ]; then
	echo "INTx disable supported and enabled on $1"
	exit 0
fi

NEW=$(printf %04x $(( $INTX | $ORIG )))
setpci -s $1 4.w=$NEW
NEW=$(( 0x$(setpci -s $1 4.w) ))

if [ $(( $INTX & $NEW )) -ne 0 ]; then
	echo "INTx disable support available on $1"
else
	echo "INTx disable support NOT available on $1"
fi

NEW=$(printf %04x $ORIG)
setpci -s $1 4.w=$NEW

Results:

# ./testINT.sh 0000:04:00.0
INTx disable support NOT available on 0000:04:00.0
# ./testINT.sh 0000:04:01.0
INTx disable support NOT available on 0000:04:01.0
# ./testINT.sh 0000:03:00.0
INTx disable support available on 0000:03:00.0
# ./testINT.sh 0000:06:00.0
INTx disable support available on 0000:06:00.0
# cat /sys/module/vfio_pci/parameters/nointxmask
N
# lspci -vvv
(...)
04:00.0 Multimedia audio controller: C-Media Electronics Inc CMI8788 [Oxygen HD Audio]
	Subsystem: ASUSTeK Computer Inc. Virtuoso 100 (Xonar D1)
	Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32 (500ns min, 6000ns max)
	Interrupt: pin A routed to IRQ 16
	Region 0: I/O ports at d000 [size=256]
	Capabilities: [c0] Power Management version 2
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Kernel driver in use: snd_virtuoso
	Kernel modules: snd_virtuoso

04:01.0 Multimedia controller: Philips Semiconductors SAA7130 Video Broadcast Decoder (rev 01)
	Subsystem: LeadTek Research Inc. WinFast DTV1000S
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 32 (21000ns min, 8000ns max)
	Interrupt: pin A routed to IRQ 17
	Region 0: Memory at f7d00000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [40] Power Management version 1
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
	Kernel driver in use: saa7134
	Kernel modules: saa7134
(...)

You have the PCI soundcard in use by the host and the DVB card assigned to a guest?

No, not at all. Currently I have both the PCI soundcard and the DVB card in use by the host and I passthrough integrated soundcard "00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)" to the guest. If I reach a hardware limit with the share of interrupts for external soundcard/dvb tuner, an alternative for me would be to stream TV from the host to the guest (as far as I understand installing Mythtv on the host could do that for example). Assuming I can make the DVB tuner work properly, otherwise I can always buy another one...

SATA controller with BD (device 06:00.0):
This is working when I first start the VM, I can access the Blu-Ray drive from my guest. But I have issues when I try to reboot the VM: sometimes the VM is frozen with message "SeaBIOS (version rel-1.7.4-27-...)" during about 2 minutes, and then it will startup normally (I can see the drive in guest after reboot). Sometimes it is definitely frozen with the same message, and the only solution is to shutdown the VM manually.

Others have reported similar problems with this device (note that it's also made by ASMedia...)  How much memory are you configuring for the VM?  The device is trying to do reads and writes to an address just shy of 7G, which may be off the end of guest memory.  Someone reported a similar fault with a Via USB3 card and replaced it with a Renesas controller and reported success.

I have 12GB memory in my host, and I assign only 4GB to my guest. I will try to assign 8GB to see if it is better.

While the faults above are the device trying to do DMA to regions it shouldn't, this indicates QEMU trying to create an IOMMU mapping that it shouldn't.  Nothing in QEMU lives at such a high address.  If it's reproducible I could probably send you a patch to print out a little more info about where that region is coming from.

It is reproductible pretty easily (happens about 50% of the time when I reboot the guest). I will check if this is reproductible with 8GB memory assigned to the guest.

Thanks a lot for your help!

EDIT:
Still the same issue with 8GB:
- 1st boot OK
- 2min freeze during first reboot => [ 5044.282211] dmar: DMAR:[DMA Write] Request device [06:00.0] fault addr 2affde000
                                                          DMAR:[fault reason 05] PTE Write access is not set
- final freeze during second reboot => [ 5221.745854] intel_iommu_map: iommu width (39) is not sufficient for the mapped address (3ffc0001000)

If I stop and restart QEMU at this point I have a new 2min freeze during 1st boot with same fault addr 2affde000

Last edited by adr (2014-01-28 22:40:57)

M4L

Member

Registered: 2014-01-29

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi There

I managed to get my system working with KVM and managed to pass through my HD 7950 and Creative Sound Recon3D to my VMs!

Few problems I get:
I'm just having and wondering if anyone else is. For some apparent reason I can't install Catalyst on my Windows VM.
I got the driver working and have no issues with running games or 3D rendering. It's just handy to have if I ever want to try and crossfire another HD 7950 passed through to my VM.

Anyone else have this problem?

Also I seem to have an issue passing through my USB 3.0 controller. Getting a problem with the device not being properly binded to VFIO. (I'll print the output shortly, just in my Windows VM fixing up a few things)

Finally I decided to not really use my on board Marvell controllers as they seem to not work with IOMMU on. I used this (http://www.spinics.net/lists/linux-pci/msg28094.html) patch to get my 9172 controller working, but the other 9220 was kicking up a fuss. I've got it switched off in the BIOS.
Anyone got a work around for getting it working? At the moment I'm using a fairly old Dell SAS 5/iR controller to just hook up my storage drives and that works alright, but it isn't as fast as the Marvell controllers.

These are my specs of my machine for those interested:
CPU: i7 3960X
Motherboard: ASRock Extreme 9
GPUs: GT 220 for my Arch Host and AMD (XFX DD) HD 7950 for my Arch and Windows VMs
SSD: Samsung 840 Pro 256GB, using LVM and have a logical volume for host and each VM
Sound Card: Creative Recon3D (I have no onboard audio and this replaced dodgy sound Creative Gameblaster card which came with motherboard)
SAS controller: Dell SAS 5/iR (It's an LSI 1068 controller I believe)
HDDs: 4 x 500GB 2.5" drives hooked up to my Intel SATA 2 controller and 4 x 1TB drives hooked up to my SAS card.

Last edited by M4L (2014-01-29 03:22:53)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Odd... I'm not sure if it's only with the Intel Root Port patch, but I get the following when I try to unload vfio-pci while a few devices are bound to it:

[  572.443297] kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
[  572.447996] BUG: unable to handle kernel paging request at ffffea000f9fe800
[  572.450999] IP: [<ffffea000f9fe800>] 0xffffea000f9fe800
[  572.453592] PGD 47e5f3067 PUD 47e5f2067 PMD 800000047bc001e3 
[  572.456074] Oops: 0011 [#1] SMP 
[  572.457579] Modules linked in: snd_virtuoso(F) snd_oxygen_lib(F) snd_mpu401_uart(F) xt_multiport(F) xt_conntrack(F) ipt_REJECT(F) sb_edac(F) snd_hda_codec_hdmi(F) snd_hda_codec_realtek(F) ip6table_filter(F) ip6_tables(F) xt_CHECKSUM(F) iptable_mangle(F) ipt_MASQUERADE(F) iptable_nat(F) nf_conntrack_ipv4(F) nf_defrag_ipv4(F) nf_nat_ipv4(F) nf_nat(F) nf_conntrack(F) xt_tcpudp(F) bridge(F) stp(F) llc(F) iptable_filter(F) ip_tables(F) x_tables(F) autofs4(F) deflate(F) ctr(F) twofish_generic(F) twofish_avx_x86_64(F) twofish_x86_64_3way(F) twofish_x86_64(F) twofish_common(F) camellia_generic(F) camellia_aesni_avx2(F) camellia_aesni_avx_x86_64(F) camellia_x86_64(F) serpent_avx2(F) serpent_avx_x86_64(F) serpent_sse2_x86_64(F) xts(F) serpent_generic(F) blowfish_generic(F) blowfish_x86_64(F) blowfish_common(F) cast5_avx_x86_64(F) cast5_generic(F) cast_common(F) des_generic(F) cmac(F) xcbc(F) rmd160(F) crypto_null(F) af_key(F) xfrm_algo(F) parport_pc(F) ppdev(F) bnep(F) rfcomm(F) bluetooth(F) joydev(F) iTCO_wdt(F) iTCO_vendor_support(F) dm_multipath(F) scsi_dh(F) x86_pkg_temp_thermal(F) intel_powerclamp(F) kvm_intel(F) kvm(F) crct10dif_pclmul(F) crc32_pclmul(F) hid_logitech_dj(F) ghash_clmulni_intel(F) aesni_intel(F) aes_x86_64(F) snd_hda_intel(F) lrw(F) gf128mul(F) snd_hda_codec(F) glue_helper(F) ablk_helper(F) snd_hwdep(F) cryptd(F) snd_seq_midi(F) i915(F) snd_seq_midi_event(F) snd_pcm(F) snd_rawmidi(F) shpchp(F) snd_seq(F) snd_seq_device(F) snd_page_alloc(F) snd_timer(F) drm_kms_helper(F) lpc_ich(F) drm(F) snd(F) soundcore(F) i2c_algo_bit(F) video(F) mac_hid(F) vfio_pci(F-) vfio_iommu_type1(F) vfio(F) w83627ehf(F) hwmon_vid(F) coretemp(F) msr(F) cpuid(F) jc42(F) nfsd(F) binfmt_misc(F) auth_rpcgss(F) nfs_acl(F) lp(F) nfs(F) parport(F) lockd(F) sunrpc(F) fscache(F) nls_iso8859_1(F) zfs(POF) zunicode(POF) zavl(POF) zcommon(POF) znvpair(POF) spl(OF) btrfs(F) xor(F) hid_generic(F) raid6_pq(F) libcrc32c(F) usbhid(F) hid(F) firewire_ohci(F) firewire_core(F) crc_itu_t(F) ahci(F) e1000e(F) libahci(F) ptp(F) pps_core(F) microcode(F) [last unloaded: snd_mpu401_uart]
[  572.547493] CPU: 7 PID: 6420 Comm: modprobe Tainted: PF          O 3.13.0-acs #1
[  572.551166] Hardware name: Supermicro X10SAT/X10SAT, BIOS 1.0 08/27/2013
[  572.555237] task: ffff88040aa98000 ti: ffff88038e808000 task.ti: ffff88038e808000
[  572.555268] RIP: 0010:[<ffffea000f9fe800>]  [<ffffea000f9fe800>] 0xffffea000f9fe800
[  572.555268] RSP: 0018:ffff88038e809d90  EFLAGS: 00010246
[  572.555270] RAX: 0000000000000088 RBX: ffff880467e63098 RCX: 0000000000000000
[  572.555270] RDX: ffff880467f7eb48 RSI: 0000000000320030 RDI: ffff880467e63098
[  572.555271] RBP: ffff88038e809db8 R08: 0000000000000246 R09: 0000000000000000
[  572.555271] R10: ffffea0011711700 R11: ffffffffa04d9127 R12: ffff880467e63140
[  572.555271] R13: ffffea000f9fe800 R14: 0000000000000000 R15: 0000000000000000
[  572.555272] FS:  00007f72a2402740(0000) GS:ffff88047ebc0000(0000) knlGS:0000000000000000
[  572.555273] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  572.555273] CR2: ffffea000f9fe800 CR3: 000000036782c000 CR4: 00000000001407e0
[  572.555274] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  572.555274] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[  572.555274] Stack:
[  572.555275]  ffffffff814aabf6 ffff88038e809df8 0000000000000004 ffff880467e63098
[  572.555276]  0000000000000004 ffff88038e809de8 ffffffff814ab68d 0000000000000004
[  572.555277]  ffff880467e63098 ffff880467e63140 0000000000000246 ffff88038e809e18
[  572.555278] Call Trace:
[  572.555281]  [<ffffffff814aabf6>] ? __rpm_callback+0x36/0xc0
[  572.555282]  [<ffffffff814ab68d>] rpm_idle+0x1bd/0x2b0
[  572.555283]  [<ffffffff814ab7dd>] __pm_runtime_idle+0x5d/0x80
[  572.555286]  [<ffffffff813b95a4>] pci_device_remove+0x74/0xb0
[  572.555290]  [<ffffffff8149f09f>] __device_release_driver+0x7f/0xf0
[  572.555292]  [<ffffffff8149fa38>] driver_detach+0xb8/0xc0
[  572.555296]  [<ffffffff8149ecd5>] bus_remove_driver+0x55/0xd0
[  572.555298]  [<ffffffff814a00ac>] driver_unregister+0x2c/0x50
[  572.555300]  [<ffffffff813b8059>] pci_unregister_driver+0x29/0x90
[  572.555304]  [<ffffffffa04de061>] vfio_pci_cleanup+0x10/0x1c [vfio_pci]
[  572.555307]  [<ffffffff810dd812>] SyS_delete_module+0x172/0x220
[  572.555309]  [<ffffffff817389bf>] tracesys+0xe1/0xe6
[  572.555318] Code: 00 00 00 40 00 40 00 01 00 00 00 00 01 10 00 00 00 ad de 00 02 20 00 00 00 ad de 00 3b 00 6e 04 88 ff ff 00 00 00 00 00 00 00 00 <80> 00 00 00 00 ff ff 02 00 00 00 00 00 00 00 00 00 00 00 00 00 
[  572.555320] RIP  [<ffffea000f9fe800>] 0xffffea000f9fe800
[  572.555320]  RSP <ffff88038e809d90>
[  572.555321] CR2: ffffea000f9fe800
[  572.561468] ---[ end trace 0e53739ef2e93df6 ]---

After this oops, lspci -nnv hangs forever.

mostlyharmless

Member

Registered: 2014-01-16

Posts: 72

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So I tried intel_iommu=pt instead of intel_iommu=on, and I still get dmesg with
WARNING: CPU: 2 PID: 1884 at drivers/vfio/vfio_iommu_type1.c:344 vfio_unmap_unpin.isra.7+0x11e/0x120()
[  166.948480] Modules linked in: ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables vhost_net vhost macvtap macvlan tun autofs4 bridge stp llc ipv6 lp parport fuse snd_hda_codec_hdmi btusb bluetooth snd_hda_codec_realtek x86_pkg_temp_thermal intel_powerclamp coretemp crc32_pclmul crc32c_intel ghash_clmulni_intel i2c_dev eeepc_wmi asus_wmi sparse_keymap rfkill video mxm_wmi radeon joydev evdev snd_hda_intel snd_hda_codec ttm snd_hwdep drm_kms_helper snd_pcm drm psmouse microcode snd_page_alloc snd_timer serio_raw e1000e snd i2c_i801 agpgart i2c_algo_bit mei_me i2c_core soundcore lpc_ich ptp mei pps_core ehci_pci processor thermal_sys hwmon wmi button loop msdos nls_iso8859_1 nls_cp437 uhci_hcd ehci_hcd pci_stub vfat fat ext2 ext4 jbd2 mbcache hid_generic usbhid hid xhci_hcd usb_storage
[  166.948505] CPU: 2 PID: 1884 Comm: qemu-system-x86 Tainted: G        W    3.13.0 #1
[  166.948506] Hardware name: System manufacturer System Product Name/RAMPAGE IV EXTREME, BIOS 9502 11/07/2013
[  166.948507]  0000000000000009 ffff8800aa9a7d10 ffffffff81644f6e 0000000000000000
[  166.948508]  ffff8800aa9a7d48 ffffffff810be16d 0000000000000000 0000000000000000
[  166.948510]  ffff8800aa9a7e30 0000000000200000 ffff88081ad6aba0 ffff8800aa9a7d58
[  166.948511] Call Trace:
[  166.948513]  [<ffffffff81644f6e>] dump_stack+0x45/0x56
[  166.948514]  [<ffffffff810be16d>] warn_slowpath_common+0x7d/0xa0
[  166.948516]  [<ffffffff810be24a>] warn_slowpath_null+0x1a/0x20
[  166.948518]  [<ffffffff814a433e>] vfio_unmap_unpin.isra.7+0x11e/0x120
[  166.948520]  [<ffffffff814a43a2>] vfio_remove_dma_overlap+0x62/0x250
[  166.948522]  [<ffffffff814a5120>] vfio_iommu_type1_ioctl+0x230/0x290
[  166.948524]  [<ffffffff814a2377>] vfio_fops_unl_ioctl+0x77/0x340
[  166.948525]  [<ffffffff811e8540>] do_vfs_ioctl+0x2e0/0x4c0
[  166.948527]  [<ffffffff811d57ac>] ? vfs_write+0x15c/0x1f0
[  166.948529]  [<ffffffff811d55de>] ? vfs_read+0xee/0x160
[  166.948530]  [<ffffffff811e87a1>] SyS_ioctl+0x81/0xa0
[  166.948532]  [<ffffffff8164e5ed>] system_call_fastpath+0x1a/0x1f
[  166.948532] ---[ end trace c90bd9cca026e365 ]---
[  168.059412] dmar: DRHD: handling fault status reg 102
[  168.059417] dmar: DMAR:[DMA Read] Request device [00:1d.0] fault addr ee000
[  168.059417] DMAR:[fault reason 06] PTE Read access is not set
[  177.056309] vfio-pci 0000:03:00.0: irq 105 for MSI/MSI-X

but everything seems to work OK.  Haven't tried reinstalling ati-catalyst, I'm going to work on getting 3D acceleration working with the radeon drivers instead for now.  I've tried running under 3.13.0 as well as 3.12.1 and everything seems to work fine without any issues of reboots so far. my qemu-git was downloaded on Dec 5th 2013.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So I tried intel_iommu=pt instead of intel_iommu=on, and I still get dmesg with
WARNING: CPU: 2 PID: 1884 at drivers/vfio/vfio_iommu_type1.c:344 vfio_unmap_unpin.isra.7+0x11e/0x120()

Last time you posted this backtrace I suggested trying this patch https://git.kernel.org/cgit/linux/kerne … fe8702b098

The suggestion remains...

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

mostlyharmless

Member

Registered: 2014-01-16

Posts: 72

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah, I haven't gotten to that yet, but I'll give it a try, thanks.   
[EDIT]
So I patched 3.13.0 and now have

[8314.585792] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=none
[ 8340.979240] vfio-pci 0000:03:00.0: enabling device (0000 -> 0003)
[ 8340.979363] vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
[ 8341.003257] vfio-pci 0000:03:00.1: enabling device (0000 -> 0002)
[ 8341.114083] vfio_cap_init: 0000:00:1d.0 hiding cap 0xa
[ 8344.137317] dmar: DRHD: handling fault status reg 2
[ 8344.137323] dmar: DMAR:[DMA Read] Request device [00:1d.0] fault addr ee000
[ 8344.137323] DMAR:[fault reason 06] PTE Read access is not set
[ 8350.516005] kvm: zapping shadow pages for mmio generation wraparound
[ 8355.387485] vfio-pci 0000:03:00.0: irq 105 for MSI/MSI-X

and on reboot

[ 8503.680819] vfio_cap_init: 0000:00:1d.0 hiding cap 0xa
[ 8506.570990] dmar: DRHD: handling fault status reg 102
[ 8506.570995] dmar: DMAR:[DMA Read] Request device [00:1d.0] fault addr afffd000
[ 8506.570995] DMAR:[fault reason 06] PTE Read access is not set
[ 8512.688516] kvm: zapping shadow pages for mmio generation wraparound
[ 8519.895012] vfio-pci 0000:03:00.0: irq 105 for MSI/MSI-X

So I guess that's better than spamming dmesg with tons of repeat page faults.  Thanks again.

Last edited by mostlyharmless (2014-01-30 05:24:03)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've now received my new video card, and I'd really like to get EFI mode working on it.  I've gotten the thing to show a solid cursor on the Radeon, but no further than that.  With the ubuntu seabios package, I get no debug output -- same with OVMF-X64-r11337-alpha.

My command line:

qemu-system-x86_64 -bios /usr/share/ovmf/OVMF.fd -M q35 -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios -smp 4,sockets=1,cores=4,threads=1 -readconfig q35-chipset.cfg -device vfio-pci,host=01:00.0,bus=ich9-pcie-port-1 -device vfio-pci,host=09:00.0,bus=ich9-pcie-port-3 -device vfio-pci,host=0c:00.0,bus=ich9-pcie-port-4 -device vfio-pci,host=07:04.0,bus=ich9-pci-bridge -net none -rtc base=localtime -vga none -device sga

If I use an older or newer OVMF, I get:

SecCoreStartupWithStack(0xFFFCC000, 0x818000)
Register PPI Notify: DCD0BE23-9586-40F4-B643-06522CED4EDE
Install PPI: 8C8CE578-8A3D-4F1C-9935-896185C32DD3
Install PPI: 5473C07A-3DCB-4DCA-BD6F-1E9689E7349A
The 0th FV start address is 0x00000820000, size is 0x000E0000, handle is 0x820000
Register PPI Notify: 49EDB1C1-BF21-4761-BB12-EB0031AABB39
Register PPI Notify: EA7CA24B-DED5-4DAD-A389-BF827E8F9B38
Install PPI: B9E0ABFE-5979-4914-977F-6DEE78C278A6
Install PPI: DBE23AA9-A345-4B97-85B6-B226F1617389
Loading PEIM at 0x00000836D80 EntryPoint=0x00000836FE0 PcdPeim.efi
Install PPI: 06E81C58-4AD7-44BC-8390-F10265F72480
Install PPI: 01F34D25-4DE2-23AD-3FF3-36353FF323F1
Loading PEIM at 0x0000083FBA0 EntryPoint=0x0000083FE00 StatusCodePei.efi
Install PPI: 229832D3-7A30-4B36-B827-F40CB7D45436
Loading PEIM at 0x000008457A0 EntryPoint=0x00000845A00 PlatformPei.efi
Platform PEIM Loaded
CMOS:
00: 11 00 44 00 19 00 05 30 01 14 26 02 10 80 00 00
10: 40 00 00 00 07 80 02 FF FF 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: FF FF 20 00 00 07 00 20 30 00 00 00 00 12 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Install PPI: 7408D748-FC8C-4EE6-9288-C4BEC092A410
PeiInstallPeiMemory MemoryBegin 0x4000000, MemoryLength 0x4000000
MemDetect called
Reserved variable store memory: 0x7FD0000; size: 128kb
Platform PEI Firmware Volume Initialization
Install PPI: 49EDB1C1-BF21-4761-BB12-EB0031AABB39
Notify: PPI Guid: 49EDB1C1-BF21-4761-BB12-EB0031AABB39, Peim notify entry point: 8237F7
The 1th FV start address is 0x00000900000, size is 0x00700000, handle is 0x900000
Temp Stack : BaseAddress=0x814000 Length=0x4000
Temp Heap  : BaseAddress=0x810000 Length=0x870
Total temporary memory:    32768 bytes.
  temporary memory stack ever used: 16384 bytes.
  temporary memory heap used:       2160 bytes.
Old Stack size 16384, New stack size 131072
Heap Offset = 0x0 Stack Offset = 0x3808000
Stack Hob: BaseAddress=0x4000000 Length=0x20000
TemporaryRamMigration(0x810000, 0x401C000, 0x8000)
Loading PEIM at 0x00007FB8000 EntryPoint=0x00007FB8260 PeiCore.efi
Reinstall PPI: 8C8CE578-8A3D-4F1C-9935-896185C32DD3
Reinstall PPI: 5473C07A-3DCB-4DCA-BD6F-1E9689E7349A
Reinstall PPI: B9E0ABFE-5979-4914-977F-6DEE78C278A6
Install PPI: F894643D-C449-42D1-8EA8-85BDD8C65BDE
Loading PEIM at 0x00007FAE000 EntryPoint=0x00007FAE260 DxeIpl.efi
Install PPI: 0AE8CE5D-E448-4437-A8D7-EBF5F194F731
Install PPI: 1A36E4E7-FAB6-476A-8E75-695A0576FDD7
DXE IPL Entry
Loading PEIM at 0x00007F75000 EntryPoint=0x00007F75260 DxeCore.efi
Loading DXE CORE at 0x00007F75000 EntryPoint=0x00007F75260
Install PPI: 605EA650-C65C-42E1-BA80-91A52AB618C6
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7FA0788
HOBLIST address in DXE = 0x7D2A018
Memory Allocation 0x00000006 0x7FD0000 - 0x7FFFFFF
Memory Allocation 0x00000004 0x820000 - 0x8FFFFF
Memory Allocation 0x00000004 0x900000 - 0xFFFFFF
Memory Allocation 0x00000004 0x7F55000 - 0x7F74FFF
Memory Allocation 0x00000004 0x7FCF000 - 0x7FCFFFF
Memory Allocation 0x00000004 0x7FB8000 - 0x7FCEFFF
Memory Allocation 0x00000004 0x7FAE000 - 0x7FB7FFF
Memory Allocation 0x00000004 0x7F75000 - 0x7FADFFF
Memory Allocation 0x00000003 0x7F75000 - 0x7FADFFF
Memory Allocation 0x00000004 0x7F55000 - 0x7F74FFF
Memory Allocation 0x00000004 0x7F13000 - 0x7F54FFF
Memory Allocation 0x00000004 0x4000000 - 0x401FFFF
FV Hob            0x900000 - 0xFFFFFF
InstallProtocolInterface: D8117CFE-94A6-11D4-9A3A-0090273FC14D 7FA0220
InstallProtocolInterface: 8F644FA9-E850-4DB1-9CE2-0B44698E8DA4 7D26B30
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 7D27698
InstallProtocolInterface: 220E73B6-6BDB-4413-8405-B974B108619A 7D26830
InstallProtocolInterface: EE4E5898-3914-4259-9D6E-DC7BD79403CF 7FA0448
Loading driver 9B680FCE-AD6B-4F3A-B60B-F59899003443
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3E40
Loading driver at 0x00007E9D000 EntryPoint=0x00007E9D2AF DevicePathDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7CB4B18
InstallProtocolInterface: 0379BE4E-D706-437D-B037-EDB82FB772A4 7EA8FE0
InstallProtocolInterface: 8B843E20-8132-4852-90CC-551A4E4A7F1C 7EA9020
InstallProtocolInterface: 05C99A21-C70F-4AD2-8A5F-35DF3343F51E 7EA9030
Loading driver 80CF7257-87AB-47F9-A3FE-D50B76D89541
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3C40
Loading driver at 0x00007E91000 EntryPoint=0x00007E912AF PcdDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7CB4598
InstallProtocolInterface: 11B34006-D85B-4D0A-A290-D5A571310EF7 7E994E0
InstallProtocolInterface: 13A3F0F6-264A-3EF0-F2E0-DEC512342F34 7E995E0
Loading driver 733CBAC2-B23F-4B92-BC8E-FB01CE5907B7
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3A40
Loading driver at 0x00007ED4000 EntryPoint=0x00007ED42AF FvbServicesRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7CB4098
QEMU Flash: Attempting flash detection at FFE00010
QemuFlashDetected => FD behaves as ROM
QemuFlashDetected => No
QEMU flash was not detected. Writable FVB is not being installed.
Error: Image at 00007ED4000 start failed: Write Protected
Loading driver FEDE0A1B-BCA2-4A9F-BB2B-D9FD7DEC2E9F
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3A40
Loading driver at 0x00007ED4000 EntryPoint=0x00007ED42AF StatusCodeRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7CB4118
InstallProtocolInterface: D2B2B828-0826-48A7-B3DF-983C006024F0 7EDA430
Loading driver B601F8C4-43B7-4784-95B1-F4226CB40CEE
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3840
Loading driver at 0x00007ECD000 EntryPoint=0x00007ECD2AF RuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C99C18
InstallProtocolInterface: B7DFB4E1-052F-449F-87BE-9818FC91B733 7ED14A0
Loading driver F80697E9-7FD6-4665-8646-88E33EF71DFC
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3640
Loading driver at 0x00007DEF000 EntryPoint=0x00007DEF2AF SecurityStubDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C99898
InstallProtocolInterface: 94AB2F58-1438-4EF1-9152-18941A3A0E68 7E611B8
InstallProtocolInterface: A46423E3-4617-49F1-B9FF-D1BFA9115839 7E611B0
Loading driver 13AC6DD0-73D0-11D4-B06B-00AA00BD6DE7
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3440
Loading driver at 0x00007DE3000 EntryPoint=0x00007DE32AF EbcDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C99518
InstallProtocolInterface: 13AC6DD1-73D0-11D4-B06B-00AA00BD6DE7 7C99418
InstallProtocolInterface: 2755590C-6F3C-42FA-9EA4-A3BA543CDA25 7C99318
InstallProtocolInterface: AAEACCFD-F27B-4C17-B610-75CA1F2DFB52 7C99218
Loading driver 79CA4208-BBA1-4A9A-8456-E1E66A81484E
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3240
Loading driver at 0x00007DDE000 EntryPoint=0x00007DDE2AF Legacy8259.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C97E98
InstallProtocolInterface: 38321DBA-4FE0-4E17-8AEC-413055EAEDC1 7DE0FC0
Loading driver A19B1FE7-C1BC-49F8-875F-54A5D542443F
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7CB3040
Loading driver at 0x00007DD8000 EntryPoint=0x00007DD82AF CpuIo2Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C97A98
InstallProtocolInterface: AD61F191-AE5F-4C0E-B9FA-E869D288C64F 7DDB300
Loading driver 1A1E4886-9517-440E-9FDE-3BE44CEE2136
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96E40
Loading driver at 0x00007DC9000 EntryPoint=0x00007DC92AF CpuDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C97698
InstallProtocolInterface: 26BACCB1-6F42-11D4-BCE7-0080C73C8881 7DD2D40
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
  Flushing GCD
Loading driver 6F0198AA-1F1D-426D-AE3E-39AB633FCC28
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96C40
Loading driver at 0x00007EC7000 EntryPoint=0x00007EC72AF KbcReset.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C97398
InstallProtocolInterface: 27CFAC88-46CC-11D4-9A38-0090273FC14D 0
Loading driver C8339973-A563-4561-B858-D8476F9DEFC4
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96A40
Loading driver at 0x00007DC4000 EntryPoint=0x00007DC42AF Metronome.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C97018
InstallProtocolInterface: 26BACCB2-6F42-11D4-BCE7-0080C73C8881 7DC7070
Loading driver 22DC2B60-FE40-42AC-B01F-3AB1FAD9AAD8
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96840
Loading driver at 0x00007EBF000 EntryPoint=0x00007EBF2AF EmuVariableFvbRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C1FC98
EMU Variable FVB Started
EMU Variable FVB: Using pre-reserved block at 7FD0000
EMU Variable FVB: Basic FV headers were invalid
Installing FVB for EMU Variable support
InstallProtocolInterface: 8F644FA9-E850-4DB1-9CE2-0B44698E8DA4 7EC3A40
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 7EC3A08
Loading driver 79E4A61C-ED73-4312-94FE-E3E7563362A9
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96640
Loading driver at 0x00007DBF000 EntryPoint=0x00007DBF2AF PrintDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C1F798
InstallProtocolInterface: F05976EF-83F1-4F3D-8619-F7595D41E538 7DC2040
Loading driver 348C4D62-BFBD-4882-9ECE-C80BB1C4783B
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96440
Loading driver at 0x00007D98000 EntryPoint=0x00007D982AF HiiDatabase.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7C1F398
InstallProtocolInterface: E9CA4775-8657-47FC-97E7-7ED65A084324 7DB8A48
InstallProtocolInterface: 0FD96974-23AA-4CDC-B9CB-98D17750322A 7DB8A90
InstallProtocolInterface: EF9FC172-A1B2-4693-B327-6D32FC416042 7DB8AB8
InstallProtocolInterface: 587E72D7-CC50-4F79-8209-CA291FC1A10F 7DB8B10
InstallProtocolInterface: 31A6406A-6BDF-4E46-B2A2-EBAA89C40920 7DB8A68
Loading driver 96B5C032-DF4C-4B6E-8232-438DCF448D0E
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96240
Loading driver at 0x00007D92000 EntryPoint=0x00007D922AF NullMemoryTestDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FED98
InstallProtocolInterface: 309DE7F1-7F5E-4ACE-B49C-531BE5AA95EF 7D95180
Loading driver F9D88642-0737-49BC-81B5-6889CD57D9EA
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7C96040
Loading driver at 0x00007D8A000 EntryPoint=0x00007D8A2AF SmbiosDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FE998
InstallProtocolInterface: 03583FF6-CB36-4940-947E-B9B39F4AFAF7 7D8F390
Loading driver 9622E42C-8E38-4A08-9E8F-54F784652F6B
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FDE40
Loading driver at 0x00007D7B000 EntryPoint=0x00007D7B2AF AcpiTableDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FE598
InstallProtocolInterface: FFE06BDD-6107-46A6-7BB2-5A9C7EC5275C 7D2B9A8
Loading driver A210F973-229D-4F4D-AA37-9895E6C9EABA
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FDC40
Loading driver at 0x00007D75000 EntryPoint=0x00007D752AF DpcDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FE198
InstallProtocolInterface: 480F8AE9-0C46-4AA9-BC89-DB9FBA619806 7D78020
Loading driver F2765DEC-6B41-11D5-8E71-00902707B35E
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FDA40
Loading driver at 0x00007D70000 EntryPoint=0x00007D702AF Timer.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FBD98
InstallProtocolInterface: 26BACCB3-6F42-11D4-BCE7-0080C73C8881 7D72F40
Loading driver 2383608E-C6D0-4E3E-858D-45DFAC3543D5
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FD840
Loading driver at 0x00007D65000 EntryPoint=0x00007D652AF PciHostBridge.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FBA18
InstallProtocolInterface: CF8034BE-6768-4D8B-B739-7CCE683A9FBE 73FC548
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 7D6CB20
InstallProtocolInterface: 2F707EBB-4A1A-11D4-9A38-0090273FC14D 73CAE08
Loading driver FE5CEA76-4F72-49E8-986F-2CD899DFFE5D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FD640
Loading driver at 0x00007D59000 EntryPoint=0x00007D592AF FaultTolerantWriteDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FB498
Ftw: Remaining work space size - FE0
Ftw: Work block header check error
Ftw: Work block header check error
Ftw: Both are invalid, init workspace
Ftw: start to reclaim work space
Ftw: reclaim work space successfully
InstallProtocolInterface: 3EBD9E82-2C78-4DE6-9786-8D4BFCB7C881 73B7028
Loading driver EBF342FE-B1D3-4EF8-957C-8048606FF671
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FD440
Loading driver at 0x00007D31000 EntryPoint=0x00007D312AF SetupBrowser.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 73FB098
InstallProtocolInterface: B9D4C360-BCFB-4F9B-9298-53C136982258 7D50F90
InstallProtocolInterface: A770C357-B693-4E6D-A6CF-D21C728E550B 7D50FC0
InstallProtocolInterface: 1F73B18D-4630-43C1-A1DE-6F80855D7DA4 7D50FA0
Loading driver 4110465D-5FF3-4F4B-B580-24ED0D06747A
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FD240
Loading driver at 0x00006C25000 EntryPoint=0x00006C252AF SmbiosPlatformDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 721DA18
Select Item: 0x0
FW CFG Signature: 0x554D4551
Select Item: 0x1
FW CFG Revision: 0x1
QemuFwCfg interface is supported.
Select Item: 0x8001
Select Item: 0x2
Loading driver 49970331-E3FA-4637-9ABC-3B7868676970
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 73FD040
Loading driver at 0x00006C1D000 EntryPoint=0x00006C1D2AF AcpiPlatform.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 721D618
Select Item: 0x0
FW CFG Signature: 0x554D4551
Select Item: 0x1
FW CFG Revision: 0x1
QemuFwCfg interface is supported.
Select Item: 0x5
Select Item: 0x19
ACPI using S3/S4 defaults
ACPI PciWindow32: Base=0x08000000 End=0xFEEFFFFF Length=0xF6F00000
ACPI PciWindow64: Base=0x00000000 End=0x00000000 Length=0x00000000
Loading driver 2226F30F-3D5B-402D-9936-A97184EB4516
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721AE40
Loading driver at 0x00006B6A000 EntryPoint=0x00006B6A2AF VariableAuthRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71F9E98
InstallProtocolInterface: CD3D0A05-9E24-437C-A891-1EE053DB7638 6BE6278
InstallProtocolInterface: 1E5668E2-8481-11D4-BCF1-0080C73C8881 0
Variable PK does not exist.
Variable SetupMode is 1
Variable SecureBoot is 0
Variable SecureBootEnable is 0
Variable CustomMode is 0
Variable VendorKeys is 1
InstallProtocolInterface: 6441F818-6362-4E44-B570-7DBA31DD2453 0
Loading driver F099D67F-71AE-4C36-B2A3-DCEB0EB2B7D8
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721AC40
Loading driver at 0x00006AB7000 EntryPoint=0x00006AB72AF WatchdogTimer.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71F9998
InstallProtocolInterface: 665E3FF5-46CC-11D4-9A38-0090273FC14D 6AB9E10
Loading driver E660EA85-058E-4B55-A54B-F02F83A24707
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721AA40
Loading driver at 0x00006A77000 EntryPoint=0x00006A772AF DisplayEngine.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71F9598
InstallProtocolInterface: 9BBE29E9-FDA1-41EC-AD52-452213742D2E 6A8EC50
Loading driver 378D7B65-8DA9-4773-B6E4-A47826A833E1
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71F7E40
Loading driver at 0x00007EB5000 EntryPoint=0x00007EB52AF PcRtc.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71E1918
InstallProtocolInterface: 27CFAC87-46CC-11D4-9A38-0090273FC14D 0
Loading driver 68E2BC17-198D-41F1-8213-065E3A2A7C12
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721A840
Loading driver at 0x00006A5D000 EntryPoint=0x00006A5D2AF SecureBootConfigDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71E1518
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6A6E390
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 71E1420
InstallProtocolInterface: 68E2BC17-198D-41F1-8213-065E3A2A7C12 71E1418
Loading driver AD608272-D07F-4964-801E-7BD3B7888652
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DEE40
Loading driver at 0x00007EAF000 EntryPoint=0x00007EAF2AF MonotonicCounterRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DDA98
InstallProtocolInterface: 1DA97072-BDDC-4B30-99F1-72A0B56FFF2A 0
Loading driver 42857F0A-13F2-4B21-8A23-53D3F714B840
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721A640
Loading driver at 0x00006A52000 EntryPoint=0x00006A522AF CapsuleRuntimeDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DD718
InstallProtocolInterface: 5053697E-2CBC-4819-90D9-0580DEEE5754 0
Loading driver FC5C7020-1A48-4198-9BE2-EAD5ABC8CF2F
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721A440
Loading driver at 0x00006A03000 EntryPoint=0x00006A032AF BdsDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DD318
Select Item: 0x0
FW CFG Signature: 0x554D4551
Select Item: 0x1
FW CFG Revision: 0x1
QemuFwCfg interface is supported.
InstallProtocolInterface: 665E3FF6-46CC-11D4-9A38-0090273FC14D 6A40018
Loading driver 93B80004-9FB3-11D4-9A3A-0090273FC14D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 721A040
Loading driver at 0x000069E4000 EntryPoint=0x000069E42AF PciBusDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DBD18
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69FDB20
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69FDDE0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69FDE00
InstallProtocolInterface: 19CB87AB-2CB9-4665-8360-DDCF6054F79D 69FDB78
Loading driver 33CB97AF-6C33-4C42-986B-07581FA366D4
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DAE40
Loading driver at 0x000069DD000 EntryPoint=0x000069DD2AF BlockMmioToBlockIoDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DB718
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69E0D60
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69E1060
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69E1080
Loading driver 83DD3B39-7CAF-4FAC-A542-E050B767E3A7
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DAC40
Loading driver at 0x000069D6000 EntryPoint=0x000069D62AF VirtioPciDeviceDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DB498
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69DA520
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69DA580
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69DA5A0
Loading driver 11D92DFB-3CA9-4F93-BA2E-4780ED3E03B5
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DAA40
Loading driver at 0x000069CE000 EntryPoint=0x000069CE2AF VirtioBlkDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71DB198
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69D2DE0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69D2E40
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69D2E60
Loading driver FAB5D4F4-83C0-4AAF-8480-442D11DF6CEA
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DA840
Loading driver at 0x000069C6000 EntryPoint=0x000069C62AF VirtioScsiDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D8E98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69CB2A0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69CB300
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69CB320
Loading driver 51CCF399-4FDF-4E55-A45B-E123F84D456A
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DA640
Loading driver at 0x000069BE000 EntryPoint=0x000069BE2AF ConPlatformDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D8B98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69C2AA0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69C2C80
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69C2CA0
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69C2AE0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69C2C80
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69C2CA0
Loading driver 408EDCEC-CF6D-477C-A5A8-B4844E3DE281
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DA440
Loading driver at 0x000069AF000 EntryPoint=0x000069AF2AF ConSplitterDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D8718
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69BA880
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69BAE30
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69BAE50
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69BA940
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69BAE70
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69BAE90
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69BA980
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69BAEB0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69BAED0
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69BA8C0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69BAEF0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69BAF10
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69BA900
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69BAF30
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69BAF50
InstallProtocolInterface: 387477C1-69C7-11D2-8E39-00A0C969723B 69BA490
InstallProtocolInterface: DD9E7534-7762-4698-8C14-F58517A625AA 69BA4C0
InstallProtocolInterface: 31878C87-0B75-11D5-9A4F-0090273FC14D 69BA518
InstallProtocolInterface: 8D59D32B-C655-4AE9-9B15-F25904992A43 69BA570
InstallProtocolInterface: 387477C2-69C7-11D2-8E39-00A0C969723B 69BA650
InstallProtocolInterface: 387477C2-69C7-11D2-8E39-00A0C969723B 69BA770
Loading driver CCCB0C28-4B24-11D5-9A5A-0090273FC14D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DA240
Loading driver at 0x000069A3000 EntryPoint=0x000069A32AF GraphicsConsoleDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D4998
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69AA580
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69AA840
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69AA860
Loading driver 9E863906-A40F-4875-977F-5B93FF237FC6
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71DA040
Loading driver at 0x00006993000 EntryPoint=0x000069932AF TerminalDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D4498
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 699E1E0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 699E6F0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 699E710
Loading driver 6B38F7B4-AD98-40E9-9093-ACA2B5A253C4
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5E40
Loading driver at 0x00006989000 EntryPoint=0x000069892AF DiskIoDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71D4198
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 698F680
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 698FBE0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 698FC00
Loading driver 1FA1F39E-FEFF-4AAE-BD7B-38A070A3B609
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5C40
Loading driver at 0x0000697D000 EntryPoint=0x0000697D2AF PartitionDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71B4E98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6985360
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6985510
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6985530
Loading driver CD3BAFB6-50FB-4FE8-8E4E-AB74D2C1A600
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5A40
Loading driver at 0x00006977000 EntryPoint=0x000069772AF EnglishDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71B4B98
InstallProtocolInterface: 1D85CD7F-F43D-11D2-9A0C-0090273FC14D 697A160
InstallProtocolInterface: A4C751FC-23AE-4C3E-92E9-4964CF63F349 697A1A0
Loading driver 0167CCC4-D0F7-4F21-A3EF-9E64B7CDCE8B
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5840
Loading driver at 0x0000696D000 EntryPoint=0x0000696D2AF ScsiBus.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71B4798
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69734C0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69735F0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6973610
Loading driver 0A66E322-3740-4CCE-AD62-BD172CECCA35
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5640
Loading driver at 0x00006960000 EntryPoint=0x000069602AF ScsiDisk.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71B4498
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6969840
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6969C50
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6969C70
Loading driver 69FD8E47-A161-4550-B01A-5594CEB2B2B2
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5440
Loading driver at 0x0000694A000 EntryPoint=0x0000694A2AF IdeBusDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 71B4198
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 695B5E0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 695BA10
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 695BA30
InstallProtocolInterface: 0784924F-E296-11D4-9A49-0090273FC14D 695B8E0
InstallProtocolInterface: 4D330321-025F-4AAC-90D8-5ED900173B63 695B8F0
Loading driver 99549F44-49BB-4820-B9D2-901329412D67
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5240
Loading driver at 0x00006944000 EntryPoint=0x000069442AF IdeController.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7182D98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6947920
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6947B00
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6947B20
Loading driver 38A0EC22-FBE7-4911-8BC1-176E0D6C1DBD
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 71B5040
Loading driver at 0x0000693E000 EntryPoint=0x0000693E2AF IsaAcpi.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7182A98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6941C40
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6941F10
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6941EF0
Loading driver 240612B5-A063-11D4-9A3A-0090273FC14D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181E40
Loading driver at 0x00006935000 EntryPoint=0x000069352AF IsaBusDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7182798
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 693B260
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 693B380
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 693B3A0
Loading driver 93B80003-9FB3-11D4-9A3A-0090273FC14D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181C40
Loading driver at 0x0000692A000 EntryPoint=0x0000692A2AF IsaSerialDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7182498
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6931E60
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6932180
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69321A0
Loading driver 3DC82376-637B-40A6-A8FC-A565417F2C38
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181A40
Loading driver at 0x0000691D000 EntryPoint=0x0000691D2AF Ps2KeyboardDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7182198
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 69259A0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6925BA0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6925BC0
Loading driver 0ABD8284-6DA3-4616-971A-83A5148067BA
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181840
Loading driver at 0x00006911000 EntryPoint=0x000069112AF IsaFloppyDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717FE98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6918DA0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6918F90
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6918FB0
Loading driver 961578FE-B6B7-44C3-AF35-6BC705CD2B1F
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181640
Loading driver at 0x0000690A000 EntryPoint=0x0000690A570 
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717FB98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 690A418
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 690A448
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 690A460
Loading driver A2F436EA-A127-4EF8-957C-8048606FF670
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181440
Loading driver at 0x000068FB000 EntryPoint=0x000068FB2AF SnpDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717F898
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6905980
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 69061D0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 69061F0
Loading driver 025BBFC7-E6A9-4B8B-82AD-6815A1AEAF4A
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181240
Loading driver at 0x000068E6000 EntryPoint=0x000068E62AF MnpDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717F598
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 68F4FA0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68F6480
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68F64A0
Loading driver E4F61863-FE2C-4B56-A8F4-08519BC439DF
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 7181040
Loading driver at 0x000068D8000 EntryPoint=0x000068D82AF VlanConfigDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717F298
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 68E1480
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68E1AA0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68E1AC0
Loading driver 529D3F93-E8E9-4E73-B1E1-BDF6A9D50113
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717CE40
Loading driver at 0x000068CA000 EntryPoint=0x000068CA2AF ArpDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717BF98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 68D3240
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68D3580
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68D35A0
Loading driver 94734718-0BBC-47FB-96A5-EE7A5AE6A2AD
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717CC40
Loading driver at 0x000068B5000 EntryPoint=0x000068B52AF Dhcp4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717BC98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 68C4700
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68C4E90
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68C4EB0
Loading driver 26841BDE-920A-4E7A-9FBE-637F477143A6
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717CA40
Loading driver at 0x000068A5000 EntryPoint=0x000068A52AF Ip4ConfigDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717B998
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 68B0780
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68B0E50
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68B0E70
Loading driver 9FB1A1F3-3B71-4324-B39A-745CBB015FFF
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717C840
Loading driver at 0x0000688A000 EntryPoint=0x0000688A2AF Ip4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717B698
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 689F460
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 689FDF0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 689FE10
Loading driver DC3641B8-2FA8-4ED3-BC1F-F9962A03454B
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717C640
Loading driver at 0x00006877000 EntryPoint=0x000068772AF Mtftp4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717B398
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6884F40
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68850D0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68850F0
Loading driver 6D6963AB-906D-4A65-A7CA-BD40E5D6AF4D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717C440
Loading driver at 0x00006855000 EntryPoint=0x000068552AF Tcp4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 717B098
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 686E8C0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 686FE50
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 686FE70
Loading driver 6D6963AB-906D-4A65-A7CA-BD40E5D6AF2B
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717C240
Loading driver at 0x00006841000 EntryPoint=0x000068412AF Udp4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7178D98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6850240
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 68503C0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 68503E0
Loading driver 3B1DEAB5-C75D-442E-9238-8E2FFB62B0BB
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 717C040
Loading driver at 0x0000682E000 EntryPoint=0x0000682E2AF UefiPxe4BcDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7178A98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 683C980
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 683CAB0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 683CAD0
Loading driver 4579B72D-7EC4-4DD4-8486-083C86B182A7
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713EE40
Loading driver at 0x0000680E000 EntryPoint=0x0000680E2AF IScsi4Dxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7178798
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 6827500
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 6827630
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6827650
InstallProtocolInterface: 59324945-EC44-4C0D-B1CD-9DB139DF070C 68282F0
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6827720
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 713DC28
Loading driver A92CDB4B-82F1-4E0B-A516-8A655D371524
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713EC40
Loading driver at 0x00006804000 EntryPoint=0x000068042AF VirtioNetDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 713CE98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 680A920
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 680A7E0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 680A800
Loading driver 2FB92EFA-2EE0-4BAE-9EB6-7464125E1EF7
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713EA40
Loading driver at 0x000067F5000 EntryPoint=0x000067F52AF UhciDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 713CB98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67FFD40
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67FFFF0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 6800010
Loading driver BDFE430E-8F2A-4DB0-9991-6F856594777E
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713E840
Loading driver at 0x000067D1000 EntryPoint=0x000067D12AF EhciDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 713C898
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67DDCA0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67DF2B0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 67DF2D0
Loading driver 240612B7-A063-11D4-9A3A-0090273FC14D
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713E640
Loading driver at 0x000067BF000 EntryPoint=0x000067BF2AF UsbBusDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 713C598
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67CC1E0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67CCC30
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 67CCC50
Loading driver 2D2E62CF-9ECF-43B7-8219-94E7FC713DFE
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713E440
Loading driver at 0x000067E7000 EntryPoint=0x000067E72AF UsbKbDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 713C298
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67F03E0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67F1200
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 67F1220
Loading driver 9FB4B4A7-42C0-4BCD-8540-9BCC6711F83E
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713E240
Loading driver at 0x000067B3000 EntryPoint=0x000067B32AF UsbMassStorageDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7139F98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67BADC0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67BBAF0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 67BBB10
Loading driver E3752948-B9A1-4770-90C4-DF41C38986BE
InstallProtocolInterface: 5B1B31A1-9562-11D2-8E3F-00A0C969723B 713E040
Loading driver at 0x000067A3000 EntryPoint=0x000067A32AF QemuVideoDxe.efi
InstallProtocolInterface: BC62157E-3E33-4FEC-9920-2D3B36D750DF 7139C98
InstallProtocolInterface: 18A031AB-B443-4D1A-A5C0-0C09261E9F71 67AA9E0
InstallProtocolInterface: 107A772C-D5E1-11D4-9A46-0090273FC14D 67AAFF0
InstallProtocolInterface: 6A7A5CFF-E8D9-4F70-BADA-75AB3025CE14 67AB010
InstallProtocolInterface: 5C198761-16A8-4E69-972C-89D67954F81D 67AAC80
[BdsDxe] Locate Variable Lock protocol - Success
[Variable] Lock: 8BE4DF61-93CA-11D2-AA0D-00E098032B8C:PlatformLangCodes
[Variable] Lock: 8BE4DF61-93CA-11D2-AA0D-00E098032B8C:LangCodes
[Variable] Lock: 8BE4DF61-93CA-11D2-AA0D-00E098032B8C:BootOptionSupport
[Variable] Lock: 8BE4DF61-93CA-11D2-AA0D-00E098032B8C:HwErrRecSupport
[Variable] Lock: 8BE4DF61-93CA-11D2-AA0D-00E098032B8C:OsIndicationsSupported
Variable Driver Auto Update Lang, Lang:eng, PlatformLang:en
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6A41DF0
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 6A41DA8
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6A41E10
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 6A41DC0
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6A42640
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 6A42678
InstallProtocolInterface: 09576E91-6D3F-11D2-8E39-00A0C969723B 6A41A60
InstallProtocolInterface: 330D4706-F2A0-4E4F-A369-B66FA8D54385 6A41A40
PlatformBdsInit
Registered NotifyDevPath Event
PlatformBdsPolicyBehavior
PCI Bus First Scanning
PciBus: Discovered PCI @ [00|00|00]

PciBus: Discovered PCI @ [00|1A|00]
   BAR[4]: Type =   Io32; Alignment = 0x1F;	Length = 0x20;	Offset = 0x20

PciBus: Discovered PCI @ [00|1A|01]
   BAR[4]: Type =   Io32; Alignment = 0x1F;	Length = 0x20;	Offset = 0x20

PciBus: Discovered PCI @ [00|1A|02]
   BAR[4]: Type =   Io32; Alignment = 0x1F;	Length = 0x20;	Offset = 0x20

PciBus: Discovered PCI @ [00|1A|07]
   BAR[0]: Type =  Mem32; Alignment = 0xFFF;	Length = 0x1000;	Offset = 0x10

PciBus: Discovered PCI @ [00|1B|00]
   BAR[0]: Type =  Mem32; Alignment = 0x3FFF;	Length = 0x4000;	Offset = 0x10

PciBus: Discovered PPB @ [00|1C|00]
ASSERT /var/lib/jenkins/jobs/edk2/workspace/rpmbuild/rpm/BUILD/edk2.git/MdeModulePkg/Library/DxeReportStatusCodeLib/ReportStatusCodeLib.c(371): DevicePath != ((void *) 0)

EDIT: Also, when booting without EFI mode, I'm still getting this all the time:

qemu-system-x86_64: hw/pci/pci.c:232: pcibus_reset: Assertion `bus->irq_count[i] == 0' failed

Last edited by DanaGoyette (2014-01-31 04:45:59)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've given up on EFI+Q35, for now.

Now, EFI with i440fx, on the other hand, works for me.
While wrestling with Q35+EFI, I chopped a couple of assertions out of the OVMF source; no idea if they're part of why i440fx is now working.

My new video card has a GraphicsOutputProtocol ROM, so it works as the boot video device, despite not having x-vga=on or the Intel VGA Arbiter patches applied!
I'm using the Intel graphics as the primary video, and until I set my BIOS not to load the x16 slot's ROM, even a native-booted Windows would fail to load the AMD driver.

My passed-through XHCI controller doesn't work at boot-time; I had to copy the xHCI driver from the "Clover" boot loader and add it as a driver option.

Some quirks: the biggest is that rebooting seems buggy or unreliable.  I sometimes have to entirely restart qemu because the reset hangs.

My command line now (with extra debugging enabled):

qemu-system-x86_64 -enable-kvm -bios /usr/share/ovmf/OVMF-new.fd -m 5G -boot order=dc -chardev stdio,id=seabios -device isa-debugcon,iobase=0x402,chardev=seabios -smp 4,sockets=1,cores=4,threads=1 -device vfio-pci,host=01:00.0 -device vfio-pci,host=09:00.0 -device vfio-pci,host=0c:00.0 -device vfio-pci,host=07:04.0 -usb -hda /dev/disk/by-id/ata-ST9500420AS_5VJDD6R9 -device usb-tablet -net none -rtc base=localtime -vga none -device sga

EDIT: I'll bet my issues with reboot are the same as the GPU reset issues mentioned above.  My host won't suspend (even without VMs running), so I can't just suspend to get the GPU alive again.

EDIT: I tried to run 3DMark in the guest, and the result was a hang, and a GPU rendered inoperative.  Looks like the device reset code is indeed broken.

EDIT: For once, my host actually suspended and resumed properly this time.  Result: GPU has revived.

Last edited by DanaGoyette (2014-02-02 07:38:24)

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

Thanks for this Thread, thats what i was looking for for years.

My System is:
Intel i5 4670
Asus H87-Plus  both confirmed VT-d support.
Host Graphics: Intel HD 4000
Guest Graphics: Geforce GFX560Ti

My Host is a fresh Arch install from last week.
My Guest should be Win8 or 8.1.

I installed all the Pakages from the OP in the first Post from 22-01-14

Using his Script with cpuset and so on. My Geforce is pci-stubbed and vfio-bind is successful to. But the card wont start. Monitor is in standby. When i use virt-manager with the card passedthrough i get "Code 43" in Windows 8.

lspci
01:00.0 VGA compatible controller: NVIDIA Corporation GF114 [GeForce GTX 560 Ti] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GF114 HDMI Audio Controller (rev a1)

ls -l /sys/bus/pci/drivers/vfio-pci/

lrwxrwxrwx 1 root root    0  2. Feb 15:08 0000:01:00.0 -> ../../../../devices/pci0000:00/0000:00:01.0/0000:01:00.0
lrwxrwxrwx 1 root root    0  2. Feb 15:08 0000:01:00.1 -> ../../../../devices/pci0000:00/0000:00:01.0/0000:01:00.1
--w------- 1 root root 4096  2. Feb 15:08 bind
lrwxrwxrwx 1 root root    0  2. Feb 15:08 module -> ../../../../module/vfio_pci
--w------- 1 root root 4096  2. Feb 15:07 new_id
--w------- 1 root root 4096  2. Feb 15:08 remove_id
--w------- 1 root root 4096  2. Feb 14:35 uevent
--w------- 1 root root 4096  2. Feb 15:07 unbind

My Gummiboot Commandline is as follows:

root=/dev/sda4 rw init=/usr/lib/systemd/systemd add_efi_memmap pci-stub.ids=10de:1200,10de:0e0c intel_iommu=on,igfx_on vfio_iommu_type1.allow_unsafe_interrupts=1

Any Ideas what i'am missing?

Thanks in advance

Tec

EDIT: here is my qemu start script, i removed the pci root, in testing but with no succsess

#!/bin/sh

CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

QEMU=/usr/bin/qemu-system-x86_64
CSET=/usr/bin/cset

VCPUS="3" #must be > 0 < cpu core number
CPU="Haswell"
MEM="8192"
#MEM_PATH="/dev/hugepages"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-#pci,netdev=hostnet0,id=net0"
USB_DEVICES=""
PCI_DEVICES=""
GPU="01:00.0"
GPU_SND="01:00.1"
SND=true
SND_DRIVER_OPTS="QEMU_ALSA_DAC_BUFFER_SIZE=512 QEMU_ALSA_DAC_PERIOD_SIZE=170 QEMU_AUDIO_DRV=alsa "
VBIOS="/home/alex/GTX560Ti.rom"
CDROM="/home/alex/win8cd.iso"
HDD="/home/alex/windows.img"
QEMU_ARGS=" -M q35 -enable-kvm -monitor stdio"
EXTRA_ARGS="-boot menu=on"
	
if [ -n "$GPU" ]; then
	EXTRA_ARGS+=" -vga none -nographic"
	DEV+=""
	if [ -n "$VBIOS" ]; then
		DEV+=" -device vfio-pci,host=$GPU,x-vga=on,romfile=$VBIOS"
	else
		DEV+=" -device vfio-pci,host=$GPU,x-vga=on"
	fi
  
	if [ -n "$GPU_SND" ]; then
		DEV+=" -device vfio-pci,host=$GPU_SND"
	fi
fi

if [ -n "$PCI_DEVICES" ]; then
	for d in $PCI_DEVICES; do
		DEV+=" -device vfio-pci,host=$d"
	done
fi 


if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
	DEV+="  -device ahci,bus=pcie.0,id=ahci"
	if [ -n "$HDD" ]; then
		STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
	fi
	if [ -n "$CDROM" ]; then
		STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
	fi
fi

if [ -n "$CPU" ]; then
	CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
	CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
	BIOS=" -bios $BIOS"
fi

if $SND ; then
	SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
	export $SND_DRIVER_OPTS
else
	SND=""
fi

if [ -n "$MEM" ]; then
	MEMORY+="-m $MEM"
	if [ -n "$MEM_PATH" ]; then
		MEMORY+=" -mem-prealloc -mem-path $MEM_PATH"
	fi
fi

if [ -n "$USB_DEVICES" ]; then
	for u in $USB_DEVICES; do
		USB_DEV+=" -usbdevice $u"
	done
fi

if [ $VCPUS -lt $CPU_CORE_NR ]; then
	$CSET shield -c $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1))
	set -m

	$QEMU $BIOS $CPU $MEMORY $QEMU_ARGS $DEV $USB_DEV $NET $SND $STORAGE  $EXTRA_ARGS &

	PID=$!
	(
	while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
		if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
			i=$((CPU_CORE_NR-VCPUS))
			for t in `ls /proc/$PID/task/`; do
				if [ $i -ne $((CPU_CORE_NR)) ]; then
					if [ $t -ne $PID ];then
						$CSET shield --shield --pid $t
						taskset -pc $i $t
						let i++
					fi
				fi
			done
			break
		fi
	done

	) &

	fg %1
	wait
	$CSET shield --reset
else
	echo "VCPU count $VCPUS is greater or equal than HOST CPU core number $CPU_CORE_NR"
fi
echo "All Done!!"

Last edited by tecnologic (2014-02-02 14:33:46)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Argh, I just got stuck in a loop of Windows 8.1 "Working on updates", then getting stuck upon reboot because the bus reset doesn't work.  Every single time, I have to suspend and resume the host before the guest will boot.

EDIT: I'm now resorting to running qemu with no passthrough devices -- maybe then it will finish the updates.

Last edited by DanaGoyette (2014-02-02 19:55:18)

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi some pages before there was n win application postet to detach the PCI device, maybe this helb on reboot of the guest system.

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It finally finished, after what must've been at least 10 reboots.

The problem with the Group Policy method is that I don't know if it will help when the system is rebooting before reaching the desktop.

I also just got the following messages (for the first time ever) upon re-adding the passthrough:

[ 2027.315800] intel_iommu_map: iommu width (39) is not sufficient for the mapped address (c2000000c0001000)
qemu-system-x86_64: vfio_dma_map(0x7fe1a4de4e30, 0xc2000000c0000000, 0x10000000, 0x7fe028000000) = -14 (Bad address)
[ 2029.066767] intel_iommu_map: iommu width (39) is not sufficient for the mapped address (c2000000c0001000)
qemu-system-x86_64: vfio_dma_map(0x7fe1a4de4e30, 0xc2000000c0000000, 0x10000000, 0x7fe028000000) = -14 (Bad address)

Last edited by DanaGoyette (2014-02-02 20:05:36)

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi!
I have an optimus laptop... Here somebody says passthrough isn't possible, but I'd like to give it a try anyway. Since passthrough works so well on my desktop, I just can't resist the urge to make all systems passthrough capable.

Laptop: Clevo w350st
CPU: i7-4800MQ (supports vt-d)
Chipset: HM87 (supports vt-d)
iGPU: Intel HD4600
dGPU: GeForce 765M
It goes without saying the main show stopper is the dGPU.

First, it has no direct video output. The plan is to use remote desktop solutions or streaming if passthrough will work.

Secondly, it's not a VGA compatible controller. lspci:
01:00.0 3D controller: NVIDIA Corporation GK106M [GeForce GTX 765M] (rev a1)
x-vga is out of question. Or is it?

Question for gurus. Is it possible to add VGA compatibility by pushing to qemu a vbios from a card that has such functions? Like rom=765MwithVBIOS.bin.
Also, how can I check whether the GPU supports UEFI GOP? Perhaps I could use it instead of vga?

If all that won't happen, maybe there's a chance it would work as a secondary GPU in the guest? Have there been any successful secondary passthroughs with NVIDIA cards?

I've tried two configurations (common parts are qemu-git from today, seabios supplied by qemu, machine q35-2.0, primary gpu cirrus, host kernel 3.13.1, no nvidia drivers on the host):
1. Connecting the GPU to pcie port as explained in the first post (sans x-vga) produces BSOD on boot "PROCESS1_INITIALIZATION_FAILED."
2. Letting virt-manager connect the GPU (-device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.2,addr=0x6) makes Win7 load, but the GPU is dead "The device cannot start (Code 10)." Interestingly, I found mysterious "NVIDIA Virtual Audio Device (WaveExtensible) (WDM)" which is connected to "Location Unknown" and is "working properly" according to Win7. Where did it come from? 765M has no HDMI audio 01:00.1.

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[…]When i use virt-manager with the card passedthrough i get "Code 43" in Windows 8.[…]

This error has popped up in this thread several times, but I think it was only with intel integrated graphics, which I don't use so I have no idea how to fix it, but there should be a solution to it somewhere around this thread.

---

i'm sorry for my poor english wirting skills…

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[…]When i use virt-manager with the card passedthrough i get "Code 43" in Windows 8.[…]

This error has popped up in this thread several times, but I think it was only with intel integrated graphics, which I don't use so I have no idea how to fix it, but there should be a solution to it somewhere around this thread.

Also try without my script, it wasnt ment for general use, it just happens to be the one i use

Last edited by nbhs (2014-02-02 23:12:14)

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all

@andy123: i searched the thread for Code 43 and found that i had to use the qemu nosnoop patch from "aw", i downloaded his last qemu-vfio tag an installed it but i got errors on qemu start, I'll post the errors tomorrow, cause i need to sleep due to an early meeting at work .

@nbhs: in my opinion i used your exact setup from the first post. But on the point of testing the qemu with graphics the display was black. What will bie the minimal qemu commandline?

qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

or without q35?

as next step i will examin the error Messages from qemu with "aw"'s latest tag.  According to the change logs there are many fixes made for Geforce.

Is it still useful to blacklist i915?

Greetings until tomorrow.

Tec

EDIT: i just started the commandline above an got the following dmesg output.

[ 4953.189292] ------------[ cut here ]------------
[ 4953.189299] WARNING: CPU: 0 PID: 2299 at kernel/jump_label.c:81 __static_key_slow_dec+0xa6/0xb0()
[ 4953.189300] jump label: negative count!
[ 4953.189300] Modules linked in: usbhid ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT xt_CHECKSUM iptable_mangle xt_tcpudp bridge stp llc fuse ip6table_filter ip6_tables iptable_filter ip_tables x_tables vfio_pci vfio_iommu_type1 vfio nls_cp437 vfat fat eeepc_wmi asus_wmi sparse_keymap rfkill iTCO_wdt iTCO_vendor_support x86_pkg_temp_thermal coretemp kvm_intel nls_utf8 kvm ntfs crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd microcode joydev pcspkr serio_raw i2c_i801 r8169 mii snd_hda_codec_realtek snd_hda_codec_hdmi fan thermal snd_hda_intel snd_hda_codec snd_hwdep snd_pcm wmi snd_page_alloc evdev snd_timer lpc_ich snd mei_me shpchp
[ 4953.189331]  mei processor soundcore ext4 crc16 mbcache jbd2 hid_generic hid sr_mod cdrom sd_mod ahci libahci libata ehci_pci xhci_hcd ehci_hcd scsi_mod usbcore usb_common pci_stub i915 video button i2c_algo_bit intel_agp intel_gtt drm_kms_helper drm i2c_core [last unloaded: usbhid]
[ 4953.189345] CPU: 0 PID: 2299 Comm: qemu-system-x86 Not tainted 3.12.9-2-ARCH #1
[ 4953.189346] Hardware name: ASUS All Series/H87-PLUS, BIOS 0805 01/03/2014
[ 4953.189347]  0000000000000009 ffff8803f3693a98 ffffffff814ed0d3 ffff8803f3693ae0
[ 4953.189349]  ffff8803f3693ad0 ffffffff81062aed ffffffffa07a5840 000000000000012c
[ 4953.189350]  ffffffffa07a5860 ffff8803f378c130 ffff88040f19a1e8 ffff8803f3693b30
[ 4953.189352] Call Trace:
[ 4953.189356]  [<ffffffff814ed0d3>] dump_stack+0x54/0x8d
[ 4953.189359]  [<ffffffff81062aed>] warn_slowpath_common+0x7d/0xa0
[ 4953.189362]  [<ffffffff81062b5c>] warn_slowpath_fmt+0x4c/0x50
[ 4953.189364]  [<ffffffff81088575>] ? lock_hrtimer_base.isra.20+0x25/0x50
[ 4953.189372]  [<ffffffffa0770360>] ? kvm_arch_vcpu_uninit+0x20/0x90 [kvm]
[ 4953.189374]  [<ffffffff811345b6>] __static_key_slow_dec+0xa6/0xb0
[ 4953.189375]  [<ffffffff811345f6>] static_key_slow_dec_deferred+0x16/0x20
[ 4953.189380]  [<ffffffffa0788a8c>] kvm_free_lapic+0x6c/0xa0 [kvm]
[ 4953.189384]  [<ffffffffa0770368>] kvm_arch_vcpu_uninit+0x28/0x90 [kvm]
[ 4953.189387]  [<ffffffffa0755c91>] kvm_vcpu_uninit+0x21/0x30 [kvm]
[ 4953.189390]  [<ffffffffa07df457>] vmx_free_vcpu+0x47/0x70 [kvm_intel]
[ 4953.189394]  [<ffffffffa076fa54>] kvm_arch_vcpu_free+0x44/0x50 [kvm]
[ 4953.189398]  [<ffffffffa0770592>] kvm_arch_destroy_vm+0xf2/0x1f0 [kvm]
[ 4953.189400]  [<ffffffff8108a2ed>] ? synchronize_srcu+0x1d/0x20
[ 4953.189404]  [<ffffffffa07584ee>] kvm_put_kvm+0x10e/0x1c0 [kvm]
[ 4953.189407]  [<ffffffffa07585d8>] kvm_vcpu_release+0x18/0x20 [kvm]
[ 4953.189409]  [<ffffffff811a62c1>] __fput+0xa1/0x230
[ 4953.189410]  [<ffffffff811a649e>] ____fput+0xe/0x10
[ 4953.189413]  [<ffffffff81081c1c>] task_work_run+0xbc/0xe0
[ 4953.189414]  [<ffffffff81064f8e>] do_exit+0x2be/0xab0
[ 4953.189416]  [<ffffffff810657ff>] do_group_exit+0x3f/0xa0
[ 4953.189418]  [<ffffffff810747ad>] get_signal_to_deliver+0x2ad/0x610
[ 4953.189420]  [<ffffffff810144d8>] do_signal+0x48/0x940
[ 4953.189422]  [<ffffffff81014e38>] do_notify_resume+0x68/0xa0
[ 4953.189424]  [<ffffffff814fbeaa>] int_signal+0x12/0x17
[ 4953.189425] ---[ end trace 2f6c48f77244ce60 ]---
[ 5091.775617] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[ 5093.831760] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5093.831774] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5093.831785] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5093.831842] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5093.831860] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5093.831877] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5096.565189] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt
[ 5096.565200] [drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

Whats this "Unclaimed register before interrupt" anyone an hint?

Last edited by tecnologic (2014-02-03 19:11:08)

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So i've done further testing.

First my Bios, i set igpu as primary, shared mem with PCIE GPU is off(caused problems with x starting).

when i now run a minimal qemu cmd

vfio-bind 0000:01:00.0 0000:01:00.1

qemu-system-x86_64 -enable-kvm -M q35 -net none -cpu host -boot menu=on -monitor stdio -debugcon file:/dev/stderr -global isa-debugcon.iobase=0x402 -bios /usr/share/qemu/bios.bin -m 5G -smp 2,sockets=1,cores=2,threads=1 -vga none -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on,romfile=GTX560Ti2.rom

i get plenty of those errors in dmesg:

[drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

and qemu says:

MP table addr=0x000f1ad0 MPC table addr=0x000f1ae0 size=208
SMBIOS ptr=0x000f1ab0 table=0x000f1930 size=381
Scan for VGA option rom
Running option rom at c000:0003
KVM internal error. Suberror: 1
emulation failure
EAX=ffff5151 EBX=00000000 ECX=00000000 EDX=00000300
ESI=00006d41 EDI=00006d5a EBP=00006d5a ESP=0000dd5c
EIP=0004eaef EFL=00010007 [-----PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 ffffffff 00809300
CS =5151 00051510 ffffffff 00809b00
SS =0000 00000000 ffffffff 00809300
DS =c000 000c0000 ffffffff 00809300
FS =0000 00000000 ffffffff 00809300
GS =0000 00000000 ffffffff 00809300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     000f6d38 00000037
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

so when dmesg say some thing of interrupt i thought of this "vfio_iommu_type1.allow_unsafe_interrupts=1" kernel option. But what does this option i found no documentation, only mailing list, which i do not right understand.

EDIT: i added "vfio_iommu_type1.allow_unsafe_interrupts=1" to my boot options, stil get Unclaimed register errors and qemu stops with Cpu register dum message.

EDIT2: nbhs mentions in the first post blacklisting the nvidia drivers, and blacklisting i915 was also mentioned. so i tryed this but lspic -k says, i915 is used. But reading futher i was sad that i915 and vga arbiter patches are included in 3.13 kernel build from nbhs. An this is the kernel i use actually.  Strange

00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Kernel driver in use: i915
	Kernel modules: i915



01:00.0 VGA compatible controller: NVIDIA Corporation GF114 [GeForce GTX 560 Ti] (rev a1)
	Subsystem: CardExpert Technology Device 0401
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau
01:00.1 Audio device: NVIDIA Corporation GF114 HDMI Audio Controller (rev a1)
	Subsystem: CardExpert Technology Device 0401
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

any chanches that using a second Geforce as host graphics could be more successful?

any ideas?

Thanks

Tec

Last edited by tecnologic (2014-02-03 23:48:54)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So i've done further testing.

First my Bios, i set igpu as primary, shared mem with PCIE GPU is off(caused problems with x starting).

when i now run a minimal qemu cmd

vfio-bind 0000:01:00.0 0000:01:00.1

qemu-system-x86_64 -enable-kvm -M q35 -net none -cpu host -boot menu=on -monitor stdio -debugcon file:/dev/stderr -global isa-debugcon.iobase=0x402 -bios /usr/share/qemu/bios.bin -m 5G -smp 2,sockets=1,cores=2,threads=1 -vga none -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on,romfile=GTX560Ti2.rom

i get plenty of those errors in dmesg:

[drm:intel_uncore_check_errors] *ERROR* Unclaimed register before interrupt

This means the VGA accesses of the Nvidia ROM are going to the iGPU and causing problems, which means you don't have the VGA arbiter patches.

and qemu says:

MP table addr=0x000f1ad0 MPC table addr=0x000f1ae0 size=208
SMBIOS ptr=0x000f1ab0 table=0x000f1930 size=381
Scan for VGA option rom
Running option rom at c000:0003
KVM internal error. Suberror: 1
emulation failure
EAX=ffff5151 EBX=00000000 ECX=00000000 EDX=00000300
ESI=00006d41 EDI=00006d5a EBP=00006d5a ESP=0000dd5c
EIP=0004eaef EFL=00010007 [-----PC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 ffffffff 00809300
CS =5151 00051510 ffffffff 00809b00
SS =0000 00000000 ffffffff 00809300
DS =c000 000c0000 ffffffff 00809300
FS =0000 00000000 ffffffff 00809300
GS =0000 00000000 ffffffff 00809300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     000f6d38 00000037
IDT=     00000000 000003ff
CR0=00000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000000
Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff

so when dmesg say some thing of interrupt i thought of this "vfio_iommu_type1.allow_unsafe_interrupts=1" kernel option. But what does this option i found no documentation, only mailing list, which i do not right understand.

allow_unsafe_interrupts is simply a switch that allows vfio to work when an interrupt remapper is not present.  Intel added interrupt remapping support in VT-d2 and among other things, it protects the kernel from MSI injection attacks by guests with an assigned device.  If you need it, you've already had to opt-in to the option, if you don't need it, adding the option isn't going to change anything.

Perhaps try setting the emulate_invalid_guest_state module option for kvm_intel to 0.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi aw,

This means the VGA accesses of the Nvidia ROM are going to the iGPU and causing problems, which means you don't have the VGA arbiter patches.

How can i check that? is there an cmd?

Perhaps try setting the emulate_invalid_guest_state module option for kvm_intel to 0.

done an restarting...

[root@Alex alex]# echo "options kvm_intel emulate_invalid_guest_state=0" >> /etc/modprobe.d/kvm-intel.conf
[root@Alex alex]# cat /etc/modprobe.d/kvm-intel.conf 
options kvm_intel emulate_invalid_guest_state=0

EDIT: Strange, same error as before adding the option, but i heard my graphicscard turn is fan on. so there is some thing going on. So i need to fix the arbiter. Can i pull the linux-vfio tag "vfio-v3.14-rc1" from your github to solve that, or should i use a dedicated patch an add it to an pkgbuild?

Tec

Last edited by tecnologic (2014-02-04 00:04:11)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi aw,

This means the VGA accesses of the Nvidia ROM are going to the iGPU and causing problems, which means you don't have the VGA arbiter patches.

How can i check that? is there an cmd?

No, but the driver message is often accompanied by seeing screen corruption on the iGPU display across the top few pixels of the screen (running in VT mode).

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

no screen corruption here, i got 2 screens in the intel card and the geforce on the digital of one of the screens so i can switch the inputs.

tecnologic

Member

Registered: 2014-02-02

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

11 days ago
vfio-v3.14-rc1 …

    3be3a07

6 days ago
vfio-pci-for-qemu-20140128.0 …

    8b6d140

seems a good combo, from your repos, are these supposed to run together?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

EDIT: Strange, same error as before adding the option, but i heard my graphicscard turn is fan on. so there is some thing going on. So i need to fix the arbiter. Can i pull the linux-vfio tag "vfio-v3.14-rc1" from your github to solve that, or should i use a dedicated patch an add it to an pkgbuild?

No, I don't carry the patches in my tree.  They've been upstream, caused DRI to be disabled in the host and got reverted.  We need to re-architect the VGA arbiter, I have some ideas but I haven't had time to work on it.  You need these patches:

http://git.kernel.org/cgit/linux/kernel … f81dea8a7d
http://git.kernel.org/cgit/linux/kernel … ba24389bee

They don't apply cleanly, but it should be pretty easy to figure out the conflict (just more stuff being added to the header since the patch).

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

11 days ago
vfio-v3.14-rc1 …

    3be3a07

6 days ago
vfio-pci-for-qemu-20140128.0 …

    8b6d140

seems a good combo, from your repos, are these supposed to run together?

They can, but you'll still need the patches in the previous post.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users