Arch Linux

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I am using couple of R9 270 assigned to windows 8.1 and that seems to work perfectly.
I am testing nvidia 220 that I have laying around from using it with mythtv I am getting error 43.
From reading adding this option will work around the error code

<domain type='kvm'...>
  <features>
    <kvm>
      <hidden state='on'/>
    </kvm>
  </features>
  ...
</domain>

Is this how I can get errror code 43 fixed ?
Did anyone tried to install windows 3.11 I only get black screen.

Last edited by tritron4 (2015-03-21 03:07:37)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did anyone tried to install windows 3.11 I only get black screen.

Please tell me you're joking.  It's not just hiding kvm, it's disabling all hyper-v extensions.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

The_Moves

Member

Registered: 2015-01-06

Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

3.11? Three dot One One

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Please tell me you're joking.

I just tried FreeDOS . It is working . (Needed to use "NVFlash" to flash my GPU's firmware through DOS and it worked) .

Last edited by Denso (2015-03-21 03:39:04)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Flashing firmware in a guest on an assigned device is extremely dis-recommended.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

3.11? Three dot One One

Yes for some old games fun 7th guest 11th hour and etc
I had found dos 6.22 flopy images and windows 3.11.
I had installed dos and cdrom drivers and I installed 3.11. There is a way to install network drivers and web browser also.
I can install windows 3.11 and it installs fine and I can exit to dos and start graphical interface but when I reboot guest win 3.11 starts with blackscreen.
I have the same issue with osx it boots to black screen when I try to install it.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Flashing firmware in a guest on an assigned device is extremely dis-recommended.

Yup . Although it worked for me , I wouldn't do it again . This GT630 without UEFI was pretty useless anyway .

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dear All,

Did anyone try Linux kernel 4.0 or 3.19 with vfio?
I see only 3.18 packadge in first post - are there any issues with new kernels (3.19 & 4.0)

Thanks!

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Flashing firmware in a guest on an assigned device is extremely dis-recommended.

OH GOD! Oh my god! Whoa.. Well.. That's a huge surprise he succeeded.

Yes for some old games fun 7th guest 11th hour and etc

Use DosBox, it's more suited for that purpose. Why would you want to passthrough something from 21 century into DOS?

Last edited by Duelist (2015-03-21 12:07:22)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dear All,

Did anyone try Linux kernel 4.0 or 3.19 with vfio?
I see only 3.18 packadge in first post - are there any issues with new kernels (3.19 & 4.0)

Thanks!

I am using 3.19.1 kernel it works fine.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dear All,

Did anyone try Linux kernel 4.0 or 3.19 with vfio?
I see only 3.18 packadge in first post - are there any issues with new kernels (3.19 & 4.0)

Thanks!

Looks like someone is maintaining a linux-vfio kernel package on AUR, i will link it on the first page https://aur.archlinux.org/packages/?O=0&K=linux-vfio

Cubex

Member

Registered: 2014-04-06

Posts: 24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello gentlemen, after some months of inactivity I decided to continue.
I noticed that after some hours the entire machine crashes (linux clock stops and keyboard leds doesn't change for example)
this happens on both windows 7 + seabios and windows 8.1 + OVMF pure

I have arch linux with unpatched linux-mainline 4.0 (ACS works without patches)
qemu from official git and propietary nvidia drivers.
430 as primary and 970 for passtrough (Asrock added primary video pci-slot selector for this motherboard as asked ^o^ )

There is any clue of what would be? journalctl and qemu monitor doesn't show anything when crash occurrs

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello gentlemen, after some months of inactivity I decided to continue.
I noticed that after some hours the entire machine crashes (linux clock stops and keyboard leds doesn't change for example)
this happens on both windows 7 + seabios and windows 8.1 + OVMF pure

I have arch linux with unpatched linux-mainline 4.0 (ACS works without patches)
qemu from official git and propietary nvidia drivers.
430 as primary and 970 for passtrough (Asrock added primary video pci-slot selector for this motherboard as asked ^o^ )

There is any clue of what would be? journalctl and qemu monitor doesn't show anything when crash occurrs

Open dmesg -w in the background and throw a glimpse on it once in a while.
Also, try adding qemu debug console for a variant with OVMF.
It would be useful to monitor GPU's power state(including temperature) AND host CPU's temp and P-state.
Also, try pushing alt-sysrq-b when your linux host is frozen, maybe it's just X window system that freezes. Pushing Alt-SysRQ-B with "magic sysrq key" option enabled in kernel will force reboot your machine. It will work even if everything is dead, but kernel is still alive.
You could also try pinging that machine, to ensure that it's a complete freeze.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi . So I discovered that I only get the following with GT610 (the card that doesn't allow VM rebooting) :

[Sat Mar 21 23:01:57 2015] irq 47: nobody cared (try booting with the "irqpoll" option)
[Sat Mar 21 23:01:57 2015] CPU: 0 PID: 3 Comm: ksoftirqd/0 Not tainted 4.0.0-1-ARCH #1
[Sat Mar 21 23:01:57 2015] Hardware name: ASUS All Series/X99-DELUXE, BIOS 1305 01/26/2015
[Sat Mar 21 23:01:57 2015]  0000000000000000 000000002442baf4 ffff88083fc03ac0 ffffffff815636ba
[Sat Mar 21 23:01:57 2015]  0000000000000000 ffff88081806b600 ffff88083fc03af0 ffffffff810d0186
[Sat Mar 21 23:01:57 2015]  ffffffff8189a880 ffff88081806b600 0000000000000000 000000000000002f
[Sat Mar 21 23:01:57 2015] Call Trace:
[Sat Mar 21 23:01:57 2015]  <IRQ>  [<ffffffff815636ba>] dump_stack+0x4c/0x6e
[Sat Mar 21 23:01:57 2015]  [<ffffffff810d0186>] __report_bad_irq+0x36/0xd0
[Sat Mar 21 23:01:57 2015]  [<ffffffff810d0527>] note_interrupt+0x257/0x2a0
[Sat Mar 21 23:01:57 2015]  [<ffffffff810cda6e>] handle_irq_event_percpu+0xae/0x1f0
[Sat Mar 21 23:01:57 2015]  [<ffffffff810cdbf1>] handle_irq_event+0x41/0x70
[Sat Mar 21 23:01:57 2015]  [<ffffffff810d0ab6>] handle_fasteoi_irq+0x86/0x140
[Sat Mar 21 23:01:57 2015]  [<ffffffff81018372>] handle_irq+0x22/0x40
[Sat Mar 21 23:01:57 2015]  [<ffffffff8156ba4f>] do_IRQ+0x4f/0xf0
[Sat Mar 21 23:01:57 2015]  [<ffffffff8156996d>] common_interrupt+0x6d/0x6d
[Sat Mar 21 23:01:57 2015]  [<ffffffff8115bea0>] ? end_page_writeback+0x60/0x60
[Sat Mar 21 23:01:57 2015]  [<ffffffff81213732>] ? mpage_end_io+0x42/0x60
[Sat Mar 21 23:01:57 2015]  [<ffffffff81285c8b>] bio_endio+0x6b/0xa0
[Sat Mar 21 23:01:57 2015]  [<ffffffff8128d144>] blk_update_request+0x94/0x3a0
[Sat Mar 21 23:01:57 2015]  [<ffffffffa00b1c53>] scsi_end_request+0x33/0x1d0 [scsi_mod]
[Sat Mar 21 23:01:57 2015]  [<ffffffffa00b372a>] scsi_io_completion+0xba/0x630 [scsi_mod]
[Sat Mar 21 23:01:57 2015]  [<ffffffffa0094561>] ? sd_done+0xb1/0x2e0 [sd_mod]
[Sat Mar 21 23:01:57 2015]  [<ffffffffa00ab17e>] scsi_finish_command+0xbe/0x100 [scsi_mod]
[Sat Mar 21 23:01:57 2015]  [<ffffffffa00b3011>] scsi_softirq_done+0x111/0x140 [scsi_mod]
[Sat Mar 21 23:01:57 2015]  [<ffffffff8129437b>] blk_done_softirq+0x8b/0xb0
[Sat Mar 21 23:01:57 2015]  [<ffffffff812942fa>] ? blk_done_softirq+0xa/0xb0
[Sat Mar 21 23:01:57 2015]  [<ffffffff81078a21>] __do_softirq+0xe1/0x2c0
[Sat Mar 21 23:01:57 2015]  [<ffffffff81078d4e>] irq_exit+0x7e/0xa0
[Sat Mar 21 23:01:57 2015]  [<ffffffff8156ba58>] do_IRQ+0x58/0xf0
[Sat Mar 21 23:01:57 2015]  [<ffffffff8156996d>] common_interrupt+0x6d/0x6d
[Sat Mar 21 23:01:57 2015]  <EOI>  [<ffffffff81092eda>] ? kthread_should_park+0x1a/0x30
[Sat Mar 21 23:01:57 2015]  [<ffffffff81096408>] ? smpboot_thread_fn+0x58/0x230
[Sat Mar 21 23:01:57 2015]  [<ffffffff810963b0>] ? SyS_setgroups+0x130/0x130
[Sat Mar 21 23:01:57 2015]  [<ffffffff81092878>] kthread+0xd8/0xf0
[Sat Mar 21 23:01:57 2015]  [<ffffffff810927a0>] ? kthread_create_on_node+0x1c0/0x1c0
[Sat Mar 21 23:01:57 2015]  [<ffffffff81568dbc>] ret_from_fork+0x7c/0xb0
[Sat Mar 21 23:01:57 2015]  [<ffffffff810927a0>] ? kthread_create_on_node+0x1c0/0x1c0
[Sat Mar 21 23:01:57 2015] handlers:
[Sat Mar 21 23:01:57 2015] [<ffffffffa0475890>] vfio_intx_handler [vfio_pci]
[Sat Mar 21 23:01:57 2015] Disabling IRQ #47

Could this lead to the reason for the VM rebooting failures ?

Any help is very much appreciated !

EDIT :

UGHHH ! setting : vfio_pci nointxmask=1  helped getting rid of the IRQ disabling message , but not the VM rebooting issue .

Last edited by Denso (2015-03-21 22:43:33)

Cubex

Member

Registered: 2014-04-06

Posts: 24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

> Open dmesg -w in the background and throw a glimpse on it once in a while.

Its not supposed that journal stores the dmesg? or maybe crash occurs so fast that journal doesnt save?

> Also, try adding qemu debug console for a variant with OVMF.

what you mean, compiling qemu in debug mode? or downloading a debug variant of OVMF?

>It would be useful to monitor GPU's power state(including temperature) AND host CPU's temp and P-state.

Temp is at normal levels, I don't know how to see P-state or gpu power state trough

>Also, try pushing alt-sysrq-b when your linux host is frozen, maybe it's just X window system that freezes. Pushing Alt-SysRQ-B with "magic sysrq key" option enabled in kernel will force reboot your machine. It will work even if everything is dead, but kernel is still alive.
>You could also try pinging that machine, to ensure that it's a complete freeze.

Its a total freeze, even sysrq doesn't respond and ping timeouts

EDIT:

Got this while using win 8.1 and UEFI, it was constant message with 5 secs interval

mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu4 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu4 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:53 I-A-X99 kernel: kvm_get_msr_common: 218 callbacks suppressed
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu8 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu8 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:48 I-A-X99 kernel: kvm_get_msr_common: 202 callbacks suppressed
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu8 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu8 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu11 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm [28001]: vcpu9 unhandled rdmsr: 0x150
mar 22 01:19:43 I-A-X99 kernel: kvm_get_msr_common: 186 callbacks suppressed
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu6 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu6 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu6 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm [28001]: vcpu6 unhandled rdmsr: 0x150
mar 22 01:19:38 I-A-X99 kernel: kvm_get_msr_common: 206 callbacks suppressed
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:33 I-A-X99 kernel: kvm_get_msr_common: 230 callbacks suppressed
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:28 I-A-X99 kernel: kvm_get_msr_common: 230 callbacks suppressed
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu1 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu3 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150
mar 22 01:19:23 I-A-X99 kernel: kvm [28001]: vcpu0 unhandled rdmsr: 0x150

Last edited by Cubex (2015-03-22 00:33:53)

tholin

Member

Registered: 2015-03-17

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You seem to have a good grasp of the options.  Personally I'd opt for using IGD for the host and replacing your guest card with one that supports UEFI.  You seem to be averse to the VGA BIOS hacking route, but that's actually one that's likely to continue working once you figure it out, with no future i915 kernel patching.  Remember you don't need to flash the card with the new ROM, you can use an external file as the ROM for the VM.  If you're willing to settle for a x1 graphics card, you can find x1-to-x16 adapters that will allow you to plug any low-profile x16 card into the x1 slot.  Double check your Xorg.0.log for DRI, DRI3 may still be enabled, but DRI2 has arbiter issues... don't ask me what the difference is, I don't know.

I've been using the IGD for a few days and realized the drivers are too buggy for daily use. Maybe it's related to the i915 patch but many other people have the same problems. To be fair the open radeon drivers also got a lot of problems but I'm used to them and know how to work around the problems. I ordered that 1X radeon card instead.

Turn out there is a reason why 1X graphic cards are so rare. They are shit. 1X does not provide enough bandwidth for a graphic card. Scrolling in browsers, moving windows and even the kvm framebuffer is so slow it's unusable. Specially if you try to use dual or triple head. So the option of using two separate radeon cards are out.

Most of the problems I have with the Intel IGD are bugs and bugs can in theory be fixed but my experience with the intel drivers are bad. They introduce bugs at about the same rate as they are fixed. Turning off SNA and going back to using UXA acceleration fix a lot of problems but I still get graphic corruption and suspend resume doesn't work.

I tried to run without the i915 vgaarb patch so see what would happen and as expected the guest stalled in boot and the host got a corrupt color palette. The main problem I have with legacy free guest is I don't have a windows 8 license and I don't know of any way of getting one, except for buying one which is not going to happen. For now it looks like I'm stuck with Intel IGD + i915 vgaarb patch and all the bugs that brings.

While I was playing a game in the vm yesterday the entire system froze and after reboot I had machine check exception in the log. I can't interpret mce:s but it was some kind of bus error. I've never seen this problem before. Perhaps that 1X graphic card was defect and caused the mce or perhaps it was caused by something else? I took out the card and haven't seen any mce since. Lets see if that lasts.

devianceluka

Member

Registered: 2014-05-19

Posts: 44

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello gentlemen, after some months of inactivity I decided to continue.
I noticed that after some hours the entire machine crashes (linux clock stops and keyboard leds doesn't change for example)
this happens on both windows 7 + seabios and windows 8.1 + OVMF pure

I have arch linux with unpatched linux-mainline 4.0 (ACS works without patches)
qemu from official git and propietary nvidia drivers.
430 as primary and 970 for passtrough (Asrock added primary video pci-slot selector for this motherboard as asked ^o^ )

There is any clue of what would be? journalctl and qemu monitor doesn't show anything when crash occurrs

So you're saying that you can passthrough devices in same group without ACS override patch in 4.0?

Draizer

Member

Registered: 2015-03-17

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So, by using OVMF instead of SeaBIOS I managed to install Windows, boot into it and install the drivers for my card, which is properly detected.  However, once I install the drivers and try to boot, I get stuck on the "Starting Windows" screen (and sometimes my whole system freezes along with it), unless I boot into safe mode or my card is disabled in device manager, which clearly shows where the issue comes from.

After checking up, I realized that my graphics card vbios (Sapphire ATI HD 7970 Ghz Edition Vapor-X 3Go) does NOT support UEFI GOP, which I suppose is the reason of my troubles. Unfortunately, since it is a custom PCB card, a generic 7970 vBIOS with UEFI support will not work (I've tried), so I've contacted Sapphire's support about this issue, as well as a Sapphire forum one-time poster who, in his unique post, complained about the same issues about the same card and apparently obtained a modified vBIOS with UEFI support. Hopefully one of these two will provide me the modified bios so that I can go forth (or at least confirm the vBIOS is not my issue). If anybody here needs it, I'll be willing to share if I get my hands on it.

I find it pretty outrageous that such a pricey card (when it came out) does not support this by default, but that's my fault for assuming it'd have UEFI support. The fact that other suppliers such as PowerColor DO support it on this model is sort of frustrating, though.

Last edited by Draizer (2015-03-22 13:41:20)

Cubex

Member

Registered: 2014-04-06

Posts: 24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So you're saying that you can passthrough devices in same group without ACS override patch in 4.0?

No, my new build has haswell-E and X99, which IOMMU groups works as intended, in the previous build with Z87 it was necesary to use the patch, 4.0 doesn't change anything

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

> Open dmesg -w in the background and throw a glimpse on it once in a while.
Its not supposed that journal stores the dmesg? or maybe crash occurs so fast that journal doesnt save?

1. I don't know how journalctl works, i know that dmesg MIGHT show you something. And if it's a full freeze, having dmesg -w on the screen will give you logs immediately, on your framebuffer or console device with almost zero latency.

> Also, try adding qemu debug console for a variant with OVMF.
what you mean, compiling qemu in debug mode? or downloading a debug variant of OVMF?

-debugcon file:debug.log -global isa-debugcon.iobase=0x402

It'll create some more verbose logs. (maybe you'll have to use OVMF, i can't recall if seabios use that debug console)

Also, usually you may see those rdmsr errors when starting something like CPU-Z. Try it and see if the system freezes.

---

So, by using OVMF instead of SeaBIOS I managed to install Windows, boot into it and install the drivers for my card, which is properly detected.

Unbelievable. Are you sure you used -pure-efi variant without CSM? Because if your GPU doesn't support GOP at all - it won't output graphics, moreover, it might not boot at all!

my graphics card vbios (Sapphire ATI HD 7970 Ghz Edition Vapor-X 3Go) does NOT support UEFI GOP

I'm a little bit surprised that this issue is present on sapphire cards too. Crap. Seems like we have to buy only reference-cards made by AMD itself to have adequate support for them.
Anyway, what you can do with it:
https://bbs.archlinux.org/viewtopic.php … 3#p1504683
As i've found out, back in 2010 or 2012 or some years ago, AMD did a presentation on UEFI-forum, claiming that their GOP EFI Driver will be universal.
UPD: found the link for that presentation: http://www.uefi.org/sites/default/files … OP_AMD.pdf
Well. It is.
Even if you have a custom PCB weird no-name GPU, as long as you have it's own bios-without-GOP AND a working GOP efi driver, you can compress the latter and stuff it into the end of your image.
There MAY be problems with ROM Chip size, well, the modified image may not fit in. BUT! Since we're dealing with QEMU, physical capability of ROM chip doesn't matter.

I, myself, have HD7750 from ASUS, a silent, passively cooled variant. I highly doubt it's reference design.

UPD2:
http://forums.amd.com/game/messageview. … did=163119
Well, seems like AMD has failed once again:

Through AMD's extensive testing, a significant number of existing platforms experienced compatibility issues with the UEFI specifications that were implemented with the Hybrid VBIOS AMD developed.  Users could experience boot failures on incompatible systems.  To ensure compatibility, the standard VGA VBIOS continues to be used, and we cannot transition with the partner manufacturers yet.  It is a System BIOS implementation issue that can be addressed with a System BIOS update.

This is not an issue on OE system builds (HP, DELL, etc..) or integrated graphics components (where VBIOS is implemented directly into SBIOS), because full UEFI compatibility is set from the ground-up.

Also, some more links:
http://hardforum.com/showthread.php?t=1725316&page=4
http://www.overclock.net/t/1389206/do-y … -your-bios <<THAT'S AN AWFUL WAY of doing stuff, which EfiRom does.

Last edited by Duelist (2015-03-22 15:17:34)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Draizer

Member

Registered: 2015-03-17

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Unbelievable. Are you sure you used -pure-efi variant without CSM? Because if your GPU doesn't support GOP at all - it won't output graphics, moreover, it might not boot at all!

Well, I'm an idiot. Now that I've taken a bit of rest and see things more clearly I'm pretty sure the reason it worked is because Windows skipped the ATI card and used QEMU's emulated card. Using -vga none leads me straight into the QEMU console. With that said, Windows does detect the right card... it'll just get stuck on its starting screen if I try to use it.

And yes, I'm sure I'm using a pure-efi binary.

I'm a little bit surprised that this issue is present on sapphire cards too. Crap. Seems like we have to buy only reference-cards made by AMD itself to have adequate support for them.
Anyway, what you can do with it:
https://bbs.archlinux.org/viewtopic.php … 3#p1504683
As i've found out, back in 2010 or 2012 or some years ago, AMD did a presentation on UEFI-forum, claiming that their GOP EFI Driver will be universal.
UPD: found the link for that presentation: http://www.uefi.org/sites/default/files … OP_AMD.pdf
Well. It is.
Even if you have a custom PCB weird no-name GPU, as long as you have it's own bios-without-GOP AND a working GOP efi driver, you can compress the latter and stuff it into the end of your image.
There MAY be problems with ROM Chip size, well, the modified image may not fit in. BUT! Since we're dealing with QEMU, physical capability of ROM chip doesn't matter.

I, myself, have HD7750 from ASUS, a silent, passively cooled variant. I highly doubt it's reference design.

I'll check that out, hopefully I'll get lucky.

Last edited by Draizer (2015-03-22 22:17:31)

dRaiser

Member

From: Poland

Registered: 2013-05-20

Posts: 51

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys,

Quick question: I've seen few times in this topic information that Nvidia GPU's require to be put in 1st pci x16 slot. What does it say about using Nvidia for host and another Nvidia for guest? Would there be problems?

Currently I have GF 9800 GTX+ for host and Radeon 4850 for Win guest and I'm going to change Radeon for something nicer and was thinking about Nvidia GTX 780.

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I would go with radeon R9 270 or  better. Nvidia will give you error 43.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I would go with radeon R9 270 or  better. Nvidia will give you error 43.

And radeon will give you reset problems.  We know how to work around the Code 43 issue (for now), we don't have a good solution for the resets.  dRaiser, whatever you're reading about the 1st slot and nvidia is wrong.  Slots come into play for ACS and if your motherboard vendor doesn't allow fine grained selection of the primary graphics device.  The only issue I'm aware of with using nvidia for the host is that the proprietary driver grabs the vga arbiter lock and doesn't release it.  So you either need to avoid vga or hack nvidia's wrapper code.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I would go with radeon R9 270 or  better. Nvidia will give you error 43.

The only issue I'm aware of with using nvidia for the host is that the proprietary driver grabs the vga arbiter lock and doesn't release it.  So you either need to avoid vga or hack nvidia's wrapper code.

I think nvidia fixed this since i'm using the stock drivers without issue.