Arch Linux

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

From what I can gather from your posts, I believe - but am not certain - that what you meant was "Setup B using SPICE for controlling the Windows VM from the Linux machine". Is that correct? If so, then you won't be able to use that for any GPU-heavy applications, since an emulated graphics card won't be even close to good enough to what you need for that, regardless of how good or bad SPICE is for streaming the video to you.

Sorry for my lack of words (English isn't my native language ) , and yes , I meant setup B . Where I get to interact with the VM directly from the host or through an external client . I was wondering whether SPICE can offer a big performance improvement over regular VNC or RDP , and from what I saw on YT it's still somewhat laggy . Maybe this was off topic when it comes to GPU passthrough , but it can be a solution for web surfing , printing documents , and watching videos . (Especially when you lack PCI-E lanes to add more cards as I stated previously) .

Thank you

No problem, it isn't my first language either. Alright, setup B it is. Do you mind my asking what the reason is? Web surfing, printing, watching videos, and pretty much anything else that does not expicitly depend on Windows-exclusive programs can be done on the Linux host, as well. And Intel's IGP are a lot more powerful than some people think, so even 1080p videos should be no problem at all. However, if you really want to do all of that on Windows, why even install Linux as the host? Just have Windows as the host and Linux in a VM (if you need the Linux at all, that is).

In either case, I don't think you should see stuttering from host-side if you use the "std" emulated graphics card (unless you intend to watch something really heavy, like a 1080p video). You don't need any extra VNC, or RDP there either, just use the normal monitor window that opens when you start the VM (I think it uses VNC internally, but I'm not sure). If you want to connect from a different, separate box, however, you might want to look into Steam Inhome-streaming. I've read on several pages that you can highjack it to get back at the desktop (i.e. have the Windows-desktop streamed to the client) and then you can just use any application you want through that. See this discussion for some ways to do that.

Well , I can't get rid of Windows altogether for work reasons . I used to run Hyper-V and a Linux VM on top of that , but it wasn't good enough to run a ZFS file server . So I switched to a Linux host + ZFS server and a Windows VM . That made me sleep at night knowing my TBs of data are safer that way

Plus , I want my linux box to be as headless as it can get

I'm pretty happy with my current GPU passthrough setup , except for the reboot issue .

I might look into Steam's streaming if my GT210 didn't work . I'm just a little lazy to open that NORCO case and switch GPUs

Thanks !

Calrama

Member

From: Berlin

Registered: 2011-10-14

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I finally got it working !
The host display is on the laptop screen and the guest screen on the secondary screen. It works fine but I had some other duties to do so I couldn't finish my configuration right now but I will soon.

About the outputs ports, it's actually working on the hdmi. I will try on the VGA on next try.

Well, I'll be. Sorry for posting some wrong information then, I was really convinced that it wouldn't work like that. Now I'm kind of curious whether the HDMI port actually belongs to the dedicated card, or if the passed through dedicated card still communicates correctly with the IGP and the IGP then forwards the guest's video output to whatever happens to be connected as a secondary screen (the latter of which would imply some serious coding on the side of the Intel devs who wrote the driver code).

The DMAR message is still there, but it doesn't seem to be an issue. Actually, there is something relative to memory in the BIOS: in the "Video (Intel IGD) Control Sub-Menu", there is a parameter "Total Graphics Memory" which can be set to 128MB, 256MB or MaxDVMT. Informations says: "Select the amount of Total Graphics Memory - Pre-Allocated + Fixed + DVMT for use by the Intel Graphics Device" As I'm actually working with OpenGL, I thought it was the framebuffer memory and nothing related to the DMA Remapping.
Well, I'm still wondering what that really means...

I'll try to get in touch with this Intel guy, he would certainly know about it.

Intel IGPs don't have any dedicated (V)RAM on their own (unlike the GDDR RAM of modern dedicated graphics cards), since they are integrated into the CPU-chip. So when your system boots up a small portion of your actual, normal RAM gets reserved as VRAM for the integrated graphics card. This option should set the size of that reserved portion.

Yep, that's right, it bothered me a lot !

Anyway, thank you a lot for your time. I'll post again tomorrow to tell you if it's stable but the big part is done, know I'm very satisfied and proud of my work. Gonna play with my new baby, hehe !

Have a nice day since then !

Cool, do have fun with it, I know I have with my OVMF-Machine.

Calrama

Member

From: Berlin

Registered: 2011-10-14

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Plus , I want my linux box to be as headless as it can get

Hm, I would just use two separate boxes, one headless server box for all the data and one desktop box, but if you're happy with that, "never change a running system", I guess *toothless grin*.

I'm pretty happy with my current GPU passthrough setup , except for the reboot issue .

Have you tried OVMF? That got rid of my problems.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Have you tried OVMF? That got rid of my problems.

Yup , all of my VMs are now running on OVMF

I expect it to be a hardware issue , propably the X99 platform is to be blamed . But my GTX770 VM is working flawlessly , so it might also be a GT610-specific issue .

I'll nail it when I try my other GT210 and see if I still have reboot issues .

doubledr

Member

Registered: 2013-12-26

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The idea of using SPICE with a low-end thin client (maybe a thin client that costs less than a dedicated GPU) sounds good enough for web browsing and watching YouTube . It can also be a solution for those who don't have enough PCI-E lanes for more GPUs . But I watched it in action on a YT video , and it seems to stutter/lag . So it won't be a good replacement for GPU passthrough .

Forgive me for being so blunt, but from my point of view you're still not answering my question about why you think about bringing in SPICE as a replacement for a graphics card passthrough. Please consider the following:

Setup A:

- One hardware box
- Linux runs natively on that box
- Windows runs as a VM on that box and gets a graphics card passed through from the Linux host

Setup B:

- One hardware box
- Linux runs natively on that box
- Windows runs as a VM on that box with an emulated graphics card (for QEMU that would be any of std,cirrus,qxl,vmware,...)

Setup C:

- Two separate hardware boxes, connected by some network (e.g. 1Gbit Ethernet, Infiniband, whatever)
- Box 1 has a native Linux running
- Box 2 has a native Windows running

These are essentially your choices (very abstract). The statement "SPICE with a low-end thin client" does not specify which, though, since the native Linux in all three setups could be such a thin client.
SPICE, VNC, or any such protocol is about how you control - in this context - the Windows machine (regardless of whether you have one or two boxes); the Windows machine (regardless of whether it is being virtualised or not) will still need access to a real, dedicated graphics card (setups A and C). I know of no way aroung that. In the far future when XenGT has become something that can actually be done with NVIDIA and AMD graphics cards, we will have a way around that by paravirtualising our graphics card analogous to how Vt-x/AMD-V paravirtualise your processor. But that's still quite a bit away.

SPICE is by definition not a replacement for a graphics card passthrough, it's about remote control and can be done regardless of whether you're doing such a passthrough, which is why I was - and still am - so confused by your posts about this.

From what I can gather from your posts, I believe - but am not certain - that what you meant was "Setup B using SPICE for controlling the Windows VM from the Linux machine". Is that correct? If so, then you won't be able to use that for any GPU-heavy applications, since an emulated graphics card won't be even close to good enough to what you need for that, regardless of how good or bad SPICE is for streaming the video to you.

Sorry for my lack of words (English isn't my native language ) , and yes , I meant setup B . Where I get to interact with the VM directly from the host or through an external client . I was wondering whether SPICE can offer a big performance improvement over regular VNC or RDP , and from what I saw on YT it's still somewhat laggy . Maybe this was off topic when it comes to GPU passthrough , but it can be a solution for web surfing , printing documents , and watching videos . (Especially when you lack PCI-E lanes to add more cards as I stated previously) .

Thank you

Right now only VMware, Microsoft, Xenserver and VirtualBox provide what you want. In kvm world... Forget it... Please wait another 3 years...

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[......] Please wait another 3 years...

I'm in no hurry My GPU passthrough setup is working good enough (for now) !

mouton_sanglant

Member

Registered: 2014-10-20

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I finally got it working !
The host display is on the laptop screen and the guest screen on the secondary screen. It works fine but I had some other duties to do so I couldn't finish my configuration right now but I will soon.

About the outputs ports, it's actually working on the hdmi. I will try on the VGA on next try.

Well, I'll be. Sorry for posting some wrong information then, I was really convinced that it wouldn't work like that. Now I'm kind of curious whether the HDMI port actually belongs to the dedicated card, or if the passed through dedicated card still communicates correctly with the IGP and the IGP then forwards the guest's video output to whatever happens to be connected as a secondary screen (the latter of which would imply some serious coding on the side of the Intel devs who wrote the driver code).

Ok, I finished to configure my virtualized system. It's working fine with drivers installed inside host, played some games, no crashes ! Performances are "passed-troughed" just as expected !

Windows detects 2 displays, so it automatically enable extended desktop. One of these 2 displays can be disabled, however, the other one can't or I won't have any signal until I hit esc (to cancel action).
I ran some additionnal testings: can't pass through via the VGA cable, it duplicates the linux screen. So it seems to confirm what you pointed about the HDMI belongs to the dedicated card. I'll try someday to install a non-virtualized Windows on another HDD, disable the discrete card and plug a screen in the hdmi port.

Now, I have to fix an issue on the host: I can't launch any Desktop Manager. I think it is because of the i915 vga arbiter patch which disable DRI. Is that right ?
Window Managers works, so I should use Mozilla and stuff like that as long as I disable any 3d acceleration stuff, right ?

Preparing the GPU so we can bind it to vfio

If you are using an intel IGP on your host, you need the i915 vga arbiter patch (this patch is included in the kernel i provided), and you need to add this kernel parameter to your bootloader:

i915.enable_hd_vgaarb=1

NOTE: This will disable DRI for you IGP!

[Edit]: I just disabled the kernel parameter "i915.enable_hd_vgaarb=1" in the bootloader and everything works fine. Is this patch fix performances issues or things like that ?

Last edited by mouton_sanglant (2014-10-21 16:21:44)

Ansa89

Member

Registered: 2014-08-30

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You need VirtIO drivers for Windows if you want to use VirtIO devices in a Windows guest, see here. If you want a direct download, you can grab the - currently - latest Fedora-compiled, signed VirtIO drivers for Windows here. That is an iso file, so you have to add it to your machine without VirtIO, i.e. as a normal IDE drive, so Windows can find it (Adding "-cdrom /path/to/virtio-win-0.1-81.iso" to your qemu command should do it). Once you booted from the Windows installation disc and got to the part where the installer "cannot find installation media" you should be able to select "Browse" or something similar (cannot remember exactly what it is called), at which point you select the Win8 directory on the drive that corresponds to the virtio driver iso (should be the only CDROM drive at this point) and select all the VirtIO drivers that he shows you for installation. Afterwards, you should be able to continue the Windows installation normally.

Ok, I'm now on VirtIO.

I've tried Steam InhomeStreaming from the guest VM directly to the Gentoo host (no real network card involved) with the Guest using a VirtIO network card (which is currently the best in performance AFAIK).
I had my left monitor show the actual output of the guest VM (connected to the passed-through graphics card) and the right monitor be the Host's Steam Inhome-Streaming client with the following results at 1080p@60Hz:
If the screen content stays nearly the same or is very slow moving, you're not going to see a difference on average. But I saw a significant difference with fast-moving content, There were many skipped frames, the FPS count plummeted from 60 down do 30. Also, regardless of whether it's fast or slow-moving content, I had spikes which had the Inhome-Streaming client lag as far behind as one second (the missing frames were then completly skipped to get to the current frames).
I have no experience with nVidia GameStream, but I doubt they have a solution for network spikes. If the above is something you can live with, then definitely go for it, but for me this was unbearable.

And you are right again: I tested HomeStreaming with both e1000 and VirtIOnet and I confirm very annoying network spikes.
What about using a physical network card passed through with "vfio-pci"?
Beside that, the fact that I can stream a game means Windows can use the GTX 650 in some way (even if I'm booting with "-vga std").

But regardless of the above two paragraphs, you might really want to get your passthrough to work on its own, before moving on to that stuff, because right now you're trying two things at the same time without having confirmed the first thing to be working properly.

You absolutely right, I will try qemu-git ASAP (sorry, but right now I'm very busy).

As always, thanks very much.

Calrama

Member

From: Berlin

Registered: 2011-10-14

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Ok, I finished to configure my virtualized system. It's working fine with drivers installed inside host, played some games, no crashes ! Performances are "passed-troughed" just as expected !

Windows detects 2 displays, so it automatically enable extended desktop. One of these 2 displays can be disabled, however, the other one can't or I won't have any signal until I hit esc (to cancel action).
I ran some additionnal testings: can't pass through via the VGA cable, it duplicates the linux screen. So it seems to confirm what you pointed about the HDMI belongs to the dedicated card. I'll try someday to install a non-virtualized Windows on another HDD, disable the discrete card and plug a screen in the hdmi port.

That's quite an interesting design (thanks for the information), do post about it should go get around to it.

Now, I have to fix an issue on the host: I can't launch any Desktop Manager. I think it is because of the i915 vga arbiter patch which disable DRI. Is that right ?
Window Managers works, so I should use Mozilla and stuff like that as long as I disable any 3d acceleration stuff, right ?

I'm afraid I cannot be of any help here, sorry. When I was still doing VGA-Passthrough, I always had full DRI-capabilities for my IGP and no problems at the host side other than sometimes my entire host system's input periphery completely freezing when booting the VM, even with the VGA-Arbitration patch and this kernel option active (despite the patch explicitly stating the opposite). But I never dug deep into this, since I moved on to OVMF to get rid of this freezing issue.

Calrama

Member

From: Berlin

Registered: 2011-10-14

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And you are right again: I tested HomeStreaming with both e1000 and VirtIOnet and I confirm very annoying network spikes.

One way to at least reduce that is to use hardware en- and decoding. To enable hardware encoding you need to install Geforce experience, click on the "shadow play" button in its GUI and flip the left-positioned virtual switch that they show in the new popup windows: Turn it on once, then off again. After that you will need to access the Inhome-Streaming options in Steam's settings (on the client), open up the advanced host settings, and enable hardware encoding (You might also want to set the same option on the VM's steam installation for good measure, but it shouldn't be necessary).

You might also try to set the client options to "fast" and set the bandwidth limit in the advanced client options to automatic (should it not already be set to it). I've found that even with hardware en- and decoding I could still detect a slight difference between host and guest while inside an FPS, because the camera didn't rotate completely smoothly on the guest. Just enough that it would get me killed in multiplayer. The only way I could rid of any noticeable latency when Inhome-streaming through an emulated VirtIO network card was to limit the bandwith manually to 3Mbit, at which 1080p looks like someone threw several buckets full of different-coloured paint at my screen from the inside (and 720p is unacceptable to me).

What about using a physical network card passed through with "vfio-pci"?

I do not own one, but I'm going to try that out when I have enough time to make an informed decision about picking one.

Beside that, the fact that I can stream a game means Windows can use the GTX 650 in some way (even if I'm booting with "-vga std").

It should. Just to be sure: You are able to stream regardless of whether your VM gets an emulated card or not (so both "-vga none" and "-vga std" work)?

You absolutely right, I will try qemu-git ASAP (sorry, but right now I'm very busy).

As always, thanks very much.

Happy to help.

Last edited by Calrama (2014-10-22 00:18:28)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Suddenly, i can not boot OVMF's built-in shell or start Windows8.1 installation CD in UEFI mode. There is just blank screen and a cursor on serial0, and nothing on any VGA or video output. I need OVMF to avoid IO_PAGE_FAULTS referring to my first GPU. The windows ISO checked to be correct and it does boot on real hardware.

http://pastebin.com/9Wb3trV4

OVMF:20141021.b745.g8c83d0c
qemu 2.1.2
Fedora 21
Kernel 3.17.1-302.fc21.x86_64

Host Nvidia driver with vgaarb patch(though it shouldn't be needed anymore when using OVMF).

Two HD7750. romfile appear to be looking good, however, i couldn't update(ATIWinFlash just crashes after start, maybe i need drivers installed) it physically, so the ROMs in fact may be fake and broken. But if that's the case - asus support ticket should help me getting updated or getting the right ROMs.

Last edited by Duelist (2014-10-22 14:48:43)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Two HD7750. romfile appear to be looking good, however, i couldn't update(ATIWinFlash just crashes after start, maybe i need drivers installed) it physically, so the ROMs in fact may be fake and broken. But if that's the case - asus support ticket should help me getting updated or getting the right ROMs.

Are you saying you have an ASUS Radeon HD7750 with UEFI support? Can I have that rom? Pretty please? My HD7750 is vga-only.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Two HD7750. romfile appear to be looking good, however, i couldn't update(ATIWinFlash just crashes after start, maybe i need drivers installed) it physically, so the ROMs in fact may be fake and broken. But if that's the case - asus support ticket should help me getting updated or getting the right ROMs.

Are you saying you have an ASUS Radeon HD7750 with UEFI support? Can I have that rom? Pretty please? My HD7750 is vga-only.

You need an Asus motherboard(Almost any, but i'm not sure about FM2(+) socket ones) and Asus HD7750. That way you are able to update it's bios.
http://www.asus.com/Graphics_Cards/HD77 … _Download/
http://www.asus.com/Graphics_Cards/HD77 … _Download/

As for exact .rom files - that will be useless because:
1.I have DCSL, they're passive cooled and they were pretty rare. The clocks, voltage and Fan profiles(yes, you've read it right, passively cooled GPU has fan profiles) are different on different models, and that may fry your card.
2.I took a diff out of two original romfiles, and we have a serial number in here. And something like CRC32-checksum in three bytes.. Don't know about the UEFI-capable ROM differences, but i've copy-pasted the serial just in case it'll be checked somehow.

Last edited by Duelist (2014-10-22 17:03:23)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You need an Asus motherboard(Almost any, but i'm not sure about FM2(+) socket ones) and Asus HD7750. That way you are able to update it's bios.

That would be too easy. I don't have an ASUS motherboard. I downloaded the update utility. It's an .exe file. I haven't tried running it yet, but I guess it won't give me the .rom file easily. I just wanted to run an EFI bios without flashing it.

As for exact .rom files - that will be useless because:
1.I have DCSL, they're passive cooled and they were pretty rare. The clocks, voltage and Fan profiles(yes, you've read it right, passively cooled GPU has fan profiles) are different on different models, and that may fry your card.
2.I took a diff out of two original romfiles, and we have a serial number in here. And something like CRC32-checksum in three bytes.. Don't know about the UEFI-capable ROM differences, but i've copy-pasted the serial just in case it'll be checked somehow.

Bummer. I certainly won't risk flashing a bios from any fanless design on my fanned HD7750-1GD5-V2. That being said, I don't think there is a serial number check in the GPU because some people even flash vbioses from different vendors (like the guy who commented on aw's blog about using Powercolor's bios on VTX3D's card).

I wish there was at least one EFI-capable vbios for HD7750 1GB DDR5 on TechPowerUP (all 51 of them are vga-only).

dtm

Member

Registered: 2014-10-22

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi Everyone,

Thanks for the fantastic thread, using it I have managed to get a fair way towards getting a working setup, at the same time breathing new life into my aging machine.

Specs:

Motherboard: Gigabyte GA-X58A-UD5 (w/VT-d enabled bios flashed)
CPU: Intel Xeon 5670
RAM: 12GB
Primary Video: Nvidia NVS 300
Passthrough Video: Nvidia GTX 570

I've installed on Debian jessie (testing) with a stock 3.17.1 kernel, to which I have added a quirk in drivers/pci/quirks.c to get my Marvell 9128 SATA3 controller working properly:

/* Declare this quirk for Marvell 9128 controller */
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_MARVELL_EXT, 0x9128,
                         quirk_dma_func1_alias);

I'm running the versions of libvirt and qemu which are standard to debian jessie, i.e. qemu 2.1.2 and libvirt 1.2.8. I've actually managed to get my video and USB3 controller bound and passed succesfully to a windows 8.1 virtual machine, and thus far it all seems to work well, although I am yet to do any benchmarking or performance testing.

The problem I'm having is when I enable intel_iommu at boot, my onboard sound is disabled. I see the following messages in dmesg:

dmesg | grep -E '(IOMMU|DMAR|1b\.0)'
[    0.000000] ACPI: DMAR 0x00000000D7EDBF00 0000B0 (v01 IntelR AWRDACPI 322E3030 DRWA 00000002)
[    0.000000] Intel-IOMMU: enabled
[    0.023999] dmar: IOMMU 0: reg_base_addr fe711000 ver 1:0 cap c9008010e60262 ecap f0207a
[    0.024005] dmar: IOMMU 1: reg_base_addr fe710000 ver 1:0 cap c90780106f0462 ecap f020fe
[    0.364780] pci 0000:00:1b.0: [8086:3a3e] type 00 class 0x040300
[    0.364794] pci 0000:00:1b.0: reg 0x10: [mem 0xfbff4000-0xfbff7fff 64bit]
[    0.364856] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.364891] pci 0000:00:1b.0: System wakeup disabled by ACPI
[    3.854860] DMAR: No ATSR found
[    3.854903] IOMMU 0 0xfe711000: using Queued invalidation
[    3.854906] IOMMU 1 0xfe710000: using Queued invalidation
[    3.854907] IOMMU: Setting RMRR:
[    3.854921] IOMMU: Setting identity map for device 0000:00:1a.0 [0xd7ee0000 - 0xd7efffff]
[    3.854956] IOMMU: Setting identity map for device 0000:00:1a.1 [0xd7ee0000 - 0xd7efffff]
[    3.854983] IOMMU: Setting identity map for device 0000:00:1a.2 [0xd7ee0000 - 0xd7efffff]
[    3.855009] IOMMU: Setting identity map for device 0000:00:1a.7 [0xd7ee0000 - 0xd7efffff]
[    3.855035] IOMMU: Setting identity map for device 0000:00:1d.0 [0xd7ee0000 - 0xd7efffff]
[    3.855062] IOMMU: Setting identity map for device 0000:00:1d.1 [0xd7ee0000 - 0xd7efffff]
[    3.855088] IOMMU: Setting identity map for device 0000:00:1d.2 [0xd7ee0000 - 0xd7efffff]
[    3.855114] IOMMU: Setting identity map for device 0000:00:1d.7 [0xd7ee0000 - 0xd7efffff]
[    3.855132] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    3.855144] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    4.146418] AMD IOMMUv2 driver by Joerg Roedel <joerg.roedel@amd.com>
[    4.146419] AMD IOMMUv2 functionality not available on this system
[    8.116548] snd_hda_intel 0000:00:1b.0: irq 42 for MSI/MSI-X
[    8.120953] dmar: DMAR:[DMA Read] Request device [00:1b.0] fault addr ffff6000
DMAR:[fault reason 02] Present bit in context entry is clear
[    9.121162] snd_hda_intel 0000:00:1b.0: Codec #2 probe error; disabling it...
[    9.125612] dmar: DMAR:[DMA Read] Request device [00:1b.0] fault addr ffff6000
DMAR:[fault reason 02] Present bit in context entry is clear
[   14.140288] snd_hda_intel 0000:00:1b.0: no AFG or MFG node found
[   14.140307] snd_hda_intel 0000:00:1b.0: no codecs initialized

If I boot without intel_iommu=on, then the codec is detected correctly and everything comes up OK. This is also the case if I boot with intel_iommu=pt, although in that case I do not see any IOMMU groups and attempting to bind devices using VFIO fails.

I was wondering if anyone had seen anything like this before or had any ideas as to what I might try to get my onboard sound working again?

kristopher004

Member

Registered: 2014-06-30

Posts: 33

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Updated from 3.16 to  3.17.1-1 and now the vm skips. When I move the mouse or drag a window the mouse or window will freeze for a sec every few seconds. This happen to anyone else when they went to kernel 3.17?

Using X79 motherboard, 4930k cpu, and passing through an nvidia titan with a nvidia 670 on the host.

This happens using ovmf/1440FX as well as with bios/q35. Using qemu-git 2.2 and libvirt-git 1.2.9 and ovmf-svn 16214

Last edited by kristopher004 (2014-10-23 04:29:04)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You need an Asus motherboard(Almost any, but i'm not sure about FM2(+) socket ones) and Asus HD7750. That way you are able to update it's bios.

That would be too easy. I don't have an ASUS motherboard. I downloaded the update utility. It's an .exe file. I haven't tried running it yet, but I guess it won't give me the .rom file easily. I just wanted to run an EFI bios without flashing it.

As for exact .rom files - that will be useless because:
1.I have DCSL, they're passive cooled and they were pretty rare. The clocks, voltage and Fan profiles(yes, you've read it right, passively cooled GPU has fan profiles) are different on different models, and that may fry your card.
2.I took a diff out of two original romfiles, and we have a serial number in here. And something like CRC32-checksum in three bytes.. Don't know about the UEFI-capable ROM differences, but i've copy-pasted the serial just in case it'll be checked somehow.

Bummer. I certainly won't risk flashing a bios from any fanless design on my fanned HD7750-1GD5-V2. That being said, I don't think there is a serial number check in the GPU because some people even flash vbioses from different vendors (like the guy who commented on aw's blog about using Powercolor's bios on VTX3D's card).

I wish there was at least one EFI-capable vbios for HD7750 1GB DDR5 on TechPowerUP (all 51 of them are vga-only).

TechPowerUp VBIOS database is outdated as hell. You can try asking asus support for exact ROMs. Or you can find some friends with somewhat decent new motherboard and windows on board.

As for the serial number - the driver may go crazy.

And you can also try running that asus thingie, it may give you ROM, but refuse to flash.

Last edited by Duelist (2014-10-23 11:20:01)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Suddenly, i can not boot OVMF's built-in shell or start Windows8.1 installation CD in UEFI mode. There is just blank screen and a cursor on serial0, and nothing on any VGA or video output.

Alright, i've turned off GPU PT, got into shell, booted from file(win8.1's EFI loader), got a progress bar about "loading files" full, and then nothing anywhere. Something hints me i should have some graphics showing up, but there is no output.

After installing windows8 without OVMF, i'm still unable to boot with gpu passthrough. Seems like the ROM files were broken, and something screws up badly. I guess i'll have to search for some other ASUS motherboard to update them physically.

Last edited by Duelist (2014-10-23 12:53:34)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

doubledr

Member

Registered: 2013-12-26

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys!

Today I successfully hardmodded my GTX 680 to report itself as GRID K2.

When I try to passthrough the card, I get no output on either display outputs though. Nothing suspicious in the logs, just:

Nov 24 20:19:42 cave-lin kernel: [  368.001538] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Nov 24 20:19:42 cave-lin kernel: [  368.001719] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Nov 24 20:20:20 cave-lin kernel: [  405.590323] kvm: zapping shadow pages for mmio generation wraparound

I havent changed anything since last time I run my config, except of device ID to assign to pci-stub during boot. Prior to that passthrough worked and I just wasnt able to install Nvidia drivers in guest.

Any advice?

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.  However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

Hi, aw.

I tried it today. Passing a real(not modded) Nvidia Grid k2 results in code 12 in my windows 2008 vm... Same as a Quadro 400... Look's like "x-vga" is a must...

l have some difficulty with libvirt. No matter what I did, if I start my vm with libvirt, I always get the "Cannot allocate memory" error. "ulimit -l unlimited" doesn't help at all... Is anyone have similar issue? Now I can only start my vm manually with a long qemu command

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I want to start looking into using OVMF, but before spending too much time on it a quick check: Can anyone here confirm if the XFX HD7850 (Dual Dissipation edition) has support for UEFI/GOP? I alread asked XFX, but got no answer so far...
Thanks!

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You can try asking asus support for exact ROMs.

Great idea! I contacted them and got the rom just now. :3

./rom-parser 113-AD41300-123.bin                                                                                                                              
Valid ROM signature found @0h, PCIR offset 224h
        PCIR: type 0, vendor: 1002, device: 683f, class: 030000
        PCIR: revision 0, vendor revision: f20
Valid ROM signature found @10000h, PCIR offset 1ch
        PCIR: type 3, vendor: 1002, device: 683f, class: 030000
        PCIR: revision 0, vendor revision: 0
                EFI: Signature Valid
        Last image

It's time to give OVMF a try...

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I want to start looking into using OVMF, but before spending too much time on it a quick check: Can anyone here confirm if the XFX HD7850 (Dual Dissipation edition) has support for UEFI/GOP? I alread asked XFX, but got no answer so far...
Thanks!

Read this: http://vfio.blogspot.de/2014/08/does-my … t-efi.html

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You can try asking asus support for exact ROMs.

Great idea! I contacted them and got the rom just now. :3

./rom-parser 113-AD41300-123.bin                                                                                                                              
Valid ROM signature found @0h, PCIR offset 224h
        PCIR: type 0, vendor: 1002, device: 683f, class: 030000
        PCIR: revision 0, vendor revision: f20
Valid ROM signature found @10000h, PCIR offset 1ch
        PCIR: type 3, vendor: 1002, device: 683f, class: 030000
        PCIR: revision 0, vendor revision: 0
                EFI: Signature Valid
        Last image

It's time to give OVMF a try...

Please, mark if you manage to get that cards doing something more useful than powering down when passing the ROM via romfile=.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys!

Today I successfully hardmodded my GTX 680 to report itself as GRID K2.

When I try to passthrough the card, I get no output on either display outputs though. Nothing suspicious in the logs, just:

Nov 24 20:19:42 cave-lin kernel: [  368.001538] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Nov 24 20:19:42 cave-lin kernel: [  368.001719] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Nov 24 20:20:20 cave-lin kernel: [  405.590323] kvm: zapping shadow pages for mmio generation wraparound

I havent changed anything since last time I run my config, except of device ID to assign to pci-stub during boot. Prior to that passthrough worked and I just wasnt able to install Nvidia drivers in guest.

Any advice?

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.  However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

Hi, aw.

I tried it today. Passing a real(not modded) Nvidia Grid k2 results in code 12 in my windows 2008 vm... Same as a Quadro 400... Look's like "x-vga" is a must...

Use 440FX not Q35

l have some difficulty with libvirt. No matter what I did, if I start my vm with libvirt, I always get the "Cannot allocate memory" error. "ulimit -l unlimited" doesn't help at all... Is anyone have similar issue? Now I can only start my vm manually with a long qemu command

If libvirt doesn't know you're doing device assignment it won't set your limits correctly.  If you use 440FX instead of Q35 you don't need any qemu:commandline sections and should be allowing libvirt to manage everything.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

deniv

Member

Registered: 2013-10-16

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Please, mark if you manage to get that cards doing something more useful than powering down when passing the ROM via romfile=.

Well, I can confirm that this rom works in vga mode. I used it to load my standard Win7 VM.
EFI, on the other hand, is a pain. First I tried to install Win8.1 just with QXL, but OVMF can't load the installation DVD (no drivers for the UDF filesystem?). I copied the DVD to a fat32 partition and managed to get the loader running, but it still fails quite soon with "can't find the media" error (maybe the installation process looks for UDF and dies because there is fat32?). Anyway, just to test EFI vbios I tried connecting the GPU annnnnd... nothing. I get a black screen and 100% one core utilisation, can't see OVMF, can't get to Windows installer.

P.S. I noticed in gpuz that this new bios (ver 015.032) bumps the default core clock to 900MHz while the original bios (ver 015.014) sets default to 820MHz.

Last edited by deniv (2014-10-24 18:26:04)