Arch Linux

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Seems like OVMF is storming today:
I am unable not only to run windows 7 with AMD, but i am trying to just install it without any attached devices - and it crashes and crashes and crashes and hangs with 4 cores 100% usage.

...i should plan a new build, this one is starting to drive me nuts.

Last edited by Duelist (2015-07-16 23:45:51)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

The_Moves

Member

Registered: 2015-01-06

Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Windows 10 + OVMF + virtio-blk-pci is a no-go .
...
Anyone using Windows 10 ?

I have been using Windows 10 for many months now, and just last night updated to the latest 10240 Build (RTM Supposidlt). I don't know what virtio-blk-pci is though? Is it like mapping a disk to the VM? I use Raw images.

Can someone with 980 try soft mod (with fake id patch on qemu) to quadro m5000 and try latest geforce driver with hyper-v on?

Slis, can you please explain or link to where you can change the PCI IDs via Qemu? I have a GTX760 that i'd like to soft-mod via Qemu. I also have a GTX465 which I soft-modded via rom hacks into a Quadro 5k. It worked fine with ESXi but the performance isn't near my GTX760 for gaming... I would love to have my 760 be represented as a Quadro so I can enable the HyperV Flags - assuming it will help performance.

Also, I have been using the latest Windows 10 NVIDIA Drivers provided by Windows Update without issue.

Patch is there...
https://bbs.archlinux.org/viewtopic.php … 0#p1475170

Is this patch still upstream? What versions of QEMU does it work with? I currently have qemu-2.3.0-5.fc22.x86_64 installed

Last edited by The_Moves (2015-07-17 04:21:36)

morat

Member

Registered: 2015-07-07

Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Can someone with 980 try soft mod (with fake id patch on qemu) to quadro m5000 and try latest geforce driver with hyper-v on?

I've got a 980, can you tell me what I need to do to try the soft-mod?

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

That patch is aw's improvisation, i don't think it will be upstream ever....

Well you can patch qemu with this patch https://bbs.archlinux.org/viewtopic.php … 0#p1475170

and change card id-s to quadro m5000, I guess its  10de:13f0, then try latest geforce driver with hyper-v on.

Last edited by slis (2015-07-17 07:01:14)

impulse_255

Member

Registered: 2015-06-29

Posts: 22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Don't you also need a rom file to pass along with the spoofed pciid? Unfortunately Maxwell Quadro rom dumps don't seem to be available anywhere. They are not in Techpowerup's VGA bios database either. I'm very interested in trying to masquerade my Titan X as M6000.

Regarding OVMF problems, I think it's something to do with the latest versions. I got a BSOD in Windows 8 even with any configuration (no passthrough, just virtualised everything). An old build, in my case the sata patched binary, worked fine.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah, forgot about bios ... it should show up soon... you can try with no rom or with default one, it may work....

morat

Member

Registered: 2015-07-07

Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah, forgot about bios ... it should show up soon... you can try with no rom or with default one, it may work....

It won't, telling the VM that the hardware ID is different from what the GPU claims will only cause more problems.
@impulse_255
Do you know the last version of OVMF that worked for you? I think it's probably the best shot I have at the moment, besides surgery on the GPU.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well I just tried my 680 as grid k2 without grid k2 romfile, and it works....

impulse_255

Member

Registered: 2015-06-29

Posts: 22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah, forgot about bios ... it should show up soon... you can try with no rom or with default one, it may work....

It won't, telling the VM that the hardware ID is different from what the GPU claims will only cause more problems.
@impulse_255
Do you know the last version of OVMF that worked for you? I think it's probably the best shot I have at the moment, besides surgery on the GPU.

I'm using the one linked from this post: https://bbs.archlinux.org/viewtopic.php … 8#p1518188

I have not tried any other versions yet. I used SeaBIOS before and switched to OVMF a week ago. Sadly, the sata boot feature isn't working, probably because of the crappy Marvell controller.

dRaiser

Member

From: Poland

Registered: 2013-05-20

Posts: 51

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've discovered that after launching (and powering off) Linux guest, I cannot later start Windows guest, need to restart host. Well, so long with Linux guest at this moment then.

Edit: It was different issue: machine was hanging when powering because it had problem to take integrated sound device. Pulseaudio kill is helpful here.

Last edited by dRaiser (2015-07-18 03:40:44)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Quadro cards operate as a secondary display, the GPU ROM isn't used.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

fld

Member

Registered: 2015-02-06

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I recently started using a M-Audio 1010lt PCI soundcard (via a ASMedia ASM1083 PCIe to PCI bridge) for my host audio. But ever since, I've been getting this strange DMAR flood in my dmesg whenever the sound card is being used:

[64721.567981] dmar: DRHD: handling fault status reg 3
[64721.567998] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
[64721.568080] dmar: DRHD: handling fault status reg 3
[64721.568082] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
[64721.716747] dmar: DRHD: handling fault status reg 3
[64721.716751] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
...

As I was originally only using just "intel_iommu=on,igfx_off", I thought adding the "iommu=pt" would help to get rid of this problem, because it is supposed to disable the DMAR. However, it would appear that adding the "iommu=pt" changed nothing.

I haven't noticed any other ill effects than my dmesg being flooded. I was hoping perhaps aw knows something about this?

The interesting bits from dmesg:

[    0.000000] Linux version 4.0.0-2-amd64 (debian-kernel@lists.debian.org) (gcc version 4.9.3 (Debian 4.9.3-1) ) #1 SMP Debian 4.0.8-1 (2015-07-11)
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.0.0-2-amd64 root=/dev/mapper/main-lvroot ro consoleblank=0 iommu=pt intel_iommu=on,igfx_off

[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000057fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000058000-0x0000000000058fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000059000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000006ea58fff] usable
[    0.000000] BIOS-e820: [mem 0x000000006ea59000-0x000000006ea5ffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000006ea60000-0x000000006f672fff] usable
[    0.000000] BIOS-e820: [mem 0x000000006f673000-0x000000006fb7cfff] reserved
[    0.000000] BIOS-e820: [mem 0x000000006fb7d000-0x000000008be94fff] usable
[    0.000000] BIOS-e820: [mem 0x000000008be95000-0x000000008c09afff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008c09b000-0x000000008c0dbfff] usable
[    0.000000] BIOS-e820: [mem 0x000000008c0dc000-0x000000008c187fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000008c188000-0x000000008cf84fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008cf85000-0x000000008cffefff] type 20
[    0.000000] BIOS-e820: [mem 0x000000008cfff000-0x000000008cffffff] usable
[    0.000000] BIOS-e820: [mem 0x000000008f800000-0x00000000cf9fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000042f5fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.31 by American Megatrends
[    0.000000] efi:  ACPI=0x8c164000  ACPI 2.0=0x8c164000  SMBIOS=0x8cf83498 
[    0.000000] efi: mem00: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000000000-0x0000000000058000) (0MB)
[    0.000000] efi: mem01: [Reserved           |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000058000-0x0000000000059000) (0MB)
[    0.000000] efi: mem02: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000059000-0x000000000009e000) (0MB)
[    0.000000] efi: mem03: [Reserved           |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000000009e000-0x00000000000a0000) (0MB)
[    0.000000] efi: mem04: [Loader Data        |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000100000-0x0000000000f6f000) (14MB)
[    0.000000] efi: mem05: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000000f6f000-0x00000000359c4000) (842MB)
[    0.000000] efi: mem06: [Loader Data        |   |  |  |  |   |WB|WT|WC|UC] range=[0x00000000359c4000-0x0000000036cda000) (19MB)
[    0.000000] efi: mem07: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000036cda000-0x0000000049afd000) (302MB)
[    0.000000] efi: mem08: [Loader Data        |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000049afd000-0x000000006bc9c000) (545MB)
[    0.000000] efi: mem09: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006bc9c000-0x000000006ea3b000) (45MB)
[    0.000000] efi: mem10: [Loader Code        |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006ea3b000-0x000000006ea59000) (0MB)
[    0.000000] efi: mem11: [ACPI Memory NVS    |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006ea59000-0x000000006ea60000) (0MB)
[    0.000000] efi: mem12: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006ea60000-0x000000006ebb5000) (1MB)
[    0.000000] efi: mem13: [Boot Code          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006ebb5000-0x000000006f648000) (10MB)
[    0.000000] efi: mem14: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006f648000-0x000000006f658000) (0MB)
[    0.000000] efi: mem15: [Boot Code          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006f658000-0x000000006f66a000) (0MB)
[    0.000000] efi: mem16: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006f66a000-0x000000006f673000) (0MB)
[    0.000000] efi: mem17: [Runtime Data       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000006f673000-0x000000006fb7d000) (5MB)
[    0.000000] efi: mem18: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006fb7d000-0x000000006fb8c000) (0MB)
[    0.000000] efi: mem19: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x000000006fb8c000-0x00000000885b4000) (394MB)
[    0.000000] efi: mem20: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x00000000885b4000-0x000000008894c000) (3MB)
[    0.000000] efi: mem21: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008894c000-0x0000000088a41000) (0MB)
[    0.000000] efi: mem22: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a41000-0x0000000088a62000) (0MB)
[    0.000000] efi: mem23: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a62000-0x0000000088a65000) (0MB)
[    0.000000] efi: mem24: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a65000-0x0000000088a6d000) (0MB)
[    0.000000] efi: mem25: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a6d000-0x0000000088a7a000) (0MB)
[    0.000000] efi: mem26: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a7a000-0x0000000088a8e000) (0MB)
[    0.000000] efi: mem27: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a8e000-0x0000000088a92000) (0MB)
[    0.000000] efi: mem28: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a92000-0x0000000088a99000) (0MB)
[    0.000000] efi: mem29: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a99000-0x0000000088a9a000) (0MB)
[    0.000000] efi: mem30: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088a9a000-0x0000000088c2a000) (1MB)
[    0.000000] efi: mem31: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088c2a000-0x0000000088c2f000) (0MB)
[    0.000000] efi: mem32: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088c2f000-0x0000000088dcf000) (1MB)
[    0.000000] efi: mem33: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088dcf000-0x0000000088dd1000) (0MB)
[    0.000000] efi: mem34: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088dd1000-0x0000000088dd8000) (0MB)
[    0.000000] efi: mem35: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088dd8000-0x0000000088ddb000) (0MB)
[    0.000000] efi: mem36: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088ddb000-0x0000000088de5000) (0MB)
[    0.000000] efi: mem37: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088de5000-0x0000000088de8000) (0MB)
[    0.000000] efi: mem38: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088de8000-0x0000000088dfb000) (0MB)
[    0.000000] efi: mem39: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088dfb000-0x0000000088dfc000) (0MB)
[    0.000000] efi: mem40: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000088dfc000-0x000000008acdb000) (30MB)
[    0.000000] efi: mem41: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008acdb000-0x000000008bba0000) (14MB)
[    0.000000] efi: mem42: [Boot Code          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008bba0000-0x000000008be95000) (2MB)
[    0.000000] efi: mem43: [Reserved           |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008be95000-0x000000008bf0d000) (0MB)
[    0.000000] efi: mem44: [Reserved           |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008bf0d000-0x000000008c09b000) (1MB)
[    0.000000] efi: mem45: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008c09b000-0x000000008c0d2000) (0MB)
[    0.000000] efi: mem46: [Loader Data        |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008c0d2000-0x000000008c0dc000) (0MB)
[    0.000000] efi: mem47: [ACPI Memory NVS    |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008c0dc000-0x000000008c178000) (0MB)
[    0.000000] efi: mem48: [ACPI Memory NVS    |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008c178000-0x000000008c17a000) (0MB)
[    0.000000] efi: mem49: [ACPI Memory NVS    |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008c17a000-0x000000008c188000) (0MB)
[    0.000000] efi: mem50: [Runtime Data       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008c188000-0x000000008ce7e000) (12MB)
[    0.000000] efi: mem51: [Runtime Data       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008ce7e000-0x000000008cee6000) (0MB)
[    0.000000] efi: mem52: [Runtime Data       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008cee6000-0x000000008cee8000) (0MB)
[    0.000000] efi: mem53: [Runtime Data       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008cee8000-0x000000008cf85000) (0MB)
[    0.000000] efi: mem54: [Runtime Code       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008cf85000-0x000000008cf9d000) (0MB)
[    0.000000] efi: mem55: [Runtime Code       |RUN|  |  |  |   |WB|WT|WC|UC] range=[0x000000008cf9d000-0x000000008cfff000) (0MB)
[    0.000000] efi: mem56: [Boot Data          |   |  |  |  |   |WB|WT|WC|UC] range=[0x000000008cfff000-0x000000008d000000) (0MB)
[    0.000000] efi: mem57: [Conventional Memory|   |  |  |  |   |WB|WT|WC|UC] range=[0x0000000100000000-0x000000042f600000) (13046MB)
[    0.000000] efi: mem58: [Reserved           |   |  |  |  |   |  |  |  |  ] range=[0x000000008f800000-0x00000000cfa00000) (1026MB)
[    0.000000] efi: mem59: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000f8000000-0x00000000fc000000) (64MB)
[    0.000000] efi: mem60: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000fec00000-0x00000000fec01000) (0MB)
[    0.000000] efi: mem61: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000fed00000-0x00000000fed04000) (0MB)
[    0.000000] efi: mem62: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000fed1c000-0x00000000fed20000) (0MB)
[    0.000000] efi: mem63: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000fee00000-0x00000000fee01000) (0MB)
[    0.000000] efi: mem64: [Memory Mapped I/O  |RUN|  |  |  |   |  |  |  |UC] range=[0x00000000ff000000-0x0000000100000000) (16MB)

[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000008C164000 000024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 0x000000008C164088 00008C (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: FACP 0x000000008C16EC08 00010C (v05 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 0x000000008C1641A8 00AA59 (v02 ALASKA A M I    00000240 INTL 20091112)
[    0.000000] ACPI: FACS 0x000000008C186080 000040
[    0.000000] ACPI: APIC 0x000000008C16ED18 000092 (v03 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: FPDT 0x000000008C16EDB0 000044 (v01 ALASKA A M I    01072009 AMI  00010013)
[    0.000000] ACPI: ASF! 0x000000008C16EDF8 0000A5 (v32 INTEL   HCG     00000001 TFSM 000F4240)
[    0.000000] ACPI: SSDT 0x000000008C16EEA0 000539 (v01 PmRef  Cpu0Ist  00003000 INTL 20051117)
[    0.000000] ACPI: SSDT 0x000000008C16F3E0 000AD8 (v01 PmRef  CpuPm    00003000 INTL 20051117)
[    0.000000] ACPI: SSDT 0x000000008C16FEB8 0001C7 (v01 PmRef  LakeTiny 00003000 INTL 20051117)
[    0.000000] ACPI: MCFG 0x000000008C170080 00003C (v01 ALASKA A M I    01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 0x000000008C1700C0 000038 (v01 ALASKA A M I    01072009 AMI. 00000005)
[    0.000000] ACPI: SSDT 0x000000008C1700F8 00036D (v01 SataRe SataTabl 00001000 INTL 20091112)
[    0.000000] ACPI: SSDT 0x000000008C170468 003493 (v01 SaSsdt SaSsdt   00003000 INTL 20091112)
[    0.000000] ACPI: AAFT 0x000000008C173900 000235 (v01 ALASKA OEMAAFT  01072009 MSFT 00000097)
[    0.000000] ACPI: DMAR 0x000000008C173B38 0000B8 (v01 INTEL  HSW      00000001 INTL 00000001)
[    0.000000] ACPI: Local APIC address 0xfee00000


[    0.020185] dmar: Host address width 39
[    0.020188] dmar: DRHD base: 0x000000fed90000 flags: 0x0
[    0.020195] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.020198] dmar: DRHD base: 0x000000fed91000 flags: 0x1
[    0.020202] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008020660462 ecap f010da
[    0.020204] dmar: RMRR base: 0x0000008c032000 end: 0x0000008c03efff
[    0.020207] dmar: RMRR base: 0x000000cf800000 end: 0x000000cf9fffff
[    0.020373] dmar: DRHD: handling fault status reg 3
[    0.020376] dmar: DMAR:[DMA Read] Request device [00:02.0] fault addr 8ff50000 
               DMAR:[fault reason 06] PTE Read access is not set                                               
[    0.020381] Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.020389] Enabled IRQ remapping in x2apic mode
[    0.020390] x2apic enabled
[    0.020395] Switched APIC routing to cluster x2apic.

[    0.324675] pci 0000:07:00.0: [1b21:1080] type 01 class 0x060401
[    0.325104] pci 0000:07:00.0: System wakeup disabled by ACPI
[    0.325200] pci 0000:08:01.0: [1412:1712] type 00 class 0x040100
[    0.325225] pci 0000:08:01.0: reg 0x10: [io  0xa040-0xa05f]
[    0.325237] pci 0000:08:01.0: reg 0x14: [io  0xa070-0xa07f]
[    0.325248] pci 0000:08:01.0: reg 0x18: [io  0xa060-0xa06f]
[    0.325259] pci 0000:08:01.0: reg 0x1c: [io  0xa000-0xa03f]
[    0.325344] pci 0000:08:01.0: supports D2
[    0.325433] pci 0000:07:00.0: PCI bridge to [bus 08] (subtractive decode)
[    0.325441] pci 0000:07:00.0:   bridge window [io  0xa000-0xafff]
[    0.325452] pci 0000:07:00.0:   bridge window [io  0xa000-0xafff] (subtractive decode)
[    0.325452] pci 0000:07:00.0:   bridge window [io  0x0000-0x0cf7 window] (subtractive decode)
[    0.325453] pci 0000:07:00.0:   bridge window [io  0x0d00-0xffff window] (subtractive decode)
[    0.325454] pci 0000:07:00.0:   bridge window [mem 0x000a0000-0x000bffff window] (subtractive decode)
[    0.325455] pci 0000:07:00.0:   bridge window [mem 0x000c0000-0x000c3fff window] (subtractive decode)
[    0.325456] pci 0000:07:00.0:   bridge window [mem 0x000c4000-0x000c7fff window] (subtractive decode)
[    0.325457] pci 0000:07:00.0:   bridge window [mem 0x000c8000-0x000cbfff window] (subtractive decode)
[    0.325458] pci 0000:07:00.0:   bridge window [mem 0x000cc000-0x000cffff window] (subtractive decode)
[    0.325459] pci 0000:07:00.0:   bridge window [mem 0x000d0000-0x000d3fff window] (subtractive decode)
[    0.325459] pci 0000:07:00.0:   bridge window [mem 0x000d4000-0x000d7fff window] (subtractive decode)
[    0.325460] pci 0000:07:00.0:   bridge window [mem 0x000d8000-0x000dbfff window] (subtractive decode)
[    0.325461] pci 0000:07:00.0:   bridge window [mem 0x000dc000-0x000dffff window] (subtractive decode)
[    0.325462] pci 0000:07:00.0:   bridge window [mem 0x000e0000-0x000e3fff window] (subtractive decode)
[    0.325463] pci 0000:07:00.0:   bridge window [mem 0x000e4000-0x000e7fff window] (subtractive decode)
[    0.325464] pci 0000:07:00.0:   bridge window [mem 0x000e8000-0x000ebfff window] (subtractive decode)
[    0.325465] pci 0000:07:00.0:   bridge window [mem 0x000ec000-0x000effff window] (subtractive decode)
[    0.325466] pci 0000:07:00.0:   bridge window [mem 0xcfa00000-0xfeafffff window] (subtractive decode)


[    0.597344] DMAR: No ATSR found
[    0.597406] IOMMU: dmar1 using Queued invalidation
[    0.597421] IOMMU: hardware identity mapping for device 0000:00:00.0
[    0.597423] IOMMU: hardware identity mapping for device 0000:00:01.0
[    0.597426] IOMMU: hardware identity mapping for device 0000:00:03.0
[    0.597428] IOMMU: hardware identity mapping for device 0000:00:14.0
[    0.597430] IOMMU: hardware identity mapping for device 0000:00:16.0
[    0.597432] IOMMU: hardware identity mapping for device 0000:00:19.0
[    0.597434] IOMMU: hardware identity mapping for device 0000:00:1a.0
[    0.597436] IOMMU: hardware identity mapping for device 0000:00:1b.0
[    0.597438] IOMMU: hardware identity mapping for device 0000:00:1c.0
[    0.597440] IOMMU: hardware identity mapping for device 0000:00:1c.2
[    0.597442] IOMMU: hardware identity mapping for device 0000:00:1c.3
[    0.597444] IOMMU: hardware identity mapping for device 0000:00:1c.4
[    0.597446] IOMMU: hardware identity mapping for device 0000:00:1c.5
[    0.597448] IOMMU: hardware identity mapping for device 0000:00:1c.6
[    0.597450] IOMMU: hardware identity mapping for device 0000:00:1d.0
[    0.597452] IOMMU: hardware identity mapping for device 0000:00:1f.0
[    0.597455] IOMMU: hardware identity mapping for device 0000:00:1f.2
[    0.597457] IOMMU: hardware identity mapping for device 0000:00:1f.3
[    0.597462] IOMMU: hardware identity mapping for device 0000:01:00.0
[    0.597464] IOMMU: hardware identity mapping for device 0000:01:00.1
[    0.597469] IOMMU: hardware identity mapping for device 0000:03:00.0
[    0.597475] IOMMU: hardware identity mapping for device 0000:04:00.0
[    0.597480] IOMMU: hardware identity mapping for device 0000:05:00.0
[    0.597485] IOMMU: hardware identity mapping for device 0000:06:00.0
[    0.597487] IOMMU: Setting RMRR:
[    0.597489] Ignoring identity map for HW passthrough device 0000:00:14.0 [0x8c032000 - 0x8c03efff]
[    0.597492] Ignoring identity map for HW passthrough device 0000:00:1a.0 [0x8c032000 - 0x8c03efff]
[    0.597495] Ignoring identity map for HW passthrough device 0000:00:1d.0 [0x8c032000 - 0x8c03efff]
[    0.597498] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.597499] Ignoring identity map for HW passthrough device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.597505] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O

[   34.346230] dmar: DRHD: handling fault status reg 3
[   34.346244] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.494973] dmar: DRHD: handling fault status reg 3
[   34.494976] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.643650] dmar: DRHD: handling fault status reg 3
[   34.643653] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.643738] dmar: DRHD: handling fault status reg 3
[   34.643740] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.792415] dmar: DRHD: handling fault status reg 3
[   34.792423] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.792532] dmar: DRHD: handling fault status reg 3
[   34.792534] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.941181] dmar: DRHD: handling fault status reg 3
[   34.941185] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   34.941269] dmar: DRHD: handling fault status reg 3
[   34.941272] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set                                                                                      
[   35.089969] dmar: DRHD: handling fault status reg 3
[   35.089983] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
...

lsgroups.sh

### Group 0 ###
    00:00.0 Host bridge: Intel Corporation 4th Gen Core Processor DRAM Controller (rev 06)
### Group 1 ###
    00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller (rev 06)
    01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X]
    01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
### Group 2 ###
    00:03.0 Audio device: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller (rev 06)
### Group 3 ###
    00:14.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI (rev 05)
### Group 4 ###
    00:16.0 Communication controller: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 (rev 04)
### Group 5 ###
    00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-V (rev 05)
### Group 6 ###
    00:1a.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 (rev 05)
### Group 7 ###
    00:1b.0 Audio device: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller (rev 05)
### Group 8 ###
    00:1c.0 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 (rev d5)
### Group 9 ###
    00:1c.2 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 (rev d5)
### Group 10 ###
    00:1c.3 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #4 (rev d5)
### Group 11 ###
    00:1c.4 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 (rev d5)
### Group 12 ###
    00:1c.5 PCI bridge: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #6 (rev d5)
### Group 13 ###
    00:1c.6 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d5)
    07:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03)
    08:01.0 Multimedia audio controller: VIA Technologies Inc. ICE1712 [Envy24] PCI Multi-Channel I/O Controller (rev 02)
### Group 14 ###
    00:1d.0 USB controller: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 (rev 05)
### Group 15 ###
    00:1f.0 ISA bridge: Intel Corporation Z87 Express LPC Controller (rev 05)
    00:1f.2 SATA controller: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] (rev 05)
    00:1f.3 SMBus: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller (rev 05)
### Group 16 ###
    03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
### Group 17 ###
    04:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
### Group 18 ###
    05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
### Group 19 ###
    06:00.0 USB controller: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller (rev 03)

lspci -vv:

07:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 03) (prog-if 01 [Subtractive decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Bus: primary=07, secondary=08, subordinate=08, sec-latency=32
        I/O behind bridge: 0000a000-0000afff
        Secondary status: 66MHz+ FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
        BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
                PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
        Capabilities: [c0] Subsystem: ASRock Incorporation Motherboard

08:01.0 Multimedia audio controller: VIA Technologies Inc. ICE1712 [Envy24] PCI Multi-Channel I/O Controller (rev 02)
        Subsystem: VIA Technologies Inc. M-Audio Delta 1010LT
        Control: I/O+ Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 32
        Interrupt: pin A routed to IRQ 19
        Region 0: I/O ports at a040 [size=32]
        Region 1: I/O ports at a070 [size=16]
        Region 2: I/O ports at a060 [size=16]
        Region 3: I/O ports at a000 [size=64]
        Capabilities: [80] Power Management version 1
                Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
        Kernel driver in use: snd_ice1712

impulse_255

Member

Registered: 2015-06-29

Posts: 22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Quadro cards operate as a secondary display, the GPU ROM isn't used.

Interesting. Does that mean that they do not work as a primary VGA at all, or that the ROM isn't needed when the card is secondary?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Quadro cards operate as a secondary display, the GPU ROM isn't used.

Interesting. Does that mean that they do not work as a primary VGA at all, or that the ROM isn't needed when the card is secondary?

Both.  Nvidia does not support the primary graphics use case with Quadro (nor GeForce, but nothing VM related is supported on GeForce) and in secondary mode, with the Nvidia graphics drivers, the ROM is not used.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I recently started using a M-Audio 1010lt PCI soundcard (via a ASMedia ASM1083 PCIe to PCI bridge) for my host audio. But ever since, I've been getting this strange DMAR flood in my dmesg whenever the sound card is being used:

[64721.567981] dmar: DRHD: handling fault status reg 3
[64721.567998] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
[64721.568080] dmar: DRHD: handling fault status reg 3
[64721.568082] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
[64721.716747] dmar: DRHD: handling fault status reg 3
[64721.716751] dmar: DMAR:[DMA Read] Request device [08:00.0] fault addr 10000000 
               DMAR:[fault reason 06] PTE Read access is not set
...

As I was originally only using just "intel_iommu=on,igfx_off", I thought adding the "iommu=pt" would help to get rid of this problem, because it is supposed to disable the DMAR. However, it would appear that adding the "iommu=pt" changed nothing.

I haven't noticed any other ill effects than my dmesg being flooded. I was hoping perhaps aw knows something about this?

So you have a crappy Via sound card behind a crappy ASMedia bridge and it doesn't work with the IOMMU... quelle surprise.  Actually it just seems like the audio card is buggy, it appears to be doing a DMA read at address 256MB, probably to force any previous DMA writes to memory, ie. a flush.  Is this address consistent between boots?  I'm guessing it is.  The hardware designers probably thought "hey, there's always memory at 256MB, we can just read from that for a DMA flush, what could go wrong".  Then along comes the IOMMU that restricts DMA from the device to only the regions the driver has mapped for the device and you get these sorts of errors.  Maybe the driver could try to map a buffer at 256MB, but it apparently doesn't.  All in all, the audio card isn't fit for use behind an IOMMU.

Now, why doesn't iommu=pt magically work?  Well, it's a 32bit (or less) device and in order to support a 32bit device on a system with potentially more than 4G of RAM we'd need to have bounce buffers.  But wait, we have an IOMMU, why would we also want bounce buffers?  So even though you requested passhrough mode, the sound card got demoted (you chopped this part out of the dmesg) and it's running with normal translation.

Get a better sound card.

EDIT: Oh, BTW in an interesting twist of technology, the sound card would probably work perfectly assigned to a VM because guest physical memory address 256MB will be DMA read-able.

Last edited by aw (2015-07-17 15:35:43)

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

impulse_255

Member

Registered: 2015-06-29

Posts: 22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Quadro cards operate as a secondary display, the GPU ROM isn't used.

Interesting. Does that mean that they do not work as a primary VGA at all, or that the ROM isn't needed when the card is secondary?

Both.  Nvidia does not support the primary graphics use case with Quadro (nor GeForce, but nothing VM related is supported on GeForce) and in secondary mode, with the Nvidia graphics drivers, the ROM is not used.

Okay, thanks!

I'll be back home in a couple of days. Then I will try spoofing my card as an M6000 and tell how it went.

sofloantonio

Member

Registered: 2015-07-18

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Anyone got this working with an a10-5800k or similar?

I have the a10 and a gtx750ti. I have a few potentially stupid questions.

So the aim is to have the host os (arch) be using the integrated graphics of the a10, and the windows installation to use the gtx 750. In bios, there is a setting which can switch between whether the pci graphics card is used by default or if the integrated graphics are used. Does this need to be changed? or does it have no effect?

also, im using two hdmi cords, both connected to a monitor which accepts two different inputs. one hdmi is connected to the graphics card, the other to the motherboard's hdmi slot. is this correct?

thanks

fld

Member

Registered: 2015-02-06

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So you have a crappy Via sound card behind a crappy ASMedia bridge and it doesn't work with the IOMMU... quelle surprise.  Actually it just seems like the audio card is buggy, it appears to be doing a DMA read at address 256MB, probably to force any previous DMA writes to memory, ie. a flush.

Sounds very plausible.

Is this address consistent between boots?  I'm guessing it is.

Right you are.

The hardware designers probably thought "hey, there's always memory at 256MB, we can just read from that for a DMA flush, what could go wrong".  Then along comes the IOMMU that restricts DMA from the device to only the regions the driver has mapped for the device and you get these sorts of errors.  Maybe the driver could try to map a buffer at 256MB, but it apparently doesn't.

So the the hardware doesn't define a memory BAR at 256M, even though it tries to read it. Could it be possible to hack the snd_ice1712 driver to reserve that memory? Or is it too late to try and "fix" it in the driver?

All in all, the audio card isn't fit for use behind an IOMMU.

Strange that the card still does appear to "fully work". The act of doing a DMA Read at a fault address must still be achieving the flush?

Now, why doesn't iommu=pt magically work?  Well, it's a 32bit (or less) device and in order to support a 32bit device on a system with potentially more than 4G of RAM we'd need to have bounce buffers.  But wait, we have an IOMMU, why would we also want bounce buffers?

Then passthrough will only work for devices that support 64 bit addressing?

  So even though you requested passhrough mode, the sound card got demoted (you chopped this part out of the dmesg) and it's running with normal translation.

Yeah, sorry about cutting away important stuff, I was trying to make it easier to read. Here is the full dmesg: http://paste.debian.net/hidden/f5743abd/

Get a better sound card.

Yeah, perhaps I'll get one of those simple USB sound cards. I'm guessing they are fairly usable by now. Only reason I'm still using the VIA sound card is because it can directly drive my headphones really nicely without any sort of background hiss or noise.

But in the meanwhile, I'm thinking ill just figure out a way for syslog to suppress those DMAR messages. Since those messages filling up my log seem to be the only bad thing that is happening.

evujumenuk

Member

Registered: 2013-07-02

Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Borderlands folks, I think this is the problem you're trying to describe:

https://dl.dropboxusercontent.com/u/198 … emetal.jpg

https://dl.dropboxusercontent.com/u/198 … ds2-vm.jpg

This vantage point in the training area seems to give me a consistently low FPS and high latency and seems invariant to the lighting.  The VM shows about 1/3rd the FPS and 3x the latency.  What is that latency actually measuring anyway?  These are separate win7x64 installs on the same Haswell/GTX660 hardware with Nvidia Experience optimizing the game in both (highest settings for all).

I don't have any solutions yet, but it's good to understand the problem.  I did run the VM with vfio debugging turned on and I'm not seeing any excessive hits to the card, which means we're not bouncing out to userspace regularly as a result of the assigned devices (GPU + audio + NIC in this case).  I see a write to 0x88154 at a rate of about 10/s.  This is an area we must trap because it's an mmio mirror of PCI config space.  That happens regardless of what's going on in the game and also happens at a basic desktop, so if it was causing any trouble I'd expect it would be across the board and not select games.

I'm suspicious of USB passthrough, but my only evidence is that hitting the escape key in the game is not as responsive as bare metal and often requires a second hit.  Assignment of the USB controller might be an option, but didn't just work in my first attempt, maybe an issue with windows drivers.

EDIT: game reported latency appears to just be the inverse of FPS, so it's SPF (seconds per frame)

EDIT2: Further analysis (special thanks to Paolo Bonzini) shows two issues that are hurting performance here.  First is a known issue with Windows that it pounds on the time source, whether it's ACPI PM timer or HPET.  Code just recently went upstream (kernel and qemu) that enables a Hyper-V enlightened timer that can help to reduce this.  To enable it add "hv-time" to your -cpu option, ex. -cpu Haswell,hv-time  This gives me about 20% more FPS and is likely to help performance of any game running in a Windows guest.  The second problem is that we see extensive use of the processor debug registers.  Each move to or from these registers traps into the hypvervisor and they appear to be batched into a context switch function, so we have a burst of them all at once.  I've submitted a support ticket to gearbox software to see if they have any suggestions to avoid these registers.

Hello performance!  Patches were just posted upstream to do lazy tracking of the debug registers.  With "-cpu Haswell,hv-time" and these patches, this is what I can get now (better than 80% of bare metal):

https://dl.dropboxusercontent.com/u/198 … vm-new.jpg

This initial implementation is Intel-only, but hopefully we can do the same for AMD.

So, what is the best current alternative to hv-time? Can't really use that with Nvidia drivers.

/USE PMTIMER?

I just thought of something.

Could one enable Invariant TSC on the -cpu flag and get a clock as good as hypervclock? The catch would be that savevm (and, by extension, migration) would not work. But I suspect that is already the case with VFIO.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So the the hardware doesn't define a memory BAR at 256M, even though it tries to read it. Could it be possible to hack the snd_ice1712 driver to reserve that memory? Or is it too late to try and "fix" it in the driver?

This isn't related to BARs, this is how the device accesses memory.  It should be possible to hack the driver to create a coherent mapping at 256MB, it appears that dma_declare_coherent_memory() would be the mechanism to do such a thing.  I also note this driver sets a coherent DMA mask of 28 bits, ie. 256MB, which is a bit at odds with declaring coherent memory starting at 256MB and an indication of just how outdated this hardware is.

Strange that the card still does appear to "fully work". The act of doing a DMA Read at a fault address must still be achieving the flush?

Potentially

Then passthrough will only work for devices that support 64 bit addressing?

64bit is not strictly required, but the device must be capable of DMA'ing to any physical memory in the system.  However, conventional PCI devices like this are a special case.  Only PCIe devices use a unique requester ID.  For conventional PCI devices, the PCIe-to-PCI bridge provides the requester ID, this is why the error reports 08:00.0 rather than 08:01.0 because that's the requester ID for the entire secondary side of the bridge.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

morat

Member

Registered: 2015-07-07

Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Well, everything works now, which is nice; everything remaining is a matter of convenience. My problem was eventually solved by using the OVMF binaries linked by @impulse_255, so there must be an issue in the newest version included in the edk.
I do have a couple of questions though:
1. I'm using a seperate physical interface for networking, a PCI wifi card, with the host using onboard Ethernet; but this means for slower and higher-latency transfer between the two, what is the simplest host-to-guest networking, without external connectivity? I thought it would be NAT, but documentation seems to suggest that it is for guest to guest connectivity. Bridging doesn't seem to make h-t-g networking a possibility, so how can I do it? EDIT: I'm terrible at reading.
2. Is there a better alternative than Synergy for gaming? Hitting the edges is a problem, and it seems to handle direct pointer captures very badly, so besides Steam Inhouse Streaming, is there a viable way to play D3D games?

Thanks for all your help getting me this far, and if anyone with similar issues to mine needs any help, I'd be more than happy.

Last edited by morat (2015-07-19 03:02:59)

wmarler

Member

From: Denver, CO

Registered: 2015-04-18

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hey forum,

I'm running into a wall. I'm using virt-manager to prepare a VM for VGA passthrough. When a VM is configured to use UEFI firmware, Windows will not install, but fails on "Copying Windows files (0%)". In the case of Win8.1/10, Windows throws up the frowny-face blue-screen with a generic rebooting message and reboots, and Win 7 freezes with a black screen. On reboot & trying to install Windows again, the disk has been partitioned (4 partitions with Win 8.1/10, 3 with 7), but the main/non-system partition cannot be installed to without formatting. The details reads "... This hard disk space is formatted with an unsupported version of the NTFS file system." The disk can be re-formatted and selected as the install target, but doing so results in the same reboot/black screen. Creating/formatting partitions manually results in the crash/black screen too. I've tried many, many iterations. I finally created 2 VM's identical to each other save for uuid, LVM storage location, and NIC MAC address, gave one the UEFI firmware ("win10b") and the other the BIOS firmware ("win10a"), and installed the BIOS one just fine.

My conclusion: there's something up with the UEFI firmware and/or my motherboard. I have no insight into what, nor how to get any insight. Watching 'journalctl -f' during the crash shows nothing, and I don't know how to get Windows to tell me anything.

I am using the UEFI firmware as provided by the Gerd Hoffman RPM repository. The version is edk2.git-ovmf-x64.noarch 0-20150716.b1120.g387e7c1
I'm on Fedora 22, kernel 4.0.8-300, with an i5-4960 CPU, and an ASRock Z87 PRO3 motherboard (no successes and 1 failure reported in the spreadsheet; disheartening, but not conclusive). The motherboard BIOS is up to date, according to the motherboard update utility.

A dumpxml of the working VM is here (win10a.xml).
A dumpxml of the non-working VM is here (win10b.xml).

The diff between these two dumps is here:

% diff win10a.xml win10b.xml 
2,4c2,4
<   <name>win10a</name>
<   <uuid>b80ea20a-5157-4afd-87f9-63b1e2a5b1bf</uuid>
<   <description>BIOS Firmware</description>
---
>   <name>win10b</name>
>   <uuid>c8c14fba-ecdd-4d91-ae7c-a9f52497d1b2</uuid>
>   <description>UEFI Firmware</description>
9a10
>     <loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
40c41
<       <source dev='/dev/vgmain/win10_1'/>
---
>       <source dev='/dev/vgmain/win10_2'/>
86c87
<       <mac address='52:54:00:e7:67:f7'/>
---
>       <mac address='52:54:00:c8:1b:45'/>

The issue is easily reproducible. 

Any suggestions other than "buy a known-good motherboard" or "use the VGA arbitration patch"? Those are obvious next steps, but neither are solutions I'm particularly interested in without exhausting all of my options.

TIA

Last edited by wmarler (2015-07-19 04:47:43)

morat

Member

Registered: 2015-07-07

Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@wmarler, I just had this exact issue:
Use this set of binaries, as linked by @impulse_255
I'd say this will probably work for you, as yours was my exact issue.
Win 7 will not work, it seems to have UEFI trouble.

Also, please code tag your diffs, and other code.

Last edited by morat (2015-07-19 04:45:24)

wmarler

Member

From: Denver, CO

Registered: 2015-04-18

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@wmarler, I just had this exact issue:
Use this set of binaries, as linked by @impulse_255
I'd say this will probably work for you, as yours was my exact issue.

Thanks, that worked. And was very easy :-D.

Win 7 will not work, it seems to have UEFI trouble.

Bummer ... that's what I have a license for atm.

Nazfellun

Member

Registered: 2012-10-22

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

2. Is there a better alternative than Synergy for gaming? Hitting the edges is a problem, and it seems to handle direct pointer captures very badly, so besides Steam Inhouse Streaming, is there a viable way to play D3D games?

Thanks for all your help getting me this far, and if anyone with similar issues to mine needs any help, I'd be more than happy.

If what you're referring to by "seems to handle direct pointer captures very badly" is what I think, ensure you have 'relativeMouseMoves = true' in your server config, bind a button to lockCursorToScreen(toggle), and then lock the cursor to the vm screen when you're playing, should solve that issue and also your edge problem.