Arch Linux

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Can someone give a few pointer what benefits OVMF would give? I understand that it is basically providing an UEFI Bios to the VM, enabling stuff like secureboot etc. But if I have a working setup with the old Seabios - will using OVMF give me any advantage?

i wonder if you have actually read/understood anything at all...

Well, probably not enough :-)

it's gpu passthrough we are talking about here - not virtualising uefi boot process in general.
the core advantage ovmf gives is no vga legacy mess (kernel patching, etc).

p.s. i may be wrong here, but i really doubt anyone cares about secureboot (or any other weird stuff) when running 3d-accelerated vm.

Yes, this is also what I care about. Right now I have my VM booting & working flawlessly with my passed-through vga card, without applying any patches whatsoever to kernel, qemu or SeaBios. Just using what is in the arch stable repository. I just want to understand if i would have any benefit setting up my VM using OVMF, like increased performance, better stability (I did not have a crash for several weeks now though).

So what is the vga legacy mess you are refering to? We don't need to discuss this here, just having some pointers to read would be awesome

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So what is the vga legacy mess you are refering to? We don't need to discuss this here, just having some pointers to read would be awesome

http://vfio.blogspot.com/2014/08/whats- … ation.html

If you're not affected by i915's VGA arbiter behavior and don't care for libvirt support, then just keep doing what you're doing.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I was researching that too, if anyone can confirm that the HD7790 has the reset problem, it might be affordable enough that I could pick one up for testing.

Yes, the HD7790 has the reset problem.

Well, FWIW I now own an XFX HD7790 card.  All I can confirm so far is that it's quite broken.  VM reboots work well with the standard Windows VGA driver, but once the Catalyst drivers are loaded a reboot results in a PAGE_FAULT_IN_NONPAGED_AREA BSOD and a slew of IOMMU faults and stack traces (AMD 990fx host).  If I replace the HD7790 with an HD7850, everything works well.  I'm also a little bummed that the UEFI ROM on this card hangs, both bare metal and VM, hopefully XFX can point me to an update.  Anyway, no clues yet and not a lot of time to dig into it lately, but I share your misery.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

anickname

Member

Registered: 2014-01-07

Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I was researching that too, if anyone can confirm that the HD7790 has the reset problem, it might be affordable enough that I could pick one up for testing.

Yes, the HD7790 has the reset problem.

Well, FWIW I now own an XFX HD7790 card.  All I can confirm so far is that it's quite broken.  VM reboots work well with the standard Windows VGA driver, but once the Catalyst drivers are loaded a reboot results in a PAGE_FAULT_IN_NONPAGED_AREA BSOD and a slew of IOMMU faults and stack traces (AMD 990fx host).  If I replace the HD7790 with an HD7850, everything works well.  I'm also a little bummed that the UEFI ROM on this card hangs, both bare metal and VM, hopefully XFX can point me to an update.  Anyway, no clues yet and not a lot of time to dig into it lately, but I share your misery.

Thank you, for your time and effort.

AKSN74

Member

From: Taichung, Taiwan

Registered: 2014-02-18

Posts: 52

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello, guys, LTNS.

I got so long that about 2 months not follow this topic. And looks like it still powerful.

Now I'm trying New Linux 3.17.0-rc6 with new qemu-git version. And try to solve hangs while guest Linux rebooting. (Linux shutdown, but VM can't reboot. Strange is, it not happened while guest OS is Windows.)
Unfortunately it still doesn't work. For now I have to shutdown VM and reboot Host if guest Linux need to reboot.

Does anyone got solutions?

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

http://vfio.blogspot.com/2014/08/whats- … ation.html

If you're not affected by i915's VGA arbiter behavior and don't care for libvirt support, then just keep doing what you're doing.

Well, I do use libvirt and virt-manager. Are you saying with OVMF I could setup the VM using libvirt without needing to pass additional arguments to the qemu-commandline (stuff like "<qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>")? That would be awesome!

Edit: Just found your blog entry http://vfio.blogspot.de/2014/08/primary … t-vga.html - Now this really looks interesting! Thanks

Last edited by TripleSpeeder (2014-09-25 08:29:19)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello, guys, LTNS.

I got so long that about 2 months not follow this topic. And looks like it still powerful.

Now I'm trying New Linux 3.17.0-rc6 with new qemu-git version. And try to solve hangs while guest Linux rebooting. (Linux shutdown, but VM can't reboot. Strange is, it not happened while guest OS is Windows.)
Unfortunately it still doesn't work. For now I have to shutdown VM and reboot Host if guest Linux need to reboot.

Does anyone got solutions?

Seems like your VGA doesn't have reset capabilities, or it is working not quite right, and linux can't reset it.
You can try:
doing pm-suspend and then wake the system - it'll be faster than a host reboot, while resetting the GPU hardware;
passing through only the VGA itself, without HDMI Audio(BDF: XX:00.1);
probably getting OVMF working would help, but i doubt it.

Some cards are broken - see https://bbs.archlinux.org/viewtopic.php … 6#p1459916 , some cards work like a charm, depends on the vendor(XFX?) and model.

ru?..

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

reper

Member

Registered: 2013-07-26

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

hi guys,
i was thinking of making a new pc with vga passthrough in mind, and i have some questions:
1) are the Asrock B85M Pro4 and the i5 4670 a good combination? if i'm not wrong they both support VT-D right? Also, i've never bought an asrock, are they good?
2) has someone had success passing one of the new nvidia's card?

i'm still a noob regarding virtualization and i fear to make a mistake.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone , first post!

This woks nicely on my ASUS X99-Deluxe + i7-5930k , passing through 1 nVidia GT610 + 1 GTX770 to two seperate VMs . You wouldn't need to patch the kernel , unless you want to passthrough the SATA controller OR the Wireless+Bluetooth controller .

Anyway , this board's back panel's USB ports are connected to a single USB 3.0 controller , so using PCI-Passthrough for USB isn't possible in my case . However , I have USB Hubs I want to use for each VM .

Is it possible to passthrough the Hub alone and thus every USB device that gets connected to it ?

Thank you .

EDIT :

lspci | grep USB :

00:14.0 USB controller: Intel Corporation Wellsburg USB xHCI Host Controller (rev 05)
00:1a.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05)
00:1d.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #1 (rev 05)
0e:00.0 USB controller: ASMedia Technology Inc. Device 1142

I tried passthing-through every controller (one at a time) to a VM , they don't work . However when passing the 00:14.0 controller , all the USB ports in the back get passed-through .

Last edited by Denso (2014-09-27 23:36:01)

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have had similar situation and had to pass

00:1a.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #2 (rev 05)
or
00:1d.0 USB controller: Intel Corporation Wellsburg USB Enhanced Host Controller #1 (rev 05)

that are connected to front usb on mbo, only that worked. Passing hubs isn't possible i think.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Wow , that makes perfect sense ...

This mobo has 2 front USB 3.0 headers , I suppose 00:1a.0 is for one and 00:1d.0 is for the other ?

Please do clarify ...

Thank you in advance !

AKSN74

Member

From: Taichung, Taiwan

Registered: 2014-02-18

Posts: 52

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello, guys, LTNS.

I got so long that about 2 months not follow this topic. And looks like it still powerful.

Now I'm trying New Linux 3.17.0-rc6 with new qemu-git version. And try to solve hangs while guest Linux rebooting. (Linux shutdown, but VM can't reboot. Strange is, it not happened while guest OS is Windows.)
Unfortunately it still doesn't work. For now I have to shutdown VM and reboot Host if guest Linux need to reboot.

Does anyone got solutions?

Seems like your VGA doesn't have reset capabilities, or it is working not quite right, and linux can't reset it.
You can try:
doing pm-suspend and then wake the system - it'll be faster than a host reboot, while resetting the GPU hardware;
passing through only the VGA itself, without HDMI Audio(BDF: XX:00.1);
probably getting OVMF working would help, but i doubt it.

Some cards are broken - see https://bbs.archlinux.org/viewtopic.php … 6#p1459916 , some cards work like a charm, depends on the vendor(XFX?) and model.

ru?..

Thanks for your reply.

Because I'm using NVIDIA 'official' GTX480 (that means vendor is NVIDIA, not others like XFX, ASUS, etc.)
And unfortunately, it will get error if passing through VGA only.

Besides, GTX480 is not using UEFI BIOS, so I guess OVMF is not working.
But I'll try again.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Wow , that makes perfect sense ...

This mobo has 2 front USB 3.0 headers , I suppose 00:1a.0 is for one and 00:1d.0 is for the other ?

Please do clarify ...

Thank you in advance !

well i don't have that mbo i got z77, but that should be correct... u need to try it 
(u may have 2.0 front and/or 3.0 it depends on mbo)

Last edited by slis (2014-09-28 07:57:24)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Because I'm using NVIDIA 'official' GTX480 (that means vendor is NVIDIA, not others like XFX, ASUS, etc.)
And unfortunately, it will get error if passing through VGA only.

Besides, GTX480 is not using UEFI BIOS, so I guess OVMF is not working.
But I'll try again.

Reference GTX480...
Well, i can't tell you anything more helpful.
You can do pm-suspend-wake-cycle every reboot as a workaround, i guess. Or you could experiment with guest OS - you've said you have linux as a guest.
Trying to poke nvidia support is useless, they'll say "go eat your KVM yourself" just like they did it in their latest drivers.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Wow , that makes perfect sense ...

This mobo has 2 front USB 3.0 headers , I suppose 00:1a.0 is for one and 00:1d.0 is for the other ?

Please do clarify ...

Thank you in advance !

well i don't have that mbo i got z77, but that should be correct... u need to try it 
(u may have 2.0 front and/or 3.0 it depends on mbo)

Hi .

I can confirm that the 2 USB 3.0 headers are also passed-through with the main controller (00:14.0) .

So I think I might need a PCI-E USB 3.0 card for this .

I hope there would be a way to passthrough individual hubs instead of controllers .

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

what about 2.0? do u have 2.0 headers on mbo? try them?

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

what about 2.0? do u have 2.0 headers on mbo? try them?

Oops ! I ignored them completely !

Will be back with results soon !

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here's something odd I've just encountered:
I have two motherboards: a SuperMicro X10SL7-F, and a SuperMicro X10SAT.
I  have two video cards I can pass through: an AMD Radeon R9 290, and an AMD Radeon E6760.

On the X1SL7-F, I've passed through the E6760 just fine -- no DMAR errors, and no bus-reset issues.
On the X10SAT, I've passed through the R9 290, and it works, aside from the annoying bus-reset issues.

I just replaced the R9 290 with the E6760, and started getting DMAR errors for both the GPU and the XHCI controller (one I've successfully passed through before).

I managed to get the errors to stop by detaching the HDMI audio and plugging a monitor into the second card.  (I'm not certain which of those two changes actually fixed it.)

I wonder if the card's UEFI firmware freaks out when there's no monitor attached?

EDIT: wait, maybe it's not fixed.  More experimentation to follow...

Last edited by DanaGoyette (2014-09-29 04:45:44)

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here's something odd I've just encountered:
I have two motherboards: a SuperMicro X10SL7-F, and a SuperMicro X10SAT.
I  have two video cards I can pass through: an AMD Radeon R9 290, and an AMD Radeon E6760.

On the X1SL7-F, I've passed through the E6760 just fine -- no DMAR errors, and no bus-reset issues.
On the X10SAT, I've passed through the R9 290, and it works, aside from the annoying bus-reset issues.

I just replaced the R9 290 with the E6760, and started getting DMAR errors for both the GPU and the XHCI controller (one I've successfully passed through before).

I managed to get the errors to stop by detaching the HDMI audio and plugging a monitor into the second card.  (I'm not certain which of those two changes actually fixed it.)

I wonder if the card's UEFI firmware freaks out when there's no monitor attached?

EDIT: wait, maybe it's not fixed.  More experimentation to follow...

Hey .

I can confirm that I get a LOT of DMAR errors on my X99-Deluxe too when using VFIO . I doesn't happen when I use "-device kvm-pci-assign" for onboard audio and USB though . Also , when I use this method , I can't reboot Windows without rebooting the whole host , otherwise it crashes the host .

Last edited by Denso (2014-09-29 05:20:36)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here are the errors I'm getting, approximately one per OVMF screen redraw (being driven via serial interface in virt-manager):

[  330.024356] xhci_hcd 0000:08:00.0: remove, state 4
[  330.026932] usb usb4: USB disconnect, device number 1
[  330.037696] xhci_hcd 0000:08:00.0: USB bus 4 deregistered
[  330.041890] xhci_hcd 0000:08:00.0: remove, state 1
[  330.044007] usb usb3: USB disconnect, device number 1
[  330.046225] usb 3-2: USB disconnect, device number 3
[  330.076301] xhci_hcd 0000:08:00.0: USB bus 3 deregistered
[  330.908248] vfio-pci 0000:01:00.0: enabling device (0140 -> 0143)
[  335.721419] dmar: DRHD: handling fault status reg 3
[  335.723856] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  335.723856] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.181491] dmar: DRHD: handling fault status reg 3
[  336.183706] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be448000 
[  336.183706] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.189725] dmar: DRHD: handling fault status reg 3
[  336.191928] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be448000 
[  336.191928] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.198007] dmar: DRHD: handling fault status reg 3
[  336.200199] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be437000 
[  336.200199] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.206293] dmar: DRHD: handling fault status reg 3
[  336.208468] dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr bff66000 
[  336.208468] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.360197] dmar: DRHD: handling fault status reg 3
[  336.361816] dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr bff66000 
[  336.361816] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.367985] dmar: DRHD: handling fault status reg 3
[  336.370762] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be436000 
[  336.370762] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.376827] dmar: DRHD: handling fault status reg 3
[  336.378948] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be436000 
[  336.378948] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  337.518236] dmar: DRHD: handling fault status reg 3
[  337.520281] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  337.520281] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  338.346795] dmar: DRHD: handling fault status reg 3
[  338.349327] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  338.349327] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  338.960278] dmar: DRHD: handling fault status reg 3
[  338.962519] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be27a000 
[  338.962519] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  338.968614] dmar: DRHD: handling fault status reg 3
[  338.970776] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be27a000 
[  338.970776] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  339.097760] dmar: DRHD: handling fault status reg 3
[  339.100430] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  339.100430] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  339.715753] dmar: DRHD: handling fault status reg 3
[  339.719830] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  339.719830] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.748928] dmar: DRHD: handling fault status reg 3
[  340.751168] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.751168] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.757762] dmar: DRHD: handling fault status reg 3
[  340.760452] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.760452] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.766677] dmar: DRHD: handling fault status reg 3
[  340.768720] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.768720] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.775453] dmar: DRHD: handling fault status reg 3
[  340.777950] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.777950] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.784387] dmar: DRHD: handling fault status reg 3
[  340.787213] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.787213] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.798013] dmar: DRHD: handling fault status reg 3
[  340.801504] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0ec000 
[  340.801504] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.807641] dmar: DRHD: handling fault status reg 3
[  340.809757] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0ec000 
[  340.809757] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.816206] dmar: DRHD: handling fault status reg 3
[  340.818295] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.818295] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.825173] dmar: DRHD: handling fault status reg 3
[  340.827588] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0df000 
[  340.827588] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.833662] dmar: DRHD: handling fault status reg 3
[  340.835817] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be3c5000 
[  340.835817] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.842135] dmar: DRHD: handling fault status reg 3
[  340.845125] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0dd000 
[  340.845125] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.851955] dmar: DRHD: handling fault status reg 3
[  340.854387] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0dd000 
[  340.854387] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.860749] dmar: DRHD: handling fault status reg 3
[  340.863658] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.863658] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.869889] dmar: DRHD: handling fault status reg 3
[  340.871905] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f6000 
[  340.871905] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.878694] dmar: DRHD: handling fault status reg 3
[  340.881421] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f2000 
[  340.881421] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.887509] dmar: DRHD: handling fault status reg 3
[  340.889685] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.889685] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.896034] dmar: DRHD: handling fault status reg 3
[  340.898945] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.898945] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.905864] dmar: DRHD: handling fault status reg 3
[  340.907925] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.907925] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.914463] dmar: DRHD: handling fault status reg 3
[  340.916103] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.916103] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.922333] dmar: DRHD: handling fault status reg 3
[  340.924267] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.924267] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.930355] dmar: DRHD: handling fault status reg 3
[  340.932667] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.932667] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.938511] dmar: DRHD: handling fault status reg 3
[  340.938516] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.938516] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.938525] dmar: DRHD: handling fault status reg 3
[  340.938531] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.938531] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.938641] dmar: DRHD: handling fault status reg 3
[  340.938648] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.938648] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.938664] dmar: DRHD: handling fault status reg 3
[  340.938672] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.938672] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941760] dmar: DRHD: handling fault status reg 3
[  340.941773] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.941773] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941778] dmar: DRHD: handling fault status reg 3
[  340.941787] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.941787] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941794] dmar: DRHD: handling fault status reg 3
[  340.941799] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.941799] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941815] dmar: DRHD: handling fault status reg 3
[  340.941821] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.941821] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941832] dmar: DRHD: handling fault status reg 3
[  340.941835] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.941835] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.941847] dmar: DRHD: handling fault status reg 3
[  340.941861] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.941861] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.947580] dmar: DRHD: handling fault status reg 3
[  340.947583] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.947583] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.947599] dmar: DRHD: handling fault status reg 3
[  340.947604] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.947604] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.947697] dmar: DRHD: handling fault status reg 3
[  340.947704] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.947704] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.947713] dmar: DRHD: handling fault status reg 3
[  340.947719] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.947719] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.947727] dmar: DRHD: handling fault status reg 3
[  340.947734] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.947734] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.949772] dmar: DRHD: handling fault status reg 3
[  340.949774] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.949774] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.950856] dmar: DRHD: handling fault status reg 3
[  340.950863] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.950863] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.950875] dmar: DRHD: handling fault status reg 3
[  340.950880] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.950880] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.950889] dmar: DRHD: handling fault status reg 3
[  340.950893] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.950893] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.955757] dmar: DRHD: handling fault status reg 3
[  340.955759] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.955759] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.956078] dmar: DRHD: handling fault status reg 3
[  340.956086] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.956086] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.957893] dmar: DRHD: handling fault status reg 3
[  340.957894] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.957894] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958259] dmar: DRHD: handling fault status reg 3
[  340.958266] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958266] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958280] dmar: DRHD: handling fault status reg 3
[  340.958288] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958288] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958297] dmar: DRHD: handling fault status reg 3
[  340.958302] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958302] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958311] dmar: DRHD: handling fault status reg 3
[  340.958317] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958317] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958329] dmar: DRHD: handling fault status reg 3
[  340.958335] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958335] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958342] dmar: DRHD: handling fault status reg 3
[  340.958348] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.958348] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958357] dmar: DRHD: handling fault status reg 3
[  340.958363] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958363] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958374] dmar: DRHD: handling fault status reg 3
[  340.958379] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958379] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958389] dmar: DRHD: handling fault status reg 3
[  340.958396] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958396] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958406] dmar: DRHD: handling fault status reg 3
[  340.958411] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958411] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958419] dmar: DRHD: handling fault status reg 3
[  340.958425] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958425] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958437] dmar: DRHD: handling fault status reg 3
[  340.958443] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958443] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958450] dmar: DRHD: handling fault status reg 3
[  340.958456] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.958456] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958466] dmar: DRHD: handling fault status reg 3
[  340.958472] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.958472] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958483] dmar: DRHD: handling fault status reg 3
[  340.958487] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.958487] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958498] dmar: DRHD: handling fault status reg 3
[  340.958505] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.958505] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958512] dmar: DRHD: handling fault status reg 3
[  340.958518] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.958518] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958529] dmar: DRHD: handling fault status reg 3
[  340.958536] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.958536] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958630] dmar: DRHD: handling fault status reg 3
[  340.958637] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.958637] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.958646] dmar: DRHD: handling fault status reg 3
[  340.958651] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.958651] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.963963] dmar: DRHD: handling fault status reg 3
[  340.963965] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.963965] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.966049] dmar: DRHD: handling fault status reg 3
[  340.966051] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.966051] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.966266] dmar: DRHD: handling fault status reg 3
[  340.966273] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.966273] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972292] dmar: DRHD: handling fault status reg 3
[  340.972296] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.972296] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972311] dmar: DRHD: handling fault status reg 3
[  340.972317] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972317] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972323] dmar: DRHD: handling fault status reg 3
[  340.972330] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972330] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972339] dmar: DRHD: handling fault status reg 3
[  340.972345] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972345] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972356] dmar: DRHD: handling fault status reg 3
[  340.972363] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972363] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972370] dmar: DRHD: handling fault status reg 3
[  340.972378] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972378] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972387] dmar: DRHD: handling fault status reg 3
[  340.972391] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972391] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972402] dmar: DRHD: handling fault status reg 3
[  340.972407] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972407] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972420] dmar: DRHD: handling fault status reg 3
[  340.972425] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.972425] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972435] dmar: DRHD: handling fault status reg 3
[  340.972440] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.972440] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972451] dmar: DRHD: handling fault status reg 3
[  340.972463] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.972463] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.972469] dmar: DRHD: handling fault status reg 3
[  340.972473] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.972473] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.974249] dmar: DRHD: handling fault status reg 3
[  340.974250] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.974250] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980331] dmar: DRHD: handling fault status reg 3
[  340.980335] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.980335] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980352] dmar: DRHD: handling fault status reg 3
[  340.980359] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980359] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980366] dmar: DRHD: handling fault status reg 3
[  340.980370] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980370] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980381] dmar: DRHD: handling fault status reg 3
[  340.980386] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980386] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980395] dmar: DRHD: handling fault status reg 3
[  340.980401] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980401] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980411] dmar: DRHD: handling fault status reg 3
[  340.980416] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980416] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980425] dmar: DRHD: handling fault status reg 3
[  340.980430] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980430] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980441] dmar: DRHD: handling fault status reg 3
[  340.980446] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980446] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980456] dmar: DRHD: handling fault status reg 3
[  340.980461] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980461] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980472] dmar: DRHD: handling fault status reg 3
[  340.980476] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980476] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980488] dmar: DRHD: handling fault status reg 3
[  340.980492] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980492] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980504] dmar: DRHD: handling fault status reg 3
[  340.980510] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980510] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980518] dmar: DRHD: handling fault status reg 3
[  340.980524] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  340.980524] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980892] dmar: DRHD: handling fault status reg 3
[  340.980897] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980897] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980906] dmar: DRHD: handling fault status reg 3
[  340.980912] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980912] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980923] dmar: DRHD: handling fault status reg 3
[  340.980929] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980929] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980939] dmar: DRHD: handling fault status reg 3
[  340.980945] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980945] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980956] dmar: DRHD: handling fault status reg 3
[  340.980961] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980961] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980972] dmar: DRHD: handling fault status reg 3
[  340.980977] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980977] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.980988] dmar: DRHD: handling fault status reg 3
[  340.980994] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.980994] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981005] dmar: DRHD: handling fault status reg 3
[  340.981011] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981011] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981021] dmar: DRHD: handling fault status reg 3
[  340.981026] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981026] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981035] dmar: DRHD: handling fault status reg 3
[  340.981042] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981042] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981052] dmar: DRHD: handling fault status reg 3
[  340.981058] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981058] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981068] dmar: DRHD: handling fault status reg 3
[  340.981074] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981074] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981085] dmar: DRHD: handling fault status reg 3
[  340.981090] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981090] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981099] dmar: DRHD: handling fault status reg 3
[  340.981105] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981105] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981115] dmar: DRHD: handling fault status reg 3
[  340.981120] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.981120] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981129] dmar: DRHD: handling fault status reg 3
[  340.981136] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.981136] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981146] dmar: DRHD: handling fault status reg 3
[  340.981151] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.981151] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981160] dmar: DRHD: handling fault status reg 3
[  340.981166] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.981166] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981177] dmar: DRHD: handling fault status reg 3
[  340.981182] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  340.981182] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981272] dmar: DRHD: handling fault status reg 3
[  340.981279] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981279] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981286] dmar: DRHD: handling fault status reg 3
[  340.981293] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981293] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981302] dmar: DRHD: handling fault status reg 3
[  340.981309] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981309] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981318] dmar: DRHD: handling fault status reg 3
[  340.981323] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981323] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981333] dmar: DRHD: handling fault status reg 3
[  340.981339] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981339] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981349] dmar: DRHD: handling fault status reg 3
[  340.981354] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981354] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981363] dmar: DRHD: handling fault status reg 3
[  340.981370] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981370] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981378] dmar: DRHD: handling fault status reg 3
[  340.981385] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981385] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981395] dmar: DRHD: handling fault status reg 3
[  340.981400] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981400] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981410] dmar: DRHD: handling fault status reg 3
[  340.981416] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981416] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981426] dmar: DRHD: handling fault status reg 3
[  340.981431] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981431] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981441] dmar: DRHD: handling fault status reg 3
[  340.981447] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981447] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981456] dmar: DRHD: handling fault status reg 3
[  340.981462] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981462] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981472] dmar: DRHD: handling fault status reg 3
[  340.981478] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981478] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981491] dmar: DRHD: handling fault status reg 3
[  340.981497] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981497] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981504] dmar: DRHD: handling fault status reg 3
[  340.981509] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981509] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981519] dmar: DRHD: handling fault status reg 3
[  340.981525] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981525] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981536] dmar: DRHD: handling fault status reg 3
[  340.981542] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981542] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981552] dmar: DRHD: handling fault status reg 3
[  340.981558] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fc000 
[  340.981558] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981784] dmar: DRHD: handling fault status reg 3
[  340.981790] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981790] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981797] dmar: DRHD: handling fault status reg 3
[  340.981803] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981803] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981812] dmar: DRHD: handling fault status reg 3
[  340.981818] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981818] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981830] dmar: DRHD: handling fault status reg 3
[  340.981835] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981835] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981844] dmar: DRHD: handling fault status reg 3
[  340.981851] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981851] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981857] dmar: DRHD: handling fault status reg 3
[  340.981862] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981862] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981873] dmar: DRHD: handling fault status reg 3
[  340.981879] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981879] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981886] dmar: DRHD: handling fault status reg 3
[  340.981892] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981892] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981903] dmar: DRHD: handling fault status reg 3
[  340.981908] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981908] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981919] dmar: DRHD: handling fault status reg 3
[  340.981925] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981925] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981933] dmar: DRHD: handling fault status reg 3
[  340.981941] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981941] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981948] dmar: DRHD: handling fault status reg 3
[  340.981954] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981954] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981965] dmar: DRHD: handling fault status reg 3
[  340.981970] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981970] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981980] dmar: DRHD: handling fault status reg 3
[  340.981986] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981986] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.981993] dmar: DRHD: handling fault status reg 3
[  340.981999] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.981999] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982008] dmar: DRHD: handling fault status reg 3
[  340.982014] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.982014] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982025] dmar: DRHD: handling fault status reg 3
[  340.982031] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.982031] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982039] dmar: DRHD: handling fault status reg 3
[  340.982045] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.982045] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982054] dmar: DRHD: handling fault status reg 3
[  340.982061] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  340.982061] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982274] dmar: DRHD: handling fault status reg 3
[  340.982280] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982280] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982290] dmar: DRHD: handling fault status reg 3
[  340.982295] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982295] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982304] dmar: DRHD: handling fault status reg 3
[  340.982312] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982312] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982319] dmar: DRHD: handling fault status reg 3
[  340.982323] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982323] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982334] dmar: DRHD: handling fault status reg 3
[  340.982338] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982338] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982350] dmar: DRHD: handling fault status reg 3
[  340.982356] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982356] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982363] dmar: DRHD: handling fault status reg 3
[  340.982370] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982370] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982636] dmar: DRHD: handling fault status reg 3
[  340.982637] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.982637] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.982944] dmar: DRHD: handling fault status reg 3
[  340.982950] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.982950] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.988472] dmar: DRHD: handling fault status reg 3
[  340.988473] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.988473] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.988569] dmar: DRHD: handling fault status reg 3
[  340.988575] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.988575] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.988584] dmar: DRHD: handling fault status reg 3
[  340.988590] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.988590] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990633] dmar: DRHD: handling fault status reg 3
[  340.990634] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990634] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990811] dmar: DRHD: handling fault status reg 3
[  340.990817] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990817] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990825] dmar: DRHD: handling fault status reg 3
[  340.990831] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990831] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990841] dmar: DRHD: handling fault status reg 3
[  340.990846] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990846] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990856] dmar: DRHD: handling fault status reg 3
[  340.990862] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990862] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.990872] dmar: DRHD: handling fault status reg 3
[  340.990878] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.990878] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.996848] dmar: DRHD: handling fault status reg 3
[  340.996850] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.996850] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.997144] dmar: DRHD: handling fault status reg 3
[  340.997149] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.997149] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.997160] dmar: DRHD: handling fault status reg 3
[  340.997165] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.997165] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999666] dmar: DRHD: handling fault status reg 3
[  340.999667] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  340.999667] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999898] dmar: DRHD: handling fault status reg 3
[  340.999913] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999913] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999931] dmar: DRHD: handling fault status reg 3
[  340.999936] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999936] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999948] dmar: DRHD: handling fault status reg 3
[  340.999954] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999954] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999961] dmar: DRHD: handling fault status reg 3
[  340.999967] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999967] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999976] dmar: DRHD: handling fault status reg 3
[  340.999981] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999981] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  340.999992] dmar: DRHD: handling fault status reg 3
[  340.999997] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  340.999997] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000006] dmar: DRHD: handling fault status reg 3
[  341.000014] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000014] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000021] dmar: DRHD: handling fault status reg 3
[  341.000027] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000027] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000038] dmar: DRHD: handling fault status reg 3
[  341.000042] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000042] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000055] dmar: DRHD: handling fault status reg 3
[  341.000060] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000060] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000067] dmar: DRHD: handling fault status reg 3
[  341.000075] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000075] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000082] dmar: DRHD: handling fault status reg 3
[  341.000088] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.000088] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000100] dmar: DRHD: handling fault status reg 3
[  341.000106] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000106] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000114] dmar: DRHD: handling fault status reg 3
[  341.000121] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000121] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000130] dmar: DRHD: handling fault status reg 3
[  341.000136] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000136] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000144] dmar: DRHD: handling fault status reg 3
[  341.000150] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000150] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000164] dmar: DRHD: handling fault status reg 3
[  341.000169] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000169] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000177] dmar: DRHD: handling fault status reg 3
[  341.000184] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.000184] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000271] dmar: DRHD: handling fault status reg 3
[  341.000276] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.000276] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.000284] dmar: DRHD: handling fault status reg 3
[  341.000290] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.000290] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.005726] dmar: DRHD: handling fault status reg 3
[  341.005727] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.005727] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.005836] dmar: DRHD: handling fault status reg 3
[  341.005840] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.005840] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.007839] dmar: DRHD: handling fault status reg 3
[  341.007840] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.007840] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.007963] dmar: DRHD: handling fault status reg 3
[  341.007968] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.007968] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.013906] dmar: DRHD: handling fault status reg 3
[  341.013907] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.013907] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.016017] dmar: DRHD: handling fault status reg 3
[  341.016019] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.016019] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022259] dmar: DRHD: handling fault status reg 3
[  341.022263] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.022263] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022277] dmar: DRHD: handling fault status reg 3
[  341.022282] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.022282] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022289] dmar: DRHD: handling fault status reg 3
[  341.022294] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.022294] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022305] dmar: DRHD: handling fault status reg 3
[  341.022310] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.022310] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022320] dmar: DRHD: handling fault status reg 3
[  341.022335] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.022335] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.022363] dmar: DRHD: handling fault status reg 3
[  341.022370] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.022370] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.024202] dmar: DRHD: handling fault status reg 3
[  341.024205] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.024205] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.024211] dmar: DRHD: handling fault status reg 3
[  341.024218] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.024218] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.024225] dmar: DRHD: handling fault status reg 3
[  341.024229] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.024229] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.024241] dmar: DRHD: handling fault status reg 3
[  341.024247] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.024247] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.024255] dmar: DRHD: handling fault status reg 3
[  341.024260] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.024260] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030273] dmar: DRHD: handling fault status reg 3
[  341.030277] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f8000 
[  341.030277] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030290] dmar: DRHD: handling fault status reg 3
[  341.030296] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.030296] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030303] dmar: DRHD: handling fault status reg 3
[  341.030310] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030310] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030320] dmar: DRHD: handling fault status reg 3
[  341.030327] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030327] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030336] dmar: DRHD: handling fault status reg 3
[  341.030341] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030341] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030351] dmar: DRHD: handling fault status reg 3
[  341.030356] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030356] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030369] dmar: DRHD: handling fault status reg 3
[  341.030375] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030375] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030381] dmar: DRHD: handling fault status reg 3
[  341.030388] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fa000 
[  341.030388] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030397] dmar: DRHD: handling fault status reg 3
[  341.030403] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030403] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030415] dmar: DRHD: handling fault status reg 3
[  341.030421] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030421] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030429] dmar: DRHD: handling fault status reg 3
[  341.030437] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030437] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030444] dmar: DRHD: handling fault status reg 3
[  341.030450] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030450] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030459] dmar: DRHD: handling fault status reg 3
[  341.030466] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030466] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030476] dmar: DRHD: handling fault status reg 3
[  341.030482] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0fb000 
[  341.030482] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030564] dmar: DRHD: handling fault status reg 3
[  341.030570] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.030570] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.030578] dmar: DRHD: handling fault status reg 3
[  341.030586] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.030586] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.032625] dmar: DRHD: handling fault status reg 3
[  341.032627] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be274000 
[  341.032627] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.032734] dmar: DRHD: handling fault status reg 3
[  341.032741] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.032741] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.032749] dmar: DRHD: handling fault status reg 3
[  341.032753] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.032753] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.038446] dmar: DRHD: handling fault status reg 3
[  341.038460] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.038460] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  341.040594] dmar: DRHD: handling fault status reg 3
[  341.040595] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be0f9000 
[  341.040595] DMAR:[fault reason 12] non-zero reserved fields in PTE

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here are the errors I'm getting, approximately one per OVMF screen redraw (being driven via serial interface in virt-manager):

[  330.024356] xhci_hcd 0000:08:00.0: remove, state 4
[  330.026932] usb usb4: USB disconnect, device number 1
[  330.037696] xhci_hcd 0000:08:00.0: USB bus 4 deregistered
[  330.041890] xhci_hcd 0000:08:00.0: remove, state 1
[  330.044007] usb usb3: USB disconnect, device number 1
[  330.046225] usb 3-2: USB disconnect, device number 3
[  330.076301] xhci_hcd 0000:08:00.0: USB bus 3 deregistered
[  330.908248] vfio-pci 0000:01:00.0: enabling device (0140 -> 0143)
[  335.721419] dmar: DRHD: handling fault status reg 3
[  335.723856] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be46c000 
[  335.723856] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.181491] dmar: DRHD: handling fault status reg 3
[  336.183706] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be448000 
[  336.183706] DMAR:[fault reason 12] non-zero reserved fields in PTE
[  336.189725] dmar: DRHD: handling fault status reg 3
[  336.191928] dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr be448000 
[  336.191928] DMAR:[fault reason 12] non-zero reserved fields in PTE

What host kernel is this?  I can only guess that it might be the IOMMU_CACHE attribute.  Kernel 3.15 includes support for multiple IOMMU domains within a VFIO container that should handle this situation.  What do you get from 'dmesg | grep ecap'?  If the hardware is lying about it's support for snoop control then then a patch like below should stop the error.

--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -126,7 +126,7 @@ static inline void dmar_writeq(void __iomem *addr, u64 val)
 #define ecap_ir_support(e)	((e >> 3) & 0x1)
 #define ecap_dev_iotlb_support(e)	(((e) >> 2) & 0x1)
 #define ecap_max_handle_mask(e) ((e >> 20) & 0xf)
-#define ecap_sc_support(e)	((e >> 7) & 0x1) /* Snooping Control */
+#define ecap_sc_support(e)	((e >> 7) & 0x0) /* Snooping Control */
 
 /* IOTLB_REG */
 #define DMA_TLB_FLUSH_GRANU_OFFSET  60

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I can confirm that I get a LOT of DMAR errors on my X99-Deluxe too when using VFIO . I doesn't happen when I use "-device kvm-pci-assign" for onboard audio and USB though . Also , when I use this method , I can't reboot Windows without rebooting the whole host , otherwise it crashes the host .

Are they the same errors as DanaGoyette reported?

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I can confirm that I get a LOT of DMAR errors on my X99-Deluxe too when using VFIO . I doesn't happen when I use "-device kvm-pci-assign" for onboard audio and USB though . Also , when I use this method , I can't reboot Windows without rebooting the whole host , otherwise it crashes the host .

Are they the same errors as DanaGoyette reported?

Nope . They are completely different . I'll try to reproduce it now and paste them here .

EDIT :

dmar: DMAR:[DMA Read] Request device [00:14.0] fault addr 100068000
DMAR:[fault reason 06] PTE Read access is not set

Also , these go away when I use "-device kvm-pci-assign" to assign onboard audio and USB , they only appear when using VFIO + the host crashes after a while :

Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4be00 already set (to 22d5e8003 not 359200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4c200 already set (to 22d5ea003 not 358e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4c600 already set (to 22d5ec003 not 358a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4ca00 already set (to 22d5ee003 not 358600083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4ce00 already set (to 22d5f0003 not 358200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4d200 already set (to 22d5f2003 not 357e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4d600 already set (to 22d5f4003 not 357a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4da00 already set (to 22d5f6003 not 357600083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4de00 already set (to 22d5f8003 not 357200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4e200 already set (to 22d5fa003 not 356e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4e600 already set (to 22d5fc003 not 356a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4ea00 already set (to 22d5fe003 not 356600083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4ee00 already set (to 22d600003 not 356200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4f200 already set (to 22d602003 not 355e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4f600 already set (to 22d604003 not 355a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4fa00 already set (to 22d606003 not 355600083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x4fe00 already set (to 22d608003 not 355200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x50200 already set (to 22d60a003 not 354e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x50600 already set (to 22d60c003 not 354a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x50a00 already set (to 22d60e003 not 354600083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x50e00 already set (to 22d610003 not 354200083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x51200 already set (to 22d612003 not 353e00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x51600 already set (to 22d614003 not 353a00083)
Sep 29 00:35:28 srv1 kernel: ERROR: DMA PTE for vPFN 0x51a00 already set (to 22d616003 not 353600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x51e00 already set (to 22d618003 not 353200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x52200 already set (to 22d61a003 not 352e00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x52600 already set (to 22d61c003 not 352a00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x52a00 already set (to 22d61e003 not 352600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x52e00 already set (to 22d620003 not 352200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x53200 already set (to 22d622003 not 351e00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x53600 already set (to 22d624003 not 351a00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x53a00 already set (to 22d626003 not 351600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x53e00 already set (to 22d628003 not 351200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x54200 already set (to 22d62a003 not 350e00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x54600 already set (to 22d62c003 not 350a00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x54a00 already set (to 22d62e003 not 350600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x54e00 already set (to 22d630003 not 350200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x55200 already set (to 22d632003 not 34fe00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x55600 already set (to 22d634003 not 34fa00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x55a00 already set (to 22d636003 not 34f600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x55e00 already set (to 22d638003 not 34f200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x56200 already set (to 22d63a003 not 34ee00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x56600 already set (to 22d63c003 not 34ea00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x56a00 already set (to 22d63e003 not 34e600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x56e00 already set (to 22d640003 not 34e200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x57200 already set (to 22d642003 not 34de00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x57600 already set (to 22d644003 not 34da00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x57a00 already set (to 22d646003 not 34d600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x57e00 already set (to 22d648003 not 34d200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x58200 already set (to 22d64a003 not 34ce00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x58600 already set (to 22d64c003 not 34ca00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x58a00 already set (to 22d64e003 not 34c600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x58e00 already set (to 22d650003 not 34c200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x59200 already set (to 22d652003 not 34be00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x59600 already set (to 22d654003 not 34ba00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x59a00 already set (to 22d656003 not 34b600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x59e00 already set (to 22d658003 not 34b200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5a200 already set (to 22d65a003 not 34ae00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5a600 already set (to 22d65c003 not 34aa00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5aa00 already set (to 22d65e003 not 34a600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ae00 already set (to 22d660003 not 34a200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5b200 already set (to 22d662003 not 349e00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5b600 already set (to 22d664003 not 349a00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ba00 already set (to 22d666003 not 349600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5be00 already set (to 22d668003 not 349200083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5c200 already set (to 22d66a003 not 348e00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5c600 already set (to 22d66c003 not 348a00083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ca00 already set (to 22d66e003 not 348600083)
Sep 29 00:35:29 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ce00 already set (to 22d670003 not 348200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5d200 already set (to 22d672003 not 347e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5d600 already set (to 22d674003 not 347a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5da00 already set (to 22d676003 not 347600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5de00 already set (to 22d678003 not 347200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5e200 already set (to 22d67a003 not 346e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5e600 already set (to 22d67c003 not 346a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ea00 already set (to 22d67e003 not 346600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5ee00 already set (to 22d680003 not 346200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5f200 already set (to 22d682003 not 345e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5f600 already set (to 22d684003 not 345a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5fa00 already set (to 22d686003 not 345600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x5fe00 already set (to 22d688003 not 345200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x60200 already set (to 22d68a003 not 344e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x60600 already set (to 22d68c003 not 344a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x60a00 already set (to 22d68e003 not 344600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x60e00 already set (to 22d690003 not 344200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x61200 already set (to 22d692003 not 343e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x61600 already set (to 22d694003 not 343a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x61a00 already set (to 22d696003 not 343600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x61e00 already set (to 22d698003 not 343200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x62200 already set (to 22d69a003 not 342e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x62600 already set (to 22d69c003 not 342a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x62a00 already set (to 22d69e003 not 342600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x62e00 already set (to 22d6a0003 not 342200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x63200 already set (to 22d6a2003 not 341e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x63600 already set (to 22d6a4003 not 341a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x63a00 already set (to 22d6a6003 not 341600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x63e00 already set (to 22d6a8003 not 341200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x64200 already set (to 22d6aa003 not 340e00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x64600 already set (to 22d6ac003 not 340a00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x64a00 already set (to 22d6ae003 not 340600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x64e00 already set (to 22d6b0003 not 340200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x65200 already set (to 22d6b2003 not 33fe00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x65600 already set (to 22d6b4003 not 33fa00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x65a00 already set (to 22d6b6003 not 33f600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x65e00 already set (to 22d6b8003 not 33f200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x66200 already set (to 22d6ba003 not 33ee00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x66600 already set (to 22d6bc003 not 33ea00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x66a00 already set (to 22d6be003 not 33e600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x66e00 already set (to 22d6c0003 not 33e200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x67200 already set (to 22d6c2003 not 33de00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x67600 already set (to 22d6c4003 not 33da00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x67a00 already set (to 22d6c6003 not 33d600083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x67e00 already set (to 22d6c8003 not 33d200083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x68200 already set (to 22d6ca003 not 33ce00083)
Sep 29 00:35:30 srv1 kernel: ERROR: DMA PTE for vPFN 0x68600 already set (to 22d6cc003 not 33ca00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x68a00 already set (to 22d6ce003 not 33c600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x68e00 already set (to 22d6d0003 not 33c200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x69200 already set (to 22d6d2003 not 33be00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x69600 already set (to 22d6d4003 not 33ba00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x69a00 already set (to 22d6d6003 not 33b600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x69e00 already set (to 22d6d8003 not 33b200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6a200 already set (to 22d6da003 not 33ae00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6a600 already set (to 22d6dc003 not 33aa00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6aa00 already set (to 22d6de003 not 33a600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ae00 already set (to 22d6e0003 not 33a200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6b200 already set (to 22d6e2003 not 339e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6b600 already set (to 22d6e4003 not 339a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ba00 already set (to 22d6e6003 not 339600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6be00 already set (to 22d6e8003 not 339200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6c200 already set (to 22d6ea003 not 338e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6c600 already set (to 22d6ec003 not 338a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ca00 already set (to 22d6ee003 not 338600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ce00 already set (to 22d6f0003 not 338200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6d200 already set (to 22d6f2003 not 337e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6d600 already set (to 22d6f4003 not 337a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6da00 already set (to 22d6f6003 not 337600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6de00 already set (to 22d6f8003 not 337200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6e200 already set (to 22d6fa003 not 336e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6e600 already set (to 22d6fc003 not 336a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ea00 already set (to 22d6fe003 not 336600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6ee00 already set (to 22d700003 not 336200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6f200 already set (to 22d702003 not 335e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6f600 already set (to 22d704003 not 335a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6fa00 already set (to 22d706003 not 335600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x6fe00 already set (to 22d708003 not 335200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x70200 already set (to 22d70a003 not 334e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x70600 already set (to 22d70c003 not 334a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x70a00 already set (to 22d70e003 not 334600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x70e00 already set (to 22d710003 not 334200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x71200 already set (to 22d712003 not 333e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x71600 already set (to 22d714003 not 333a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x71a00 already set (to 22d716003 not 333600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x71e00 already set (to 22d718003 not 333200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x72200 already set (to 22d71a003 not 332e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x72600 already set (to 22d71c003 not 332a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x72a00 already set (to 22d71e003 not 332600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x72e00 already set (to 22d720003 not 332200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x73200 already set (to 22d722003 not 331e00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x73600 already set (to 22d724003 not 331a00083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x73a00 already set (to 22d726003 not 331600083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x73e00 already set (to 22d728003 not 331200083)
Sep 29 00:35:31 srv1 kernel: ERROR: DMA PTE for vPFN 0x74200 already set (to 22d72a003 not 330e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x74600 already set (to 22d72c003 not 330a00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x74a00 already set (to 22d72e003 not 330600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x74e00 already set (to 22d730003 not 330200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x75200 already set (to 22d732003 not 32fe00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x75600 already set (to 22d734003 not 32fa00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x75a00 already set (to 22d736003 not 32f600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x75e00 already set (to 22d738003 not 32f200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x76200 already set (to 22d73a003 not 32ee00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x76600 already set (to 22d73c003 not 32ea00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x76a00 already set (to 22d73e003 not 32e600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x76e00 already set (to 22d740003 not 32e200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x77200 already set (to 22d742003 not 32de00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x77600 already set (to 22d744003 not 32da00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x77a00 already set (to 22d746003 not 32d600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x77e00 already set (to 22d748003 not 32d200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x78200 already set (to 22d74a003 not 32ce00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x78600 already set (to 22d74c003 not 32ca00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x78a00 already set (to 22d74e003 not 32c600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x78e00 already set (to 22d750003 not 32c200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x79200 already set (to 22d752003 not 32be00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x79600 already set (to 22d754003 not 32ba00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x79a00 already set (to 22d756003 not 32b600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x79e00 already set (to 22d758003 not 32b200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7a200 already set (to 22d75a003 not 32ae00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7a600 already set (to 22d75c003 not 32aa00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7aa00 already set (to 22d75e003 not 32a600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ae00 already set (to 22d760003 not 32a200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7b200 already set (to 22d762003 not 329e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7b600 already set (to 22d764003 not 329a00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ba00 already set (to 22d766003 not 329600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7be00 already set (to 22d768003 not 329200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7c200 already set (to 22d76a003 not 328e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7c600 already set (to 22d76c003 not 328a00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ca00 already set (to 22d76e003 not 328600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ce00 already set (to 22d770003 not 328200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7d200 already set (to 22d772003 not 327e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7d600 already set (to 22d774003 not 327a00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7da00 already set (to 22d776003 not 327600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7de00 already set (to 22d778003 not 327200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7e200 already set (to 22d77a003 not 326e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7e600 already set (to 22d77c003 not 326a00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ea00 already set (to 22d77e003 not 326600083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7ee00 already set (to 22d780003 not 326200083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7f200 already set (to 22d782003 not 325e00083)
Sep 29 00:35:32 srv1 kernel: ERROR: DMA PTE for vPFN 0x7f600 already set (to 22d784003 not 325a00083)
Sep 29 00:35:33 srv1 kernel: ERROR: DMA PTE for vPFN 0x7fa00 already set (to 22d786003 not 325600083)
Sep 29 00:35:33 srv1 kernel: ERROR: DMA PTE for vPFN 0x7fe00 already set (to 22d788003 not 325200083)

EDIT 2 :

Just now this happened :

[17796.619734] pcieport 0000:00:01.0: AER: Corrected error received: id=0008
[17796.619746] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
[17796.619769] pcieport 0000:00:01.0:   device [8086:2f02] error status/mask=00001000/00002000
[17796.619787] pcieport 0000:00:01.0:    [12] Replay Timer Timeout
[18601.941321] pcieport 0000:00:01.0: AER: Corrected error received: id=0008
[18601.941334] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
[18601.941358] pcieport 0000:00:01.0:   device [8086:2f02] error status/mask=00001000/00002000
[18601.941376] pcieport 0000:00:01.0:    [12] Replay Timer Timeout
[19841.746240] pcieport 0000:00:01.0: AER: Corrected error received: id=0008
[19841.746251] pcieport 0000:00:01.0: PCIe Bus Error: severity=Corrected, type=Data Link Layer, id=0008(Transmitter ID)
[19841.746279] pcieport 0000:00:01.0:   device [8086:2f02] error status/mask=00001000/00002000
[19841.746297] pcieport 0000:00:01.0:    [12] Replay Timer Timeout

00:01.0 PCI bridge: Intel Corporation Haswell-E PCI Express Root Port 1 (rev 02)

Last edited by Denso (2014-09-29 07:53:17)

DanaGoyette

Member

Registered: 2014-01-03

Posts: 46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What host kernel is this?  I can only guess that it might be the IOMMU_CACHE attribute.  Kernel 3.15 includes support for multiple IOMMU domains within a VFIO container that should handle this situation.  What do you get from 'dmesg | grep ecap'?  If the hardware is lying about it's support for snoop control then then a patch like below should stop the error.

Kernel is Linux 3.16.1, with ACS override patch applied, but not enabled when using the X10SAT.
I used the ACS Override patch when testing on the X10SL7-F, because the LSI SAS controller is in a CPU-based PCIe port, and the X16 slot gets tied to it.  (I can't imagine an LSI SAS card ever directly talking to a video card...)

Lines that seem relevant:

[    0.061450] dmar: Host address width 39
[    0.061452] dmar: DRHD base: 0x000000fed90000 flags: 0x0
[    0.061460] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020660462 ecap f0101a
[    0.061461] dmar: DRHD base: 0x000000fed91000 flags: 0x1
[    0.061466] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap d2008c20660462 ecap f010da
[    0.061468] dmar: RMRR base: 0x0000006dea5000 end: 0x0000006deb3fff
[    0.061469] dmar: RMRR base: 0x0000006f000000 end: 0x0000007f1fffff
[    0.061534] IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
[    0.061535] HPET id 0 under DRHD base 0xfed91000
[    0.061536] Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.061537] Your BIOS is broken and requested that x2apic be disabled.
[    0.061537] This will slightly decrease performance.
[    0.061537] Use 'intremap=no_x2apic_optout' to override BIOS request.
[    0.061649] Enabled IRQ remapping in xapic mode
[    0.061650] x2apic not enabled, IRQ remapping is in xapic mode

[    0.627093] DMAR: No ATSR found
[    0.627114] IOMMU 1 0xfed91000: using Queued invalidation
[    0.627116] IOMMU: Setting RMRR:
[    0.627126] IOMMU: Setting identity map for device 0000:00:14.0 [0x6dea5000 - 0x6deb3fff]
[    0.627145] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.627153] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.627166] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O

[  393.579011] vfio-pci 0000:01:00.0: enabling device (0400 -> 0403)

Last edited by DanaGoyette (2014-09-29 06:36:06)

Lauer

Member

Registered: 2014-09-11

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I get a compat_monitor0 screen similar to Slabity. However, I am using an intel IGP for the host, and an AMD 6990M for the Windows 7 guest.

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=e7fe3792-3cba-cf01-605e-37923cbacf01 rw quiet i915.enable_hd_vgaarb=1 pci-stub.ids=1002:6720,1002:aa88 intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 kvm_intel.emulate_invalid_guest_state=0

I'm using the OP's linux-mainline package, and have also used the acs_override=downstream option with the same result.

Same result when running vga-clear vga-set.

dmesg | grep vgaarb

[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=e7fe3792-3cba-cf01-605e-37923cbacf01 rw quiet i915.enable_hd_vgaarb=1 pci-stub.ids=1002:6720,1002:aa88 intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 kvm_intel.emulate_invalid_guest_state=0
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=e7fe3792-3cba-cf01-605e-37923cbacf01 rw quiet i915.enable_hd_vgaarb=1 pci-stub.ids=1002:6720,1002:aa88 intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 kvm_intel.emulate_invalid_guest_state=0
[    0.504315] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    0.504320] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=none,locks=none
[    0.504322] vgaarb: loaded
[    0.504323] vgaarb: bridge control possible 0000:01:00.0
[    0.504324] vgaarb: no bridge control possible 0000:00:02.0
[    1.723418] vgaarb: device changed decodes: PCI:0000:00:02.0,olddecodes=io+mem,decodes=io:owns=io+mem

Any advice would be appreciated.