You are not logged in.

Pages: 1 … 84 85 86 87 88 … 234

Topic closed

#2126 2014-06-17 20:56:47

lojsar
Member
Registered: 2014-06-16
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

belliash wrote:
lojsar wrote:

...

Most probably you nvidia gpu is not connected to HDMI, but to igp. Thats why it thinks it has CRT-0 connected.

I see my post was not very clear. The HDMI output works. It's just that I get a CRT-0 output too, that I have to disable to get a sane resolution. I tried to clarify this in my post now.

belliash wrote:

BTW: I'm also fighting with passing through my nVidia GPU, but I always got error 43 and 10/12 (cant remember) before installing nvidia driver in windows.
Can you tell sth more about what you done? Maybe some instruction please?

I haven't figured out how to make it work in Windows yet. When I tried Windows I only got Code 10, so I changed to Linux to get more error messages (I recommend NVreg_ResmanDebugLevel=0)

Offline

#2127 2014-06-17 21:08:30

belliash
Member
Registered: 2014-04-25
Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lojsar wrote:
belliash wrote:
lojsar wrote:

...

Most probably you nvidia gpu is not connected to HDMI, but to igp. Thats why it thinks it has CRT-0 connected.

I see my post was not very clear. The HDMI output works. It's just that I get a CRT-0 output too, that I have to disable to get a sane resolution. I tried to clarify this in my post now.

belliash wrote:

BTW: I'm also fighting with passing through my nVidia GPU, but I always got error 43 and 10/12 (cant remember) before installing nvidia driver in windows.
Can you tell sth more about what you done? Maybe some instruction please?

I haven't figured out how to make it work in Windows yet. When I tried Windows I only got Code 10, so I changed to Linux to get more error messages (I recommend NVreg_ResmanDebugLevel=0)

Can you check one thing?
Please pass your nvidia gpu as a second card to VM (use also a cirrus/vmware/etc) and install Windows. Then please install binary nvidia drivers and reboot host.
Finally launch guest and if you see you nvidia card detected properly but with exclamation mark, please check error code. If it will be 43, please disable and reenable GPU from device manager.

These steps help to get dgpu working in guest windows os. However it remains secondary gpu, so its useless. I wonder if your problem is the same is mine...

Offline

#2128 2014-06-18 10:43:27

Nesousx
Member
Registered: 2012-03-27
Posts: 46
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

First of all, thanks OP for the tutorial.

I am running Ubuntu 14.04 and trying to run KVM VGA passthrough. I have successfully done several VGA passthrough with various distros and with Xen too. I always use IGP as main card for my Linux rig, and PCIe card as my Windows GPU. Everything works fine. I even got it working with 2 ATI cards, a 6450 in 1st PCie for the Linux host and a 7950 in 2nd PCIe slot for Windows guest. However, gaming performance are really poor since the 2nd PCIe slot is limited to 1x, and that I don't have enough physical space in my case to plug the cards differently.

I'd like to get a WQHD compliant setup. Now, what I want to do and can't seem to get working is to have my Linux rig use my 2nd PCIe (ATI 6450) card as GPU and my Windows rig use the first PCIe card (ATI 7950). The best I could get is a working and accelerated Linux desktop environnement on second GPU, pci-stub claimed the corrrect devices, vfio module and kvm loaded, etc. When I start my Windows guest with PCI devices attached to it, it starts but I have no display on guest and my qemu logfile is literraly spammed (until the /var parition is full) with something like:

qemu-system-x86_64: vfio_bar_write failed: Device or resourc e busy

I am not providing much log atm, because it might not be needed. Indeed, I remember reading that I can not pass the secondary video adapter to guest. So is it possible to get the 2nd PCIe card passed to guest and the 1st PCIe card for host?

Thanks in avance,

NB: if I absolutely want to pass the 2nd to guest, it is because I have no choice with my current case / motherboard.

Offline

#2129 2014-06-18 10:50:07

sinny
Member
Registered: 2014-05-24
Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Nesousx wrote:

Hi all,

First of all, thanks OP for the tutorial.

I am running Ubuntu 14.04 and trying to run KVM VGA passthrough. I have successfully done several VGA passthrough with various distros and with Xen too. I always use IGP as main card for my Linux rig, and PCIe card as my Windows GPU. Everything works fine. I even got it working with 2 ATI cards, a 6450 in 1st PCie for the Linux host and a 7950 in 2nd PCIe slot for Windows guest. However, gaming performance are really poor since the 2nd PCIe slot is limited to 1x, and that I don't have enough physical space in my case to plug the cards differently.

I'd like to get a WQHD compliant setup. Now, what I want to do and can't seem to get working is to have my Linux rig use my 2nd PCIe (ATI 6450) card as GPU and my Windows rig use the first PCIe card (ATI 7950). The best I could get is a working and accelerated Linux desktop environnement on second GPU, pci-stub claimed the corrrect devices, vfio module and kvm loaded, etc. When I start my Windows guest with PCI devices attached to it, it starts but I have no display on guest and my qemu logfile is literraly spammed (until the /var parition is full) with something like:

qemu-system-x86_64: vfio_bar_write failed: Device or resourc e busy

I am not providing much log atm, because it might not be needed. Indeed, I remember reading that I can not pass the secondary video adapter to guest. So is it possible to get the 2nd PCIe card passed to guest and the 1st PCIe card for host?

Thanks in avance,

NB: if I absolutely want to pass the 2nd to guest, it is because I have no choice with my current case / motherboard.

what card is used by bios? are you sure you are not getting hit by lack of bios configurability in terms of what non-integrated GPU is used as primary adapter?
using IGP for host os is not an option?

Offline

#2130 2014-06-18 11:14:30

Nesousx
Member
Registered: 2012-03-27
Posts: 46
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

sinny wrote:

what card is used by bios? are you sure you are not getting hit by lack of bios configurability in terms of what non-integrated GPU is used as primary adapter?
using IGP for host os is not an option?

Card used by bios is the 1st PCIe card (aka 7950, the one I want passed to Windows).

My bios kinda sucks (Asrock Z77 Pro 4M); since I can only select from IGP or 1st PCIe card. I wish I could use my 2nd PCIe card as "main GPU".

IGP for host is no longer an option since I plan on buying a WQHD screen and IGP doesn't support that resolution. sad

Offline

#2131 2014-06-18 11:45:30

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Nesousx wrote:
sinny wrote:

what card is used by bios? are you sure you are not getting hit by lack of bios configurability in terms of what non-integrated GPU is used as primary adapter?
using IGP for host os is not an option?

Card used by bios is the 1st PCIe card (aka 7950, the one I want passed to Windows).

My bios kinda sucks (Asrock Z77 Pro 4M); since I can only select from IGP or 1st PCIe card. I wish I could use my 2nd PCIe card as "main GPU".

IGP for host is no longer an option since I plan on buying a WQHD screen and IGP doesn't support that resolution. sad

Using UEFI i'm able to passthrough my primary card by disabling efifb with video=efifb:off as a kernel parameter

Offline

#2132 2014-06-18 14:59:09

Nesousx
Member
Registered: 2012-03-27
Posts: 46
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

Using UEFI i'm able to passthrough my primary card by disabling efifb with video=efifb:off as a kernel parameter

Great news, if it works for you then I might get something working too.

So far the kernel parameter didin't doo anything, but I believe I have to fix that libvirtd error first. Just curious, did you anything special with UEFI to make it work?

And if it helps anyone, the error I'am getting spammed with in my guest domain's log is:

qemu-system-x86_64: vfio_bar_write(,0xf7bf4e0, 0x0, 4) failed: Device or resource busy

Last edited by Nesousx (2014-06-18 15:49:41)

Offline

#2133 2014-06-18 17:02:16

slis
Member
Registered: 2014-06-02
Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lil update on my progress...


got gigabyte g1 sniper 3...
4-way sli

i7 2600s

it works with 4 gpu-s...

2x 560ti
1x 660
1x 8800gt

all working... debian, linux-vfio 3.15, acs=downstream

ill post benchs latter with better gpu-s...
Yj9Eg0Us.jpg N1ROzj5s.jpg

Last edited by slis (2014-06-18 17:07:31)

Offline

#2134 2014-06-18 17:42:35

lojsar
Member
Registered: 2014-06-16
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
lojsar wrote:

When I open the nVidia device, the driver complains that it can't load the VBIOS. I suspected that it wanted to find it in ACPI, so I modified the nvidia driver, replacing the function that reads VBIOS from ACPI with one that reads it from my dumpfile. This worked!
Is there a way I can avoid modifying the driver? I would like to launch Windows too! Is it "written" somewhere on the GPU that VBIOS is located in ACPI?

Is the VBIOS you extracted actually a PCI ROM?  Run 'xxd nouveau-vbios.rom | less' and you should see these important markers:

You were of course correct! At some point I mistyped a character into the romfile= parameter, which apparently wasn't fatal for qemu, it just started without a ROM since the file didn't exist. Now I don't need my driver modification anymore. Thank you again! and sorry for being such an idiot :(

Furthermore, I had great success modifying the VBIOS. Using a program called "nvbios" from "envytools" together with nVidia's documentation of the DCB ( ftp://download.nvidia.com/open-gpu-doc/ … _dcb_flags ), I was able to replace the VGA output. After this, I see BIOS output on my monitor! I was also able to run Windows (with -vga none), and install the nVidia drivers!

Rebooting the guest VM doesn't seem to work tho. I've observed the same behavior when rebooting inside the guest, and shutting down the guest and starting it again. It stays a few second at "Running option rom at ce00:0003", and if I don't kill qemu fast enough the whole (host) computer freezes. I will look into this now, as rebooting seems to be an important part of Windows.

Can you check one thing?
Please pass your nvidia gpu as a second card to VM (use also a cirrus/vmware/etc) and install Windows. Then please install binary nvidia drivers and reboot host.
Finally launch guest and if you see you nvidia card detected properly but with exclamation mark, please check error code. If it will be 43, please disable and reenable GPU from device manager.

These steps help to get dgpu working in guest windows os. However it remains secondary gpu, so its useless. I wonder if your problem is the same is mine...

I didn't try this since it worked as primary GPU. My only modifications to qemu now is the kvm=off patch. You could perhaps check the DCB of your card. I assumed the VGA stuff only used the first output, which in my case wasn't really connected.

Last edited by lojsar (2014-06-18 19:04:47)

Offline

#2135 2014-06-18 17:56:38

jajacomp
Member
Registered: 2014-06-18
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi! I have problem. I use:
- motherboard Asus z87pro
- first VGA - nvidia gtx 650 (HOST)
- second VGA - ATI 7770 (GUEST)

If i use first vga for system then i can't passthrough second card to virtual machine (Error = vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver)
if i use integrated video card (Intel hd 4600) for system and passthrough both video card (vfio-bind 0000:01:00.0 0000:01:00.1 0000:02:00.0 0000:02:00.1) then i can use ATI card without problems.

version qemu is latest from git
kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"


Please! Help! I want use nvidia for main system

Last edited by jajacomp (2014-06-18 17:58:48)

Offline

#2136 2014-06-18 18:56:01

belliash
Member
Registered: 2014-04-25
Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lojsar wrote:

I didn't try this since it worked as primary GPU. My only modifications to qemu now is the kvm=off patch. You could perhaps check the DCB of your card. I assumed the VGA stuff only used the first output, which in my case wasn't really connected.

Can you tell me exactly how and what should I check?

Offline

#2137 2014-06-18 18:59:10

sinny
Member
Registered: 2014-05-24
Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

jajacomp wrote:

Hi! I have problem. I use:
- motherboard Asus z87pro
- first VGA - nvidia gtx 650 (HOST)
- second VGA - ATI 7770 (GUEST)

If i use first vga for system then i can't passthrough second card to virtual machine (Error = vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver)
if i use integrated video card (Intel hd 4600) for system and passthrough both video card (vfio-bind 0000:01:00.0 0000:01:00.1 0000:02:00.0 0000:02:00.1) then i can use ATI card without problems.

version qemu is latest from git
kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"


Please! Help! I want use nvidia for main system

is it just me seeing things or there are no nvidia devices in pci-stub.ids kernel parameter (amd and intel only; why intel btw?)?

Offline

#2138 2014-06-18 19:09:47

lojsar
Member
Registered: 2014-06-16
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

belliash wrote:
lojsar wrote:

I didn't try this since it worked as primary GPU. My only modifications to qemu now is the kvm=off patch. You could perhaps check the DCB of your card. I assumed the VGA stuff only used the first output, which in my case wasn't really connected.

Can you tell me exactly how and what should I check?

Run nvbios (from https://github.com/envytools/envytools/ ) on your VBIOS dump. Then check the DCB section.
Do you also have a laptop with Optimus btw? If it's a normal desktop computer I don't think you will have the same problem.

Maybe you could try Linux as a guest instead. I found it easier to debug.

Offline

#2139 2014-06-18 19:11:15

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

jajacomp wrote:

kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"

The ACS override option requires a kernel patch, did you apply it?

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#2140 2014-06-18 19:13:14

belliash
Member
Registered: 2014-04-25
Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lojsar wrote:
belliash wrote:
lojsar wrote:

I didn't try this since it worked as primary GPU. My only modifications to qemu now is the kvm=off patch. You could perhaps check the DCB of your card. I assumed the VGA stuff only used the first output, which in my case wasn't really connected.

Can you tell me exactly how and what should I check?

Run nvbios (from https://github.com/envytools/envytools/ ) on your VBIOS dump. Then check the DCB section.
Do you also have a laptop with Optimus btw? If it's a normal desktop computer I don't think you will have the same problem.

Maybe you could try Linux as a guest instead. I found it easier to debug.

Yes, It's notebook equipped with gtx770m

Offline

#2141 2014-06-18 19:19:14

mbroemme
Member
From: Cologne
Registered: 2014-02-05
Posts: 40

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw: Do you know if https://lkml.org/lkml/2014/5/25/94 is already in 3.16-rc1?

Offline

#2142 2014-06-18 19:22:26

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

mbroemme wrote:

aw: Do you know if https://lkml.org/lkml/2014/5/25/94 is already in 3.16-rc1?

It is not https://git.kernel.org/cgit/linux/kerne … a/vgaarb.c

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#2143 2014-06-18 19:42:29

belliash
Member
Registered: 2014-04-25
Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@lojsar: All I got about DCB is:

Unknown DCB table version 7.13
Failed to parse DCB table at 0xffff version 7.13


Im not sure if i done it correctly, but I used vbios image ready to flash to card.

Last edited by belliash (2014-06-18 19:44:33)

Offline

#2144 2014-06-18 20:10:58

jajacomp
Member
Registered: 2014-06-18
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

sinny wrote:
jajacomp wrote:

Hi! I have problem. I use:
- motherboard Asus z87pro
- first VGA - nvidia gtx 650 (HOST)
- second VGA - ATI 7770 (GUEST)

If i use first vga for system then i can't passthrough second card to virtual machine (Error = vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver)
if i use integrated video card (Intel hd 4600) for system and passthrough both video card (vfio-bind 0000:01:00.0 0000:01:00.1 0000:02:00.0 0000:02:00.1) then i can use ATI card without problems.

version qemu is latest from git
kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"


Please! Help! I want use nvidia for main system

is it just me seeing things or there are no nvidia devices in pci-stub.ids kernel parameter (amd and intel only; why intel btw?)?

I add to pci-stub this:
02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X] [1002:683d]
02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
and this
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06)

Offline

#2145 2014-06-18 20:11:32

jajacomp
Member
Registered: 2014-06-18
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
jajacomp wrote:

kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"

The ACS override option requires a kernel patch, did you apply it?

Hm, no. It can help?

Offline

#2146 2014-06-18 20:13:14

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

jajacomp wrote:
sinny wrote:
jajacomp wrote:

Hi! I have problem. I use:
- motherboard Asus z87pro
- first VGA - nvidia gtx 650 (HOST)
- second VGA - ATI 7770 (GUEST)

If i use first vga for system then i can't passthrough second card to virtual machine (Error = vfio: error, group 1 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver)
if i use integrated video card (Intel hd 4600) for system and passthrough both video card (vfio-bind 0000:01:00.0 0000:01:00.1 0000:02:00.0 0000:02:00.1) then i can use ATI card without problems.

version qemu is latest from git
kernel 3.15.0
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on pci-stub.ids=1002:683d,1002:aab0,8086:0c05 vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream"


Please! Help! I want use nvidia for main system

is it just me seeing things or there are no nvidia devices in pci-stub.ids kernel parameter (amd and intel only; why intel btw?)?

I add to pci-stub this:
02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X] [1002:683d]
02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
and this
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06)

Forcing root ports or bridges to bind to pci-stub is a really bad idea.  They should also not be bound to vfio-pci.  Leave them alone.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#2147 2014-06-18 20:33:05

jajacomp
Member
Registered: 2014-06-18
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
jajacomp wrote:
sinny wrote:

is it just me seeing things or there are no nvidia devices in pci-stub.ids kernel parameter (amd and intel only; why intel btw?)?

I add to pci-stub this:
02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde XT [Radeon HD 7770/8760 / R7 250X] [1002:683d]
02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
and this
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x8 Controller [8086:0c05] (rev 06)

Forcing root ports or bridges to bind to pci-stub is a really bad idea.  They should also not be bound to vfio-pci.  Leave them alone.

Ok. I delete it. Where i can find ACS patch for 3.15 kernel?

Perfect!!! I apply patch and all work! Thank!!!

Last edited by jajacomp (2014-06-18 21:05:35)

Offline

#2148 2014-06-18 21:31:51

lojsar
Member
Registered: 2014-06-16
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Do you know how to make rebooting work?

If I boot Windows 7 now, and
a) restart it from Windows
b) shut down from Windows, and start qemu again
the host will freeze when seabios is "Running option rom at c000:0003".

I tried to search around in this thread, and found a post on howto reset the PCI bus by oring 0x40 into a register. After doing this the host doesn't freeze anymore, but I still doesn't get past "Running option rom at c000:0003". I attached some lspci output.

Are there any more tricks? :

After booting the laptop and running vfio-bind.sh

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0000000-f10fffff
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: fee002f8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: vfio-pci

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108M [GeForce GT 525M] [10de:0df5] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Dell Device [1028:04b6]
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 11
	Region 0: Memory at f0000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Region 3: Memory at d0000000 (64-bit, prefetchable) [size=32M]
	Region 5: I/O ports at 3000 [disabled] [size=128]
	Expansion ROM at f1000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <4us, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM L1 Enabled; RCB 128 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia

After running Windows 7 in qemu and shutting down (all the BARs are [disabled] + many more changes)

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0000000-f10fffff
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: fee002f8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: vfio-pci

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108M [GeForce GT 525M] [10de:0df5] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Dell Device [1028:04b6]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f0000000 (32-bit, non-prefetchable) [disabled] [size=16M]
	Region 1: Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Region 3: Memory at d0000000 (64-bit, prefetchable) [disabled] [size=32M]
	Region 5: I/O ports at 3000 [disabled] [size=128]
	Expansion ROM at f1000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM L1 Enabled; RCB 128 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia

After running vfio-unbind.sh (I didn't see any interesting changes)

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0000000-f10fffff
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: fee002f8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108M [GeForce GT 525M] [10de:0df5] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Dell Device [1028:04b6]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: Memory at f0000000 (32-bit, non-prefetchable) [disabled] [size=16M]
	Region 1: Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Region 3: Memory at d0000000 (64-bit, prefetchable) [disabled] [size=32M]
	Region 5: I/O ports at 3000 [disabled] [size=128]
	Expansion ROM at f1000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM L1 Enabled; RCB 128 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel modules: nouveau, nvidia

After running setpci on the bridge, writing 0x40 and clearing. Now the BARs are back! IO ports have also been enabled (they weren't after booting)

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0000000-f10fffff
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: fee002f8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108M [GeForce GT 525M] [10de:0df5] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Dell Device [1028:04b6]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: [virtual] Memory at f0000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Region 3: Memory at d0000000 (64-bit, prefetchable) [size=32M]
	Region 5: I/O ports at 3000 [size=128]
	[virtual] Expansion ROM at f1000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel modules: nouveau, nvidia

After running vfio-bind.sh again (I didn't see any interesting changes)

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port [8086:0101] (rev 09) (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 00003000-00003fff
	Memory behind bridge: f0000000-f10fffff
	Prefetchable memory behind bridge: 00000000c0000000-00000000d1ffffff
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort+ <SERR- <PERR-
	BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [88] Subsystem: Intel Corporation Device [8086:2010]
	Capabilities: [80] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [90] MSI: Enable- Count=1/1 Maskable- 64bit-
		Address: fee002f8  Data: 0000
	Capabilities: [a0] Express (v2) Root Port (Slot+), MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
			ExtTag- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 128 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #2, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <256ns, L1 <4us
			ClockPM- Surprise- LLActRep- BwNot+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt+ ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75.000W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet+ LinkState-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna- CRSVisible-
		RootCap: CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR+, OBFF Not Supported ARIFwd-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd-
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed+ WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [140 v1] Root Complex Link
		Desc:	PortNumber=02 ComponentID=01 EltType=Config
		Link0:	Desc:	TargetPort=00 TargetComponent=01 AssocRCRB- LinkType=MemMapped LinkValid+
			Addr:	00000000fed19000
	Kernel driver in use: vfio-pci

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108M [GeForce GT 525M] [10de:0df5] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Dell Device [1028:04b6]
	Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 16
	Region 0: [virtual] Memory at f0000000 (32-bit, non-prefetchable) [size=16M]
	Region 1: Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Region 3: Memory at d0000000 (64-bit, prefetchable) [size=32M]
	Region 5: I/O ports at 3000 [size=128]
	[virtual] Expansion ROM at f1000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 5GT/s, Width x16, ASPM L0s L1, Latency L0 <1us, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 128 bytes Disabled- Retrain- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis+, LTR-, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100 v1] Virtual Channel
		Caps:	LPEVC=0 RefClk=100ns PATEntryBits=1
		Arb:	Fixed- WRR32- WRR64- WRR128-
		Ctrl:	ArbSelect=Fixed
		Status:	InProgress-
		VC0:	Caps:	PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
			Arb:	Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
			Ctrl:	Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
			Status:	NegoPending- InProgress-
	Capabilities: [128 v1] Power Budgeting <?>
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia
belliash wrote:

@lojsar: All I got about DCB is:

Unknown DCB table version 7.13
Failed to parse DCB table at 0xffff version 7.13


Im not sure if i done it correctly, but I used vbios image ready to flash to card.

That looks completely wrong since the latest version is 4.0. Maybe you can upload the VBIOS. Do you use the same file as romfile= when starting qemu?

Offline

#2149 2014-06-18 22:22:23

abdullah
Member
Registered: 2014-06-10
Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I compiled the kernel 3.16-rc1 with VFIO,
Installed
Qemu 2.0.0
LibVirt 1.2.2

intel_immou=on was added to grub and updated.
changed the privileges in qemu.conf.
Now, when running virt-manager I get the error "vfio group".

Still testing...
I will be grateful for any attempt to help

Offline

#2150 2014-06-18 22:54:21

jajacomp
Member
Registered: 2014-06-18
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have problem with screen size on monitor. If i set resolution 1080 - then image don't fill all screen, just 80%. How fix if? Thank!

Offline

Pages: 1 … 84 85 86 87 88 … 234

Topic closed

Board footer

Powered by FluxBB