Arch Linux

peaquino

Member

Registered: 2013-12-09

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all!

I would like to report partial success with my APU setup (A85X, A10-5800K and 7870XT). With latest mainline kernel and qemu-git (from AUR, not the patched versions from first page) I was able to start Windows 8.1 with latest Catalyst and even play few minutes of Metro 2033. Unfortunatelly, the game is hanging up after a while (host stays responsive). It looks like it hangs during intense disk activity (ex. loading textures when entering new area). I tried booting the system with nohz=off and voluntary preemption, it doesn't seem to make any difference. Do you have any idea what else can I check?

paradexes

Member

Registered: 2013-11-30

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@techdude300: i have exactly same problem with memory, pls let me/us know if you find a fix/workaround;

@paradexes: i had same output, but i saw it works prefix sudo ...  so looking into it i found this :

...The final step is to provide the user with access to the group if unprivileged operation is desired (note that /dev/vfio/vfio provides no capabilities on its own and is therefore expected to be set to mode 0666 by the system)...

# chown user:user /dev/vfio/17

Afterwards i think should work. This is like a default and needs a change, i would vote for this to be mentioned by OP because it was not obvious for me either.

Thanks for the advice. It seems obvious however it still didnt work. I am running these commands as root. Im going in sudo -s instead of just running sudo + command as it gets tedious after hours of that. At any rate seeing as its being run as root and I am still getting this error Im trying to figure out what else it could be. Thanks for the help so far

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

This thread has been great. Thanks for the all the info so far. Also I have hardware now that does work and is supported for IOMMU. However now I am running into an issue when I run the qemu commands. Here is my commands and output:

qemu-system-x86_64 -enable-kvm -M q35 -m 16024 -cpu host -smp 6,sockets=1,cores=6,threads=1 -bios /usr/share/qemu/bios.bin -vga none -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on -device vfio-pci,host=03:00.1,bus=root.1,addr=00.1

qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 17 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 17
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed.
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

What am I missing here? It seems like its almost there but I am sure I am issing something here. The error seems to indicate to put this in an iommu_group which I was under the impression I did.

Edit: Specs. Im running on an TYAN S8225 board. Running 32Gb of memory with 2x 4234 Opterons, OS: Linuxmint 16 with 3.12.6 kernel. I have an nvidia 660ti as the host card and my intended card is an AMD R9 270X.

Thanks

The error message is trying really hard to tell you what the problem is: "please ensure all devices within the iommu_group are bound to their vfio bus driver".  So, run lsgroup, look at everything in group 17.  If it's an endpoint it must be bound to vfio-pci (or pci-stub if you don't plan to assign it).  If it's a root port, leave it alone.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

paradexes

Member

Registered: 2013-11-30

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

This thread has been great. Thanks for the all the info so far. Also I have hardware now that does work and is supported for IOMMU. However now I am running into an issue when I run the qemu commands. Here is my commands and output:

qemu-system-x86_64 -enable-kvm -M q35 -m 16024 -cpu host -smp 6,sockets=1,cores=6,threads=1 -bios /usr/share/qemu/bios.bin -vga none -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on -device vfio-pci,host=03:00.1,bus=root.1,addr=00.1

qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: error, group 17 is not viable, please ensure all devices within the iommu_group are bound to their vfio bus driver.
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: vfio: failed to get group 17
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device initialization failed.
qemu-system-x86_64: -device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Device 'vfio-pci' could not be initialized

What am I missing here? It seems like its almost there but I am sure I am issing something here. The error seems to indicate to put this in an iommu_group which I was under the impression I did.

Edit: Specs. Im running on an TYAN S8225 board. Running 32Gb of memory with 2x 4234 Opterons, OS: Linuxmint 16 with 3.12.6 kernel. I have an nvidia 660ti as the host card and my intended card is an AMD R9 270X.

Thanks

The error message is trying really hard to tell you what the problem is: "please ensure all devices within the iommu_group are bound to their vfio bus driver".  So, run lsgroup, look at everything in group 17.  If it's an endpoint it must be bound to vfio-pci (or pci-stub if you don't plan to assign it).  If it's a root port, leave it alone.

aw thanks for the input. This device is an AMD R9 270X. The host card is an nvidia 660ti. The output appears to be correct. Forgive the n00bness in the question but how do I know if its a root port vs not? right now the card is coming up with the usual lspci and grep commands as 0000:03:00.0 and 0000:03:00.1 (hdmi part) They appear to be bound correctly using the vfio-pci script in the OP. This seems like the last piece of the puzzle. I suppose a step by step on what would be needed to fix this error? I have done some reading but nothing is really clearing this part up here. Thanks.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The error message is trying really hard to tell you what the problem is: "please ensure all devices within the iommu_group are bound to their vfio bus driver".  So, run lsgroup, look at everything in group 17.  If it's an endpoint it must be bound to vfio-pci (or pci-stub if you don't plan to assign it).  If it's a root port, leave it alone.

aw thanks for the input. This device is an AMD R9 270X. The host card is an nvidia 660ti. The output appears to be correct. Forgive the n00bness in the question but how do I know if its a root port vs not? right now the card is coming up with the usual lspci and grep commands as 0000:03:00.0 and 0000:03:00.1 (hdmi part) They appear to be bound correctly using the vfio-pci script in the OP. This seems like the last piece of the puzzle. I suppose a step by step on what would be needed to fix this error? I have done some reading but nothing is really clearing this part up here. Thanks.

lsgroup will give you the lspci description of the device, you can interrogate further with lspci -vvv -s <pci device>, or just paste the lsgroup output for the group and we can tell you.  Chances are that there are more devices in the group than just the GPU and audio device if you're getting this error.  You can also use 'find /sys/kernel/iommu_groups' to understand the grouping if you don't have lsgroup (it's pasted somewhere in this forum thread).  If you want to change the grouping beyond what the hardware tells us is safe, you can use the ACS override patch to supply boot options to do this, but note the risks I mentioned a page or two back about using this.  I hope to be able to release a patch that will make this unnecessary for a bunch of Intel root ports soon.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Val532

Member

Registered: 2013-11-13

Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I hope to be able to release a patch that will make this unnecessary for a bunch of Intel root ports soon.

So you work on new proper patch for ACS on Intel ??? Or it's just a wish ?

And is someone has managed to use a Sata card with vfio ?

I have two AsMedia 1062 (2*sata3) and it does not work correctly.

Thanks

paradexes

Member

Registered: 2013-11-30

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The error message is trying really hard to tell you what the problem is: "please ensure all devices within the iommu_group are bound to their vfio bus driver".  So, run lsgroup, look at everything in group 17.  If it's an endpoint it must be bound to vfio-pci (or pci-stub if you don't plan to assign it).  If it's a root port, leave it alone.

aw thanks for the input. This device is an AMD R9 270X. The host card is an nvidia 660ti. The output appears to be correct. Forgive the n00bness in the question but how do I know if its a root port vs not? right now the card is coming up with the usual lspci and grep commands as 0000:03:00.0 and 0000:03:00.1 (hdmi part) They appear to be bound correctly using the vfio-pci script in the OP. This seems like the last piece of the puzzle. I suppose a step by step on what would be needed to fix this error? I have done some reading but nothing is really clearing this part up here. Thanks.

lsgroup will give you the lspci description of the device, you can interrogate further with lspci -vvv -s <pci device>, or just paste the lsgroup output for the group and we can tell you.  Chances are that there are more devices in the group than just the GPU and audio device if you're getting this error.  You can also use 'find /sys/kernel/iommu_groups' to understand the grouping if you don't have lsgroup (it's pasted somewhere in this forum thread).  If you want to change the grouping beyond what the hardware tells us is safe, you can use the ACS override patch to supply boot options to do this, but note the risks I mentioned a page or two back about using this.  I hope to be able to release a patch that will make this unnecessary for a bunch of Intel root ports soon.

Ok here is the output I have from the lsgroup script. (applicable to group 17 since thats where I am having the issue. also below is the output to the lspci and find commands you specified thanks again.

~#sh lsgroup.sh
### Group 17 ###
    03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT [Radeon R9 270X]
    03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]

~#find /sys/kernel/iommu_groups/
/sys/kernel/iommu_groups/17
/sys/kernel/iommu_groups/17/devices
/sys/kernel/iommu_groups/17/devices/0000:03:00.0
/sys/kernel/iommu_groups/17/devices/0000:03:00.1

~ # lspci -vvv -s 0000:03:00.0
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT [Radeon R9 270X] (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd Device 2272
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 32
	Region 0: Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Region 2: Memory at fcc80000 (64-bit, non-prefetchable) [size=256K]
	Region 4: I/O ports at b000 [size=256]
	Expansion ROM at fcc60000 [disabled] [size=128K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [270 v1] #19
	Capabilities: [2b0 v1] Address Translation Service (ATS)
		ATSCap:	Invalidate Queue Depth: 00
		ATSCtl:	Enable-, Smallest Translation Unit: 00
	Capabilities: [2c0 v1] #13
	Capabilities: [2d0 v1] #1b
	Kernel driver in use: vfio-pci

~ # lspci -vvv -s 0000:03:00.1
03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
	Subsystem: Gigabyte Technology Co., Ltd Device aab0
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin B routed to IRQ 124
	Region 0: Memory at fccfc000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Latency L0 <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot-
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 5GT/s, Width x16, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Not Supported, TimeoutDis-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-, Selectable De-emphasis: -6dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00000  Data: 0000
	Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Kernel driver in use: snd_hda_intel

I think from what I am seeing its actually the HDMI 03.00.1 that is actually causing my problem? I presume it would need to have had the same vfio-pci driver as well. That seems tobe the one thing that stood out to me on this. That said I have already done what I needed in order to pass that as well as the card itself. Anything I did to the 03.00.0 I also did to 03.00.1 as well.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Ok here is the output I have from the lsgroup script. (applicable to group 17 since thats where I am having the issue. also below is the output to the lspci and find commands you specified thanks again.

~#sh lsgroup.sh
### Group 17 ###
    03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT [Radeon R9 270X]
    03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]

~ # lspci -vvv -s 0000:03:00.0
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Curacao XT [Radeon R9 270X] (prog-if 00 [VGA controller])
	Kernel driver in use: vfio-pci

~ # lspci -vvv -s 0000:03:00.1
03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
	Kernel driver in use: snd_hda_intel

I think from what I am seeing its actually the HDMI 03.00.1 that is actually causing my problem? I presume it would need to have had the same vfio-pci driver as well. That seems tobe the one thing that stood out to me on this. That said I have already done what I needed in order to pass that as well as the card itself. Anything I did to the 03.00.0 I also did to 03.00.1 as well.

Yes, you can see from lspci that 3:00.1 is bound to snd_hda_intel, not vfio_pci.  Check your setup procedure.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

firewing1

Member

Registered: 2013-12-17

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

For those of us experiencing the memory allocation bug (guest VM memory is not freed after shutdown), are you passing through USB devices from the host? I just noticed that I encountered the memory issue only after attempting to add a USB mouse attached to the host to my VM,

tuoni

Member

Registered: 2013-12-19

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

For those of us experiencing the memory allocation bug (guest VM memory is not freed after shutdown), are you passing through USB devices from the host? I just noticed that I encountered the memory issue only after attempting to add a USB mouse attached to the host to my VM,

Nope, I've got the memory allocation bug, but the only thing I'm passing through is the VGA card.  This is my invocation:

sudo QEMU_AUDIO_DRV=alsa \
QEMU_AUDIO_TIMER_PERIOD=0 \
qemu-system-x86_64 -enable-kvm -M q35 -m 3072 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-monitor stdio -bios /usr/share/qemu/bios.bin -vga none \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/home/tuoni/kvm/win7.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-soundhw hda,ac97

I'm pretty sure some of those switches are sub-optimal, but I'm just sticking with the first thing I got working

EDIT: One thing I have noticed is there has been a marked decrease in my host (i915) graphics performance since using mainline-rc3 + i915 patch.  I'm presuming it's the kernel rather than the patch...?

Last edited by tuoni (2013-12-26 08:29:08)

xani

Member

Registered: 2013-12-24

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And is someone has managed to use a Sata card with vfio ?

I have two AsMedia 1062 (2*sata3) and it does not work correctly.

I don't have any problems with this controller.

05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])

I am passing it like this

-device vfio-pci,host=05:00.0,bus=pcie.0

Windows 7 is recognising it without any hassle.

For those of us experiencing the memory allocation bug (guest VM memory is not freed after shutdown), are you passing through USB devices from the host? I just noticed that I encountered the memory issue only after attempting to add a USB mouse attached to the host to my VM,

For me it doesn't matter if I'm using usb passthrough or not. It happens when using pci-assing or vfio-pci. Did you try my kernel package? Here's new one. It includes acs override and i915 arbiter fix patches.

Nope, I've got the memory allocation bug, but the only thing I'm passing through is the VGA card.

@tuoni you can test if my package helps on mem leaks too.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And is someone has managed to use a Sata card with vfio ?

I have two AsMedia 1062 (2*sata3) and it does not work correctly.

I don't have any problems with this controller.

05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])

I am passing it like this

-device vfio-pci,host=05:00.0,bus=pcie.0

Windows 7 is recognising it without any hassle.

For those of us experiencing the memory allocation bug (guest VM memory is not freed after shutdown), are you passing through USB devices from the host? I just noticed that I encountered the memory issue only after attempting to add a USB mouse attached to the host to my VM,

For me it doesn't matter if I'm using usb passthrough or not. It happens when using pci-assing or vfio-pci. Did you try my kernel package? Here's new one. It includes acs override and i915 arbiter fix patches.

Nope, I've got the memory allocation bug, but the only thing I'm passing through is the VGA card.

@tuoni you can test if my package helps on mem leaks too.

Thanks a lot for the kernel package, i have linked it at the front page.

xani

Member

Registered: 2013-12-24

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks a lot for the kernel package, i have linked it at the front page.

No problem, it is you that started this little vga passthrough movement in here

Package that I posted also includes patch that disables gcc asm goto instructions, I think that only folks that have problem with memory leaks should use it. It is really low level x86 and gcc stuff that I don't fully understand.
Anyway, I made new one with all those fixes except asm goto: linux-mainline.tar.gz.
This is ofc yours updated to rc5 and patches so I don't want to take all the credit

Val532

Member

Registered: 2013-11-13

Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And is someone has managed to use a Sata card with vfio ?

I have two AsMedia 1062 (2*sata3) and it does not work correctly.

I don't have any problems with this controller.

05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])

I am passing it like this

-device vfio-pci,host=05:00.0,bus=pcie.0

Windows 7 is recognising it without any hassle.

It's strange for me it's work but only once, i can start my VM only once, if my VM reboot (cold reboot or hot reboot) it's bug on VM bios screen. And i have some dmesg :

[ 404.206866] dmar: DRHD: handling fault status reg 2
[ 404.206870] dmar: DMAR:[DMA Write] Request device [07:00.0] fault addr 2affdf000
[ 404.206870] DMAR:[fault reason 05] PTE Write access is not set
[ 404.206877] dmar: DRHD: handling fault status reg 2
[ 404.206879] dmar: DMAR:[DMA Read] Request device [07:00.0] fault addr 2affda000
[ 404.206879] DMAR:[fault reason 06] PTE Read access is not set

What is your config ?

Last edited by Val532 (2013-12-26 15:55:26)

xani

Member

Registered: 2013-12-24

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And is someone has managed to use a Sata card with vfio ?

I have two AsMedia 1062 (2*sata3) and it does not work correctly.

I don't have any problems with this controller.

05:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])

I am passing it like this

-device vfio-pci,host=05:00.0,bus=pcie.0

Windows 7 is recognising it without any hassle.

It's strange for me it's work but only once, i can start my VM only once, if my VM reboot (cold reboot or hot reboot) it's bug on VM bios screen. And i have some dmesg :

[ 404.206866] dmar: DRHD: handling fault status reg 2
[ 404.206870] dmar: DMAR:[DMA Write] Request device [07:00.0] fault addr 2affdf000
[ 404.206870] DMAR:[fault reason 05] PTE Write access is not set
[ 404.206877] dmar: DRHD: handling fault status reg 2
[ 404.206879] dmar: DMAR:[DMA Read] Request device [07:00.0] fault addr 2affda000
[ 404.206879] DMAR:[fault reason 06] PTE Read access is not set

What is your config ?

Sorry to misguide you, but I noticed that this controller is responsible for seabios hang after I posted, and I wanted to test more before writing about it.
I think that when I was hunting for mem leak issue I changed my config to use this controller and I forgot about it.
Anyway, I think that vfio is not properly resetting it. Seabios showed that it cannot send commands to it, but this is only hunch.
I am going back to virtio because I don't have time to debug this and my sata drive that I used to passthrough is slower than using virtio...

Here's my current config if it can help somehow:

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-linux-mainline root=UUID=a63d1c94-4682-431c-9788-0cccd29ee598 rw intel_iommu=on pci-stub.ids=1002:6818,1002:aab0,8086:0
c0c,8086:153b pcie_acs_override=downstream

sudo qemu-system-x86_64 -M q35 \
-m 8196 -enable-kvm -cpu host -monitor stdio \
-smp 4,sockets=1,cores=4,threads=1 \
-rtc clock=host,base=localtime \
-vga none -net none \
-parallel none -serial none -usb \
-drive cache=off,id=disk,file=/home/xani/old_home/xani/obrazy/windows/win-7.raw,format=raw \
-drive if=virtio,cache=off,id=diskc,file=/home/xani/old_home/xani/obrazy/windows/data.raw,format=raw \
-device ahci,bus=pcie.0,id=ahci \
-device ide-hd,bus=ahci.0,drive=disk \
-drive id=isocd,file=/home/xani/downloads/virtio-win-0.1-74.iso \
-device ide-cd,bus=ahci.1,drive=isocd \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=pcie.0 \
-device vfio-pci,host=00:19.0,bus=pcie.0 \
-chardev file,id=seabios,path=/home/xani/seabios.log -device isa-debugcon,iobase=0x402,chardev=seabios

Last edited by xani (2013-12-26 16:16:17)

noobman

Member

Registered: 2013-12-13

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here's my directory from which you can test that memory leak issues.
linux-mainline.tar.gz
Provide me with feedback if you have the same problem with seabios after reverting those patches.

Hi and tyvm. I just used your linux-mainline-x and now the memory behaves nicely: i get it all back now upon closing the vm via qemu window. So i can confirm this fixed me. I just started using it and works great.
I will also try the second link asap, just holidays and etc. Speaking of which: Happy holidays to everybody

Val532

Member

Registered: 2013-11-13

Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@xani :

Ok so i'm not the only one to have this probleme, the funny fact is when you lauch a linux live cd and boot on it with the AsMedia 1062 you can access to your hard drive but you can reboot on it, but you have to wait some minute when Seabios hang (it finish to boot on CD)

So an other Sata controler ?

doubledr

Member

Registered: 2013-12-26

Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I just registered to say thank you to every one. I managed to use vfio to passthrough my nvidia 760 GTX.

My setup:
CPU: Xeon 1230V2
MotherBoard: Tyan S5512. It has an onboard video chip AST 2150 for IPMI, which turns out to be a cause of Code 43.

Host: Gentoo linux: kernel 3.12.6 with 3 patches from this link (which has been shown in this thread): https://lists.fedoraproject.org/piperma … 03916.html NoSnoop patch in top post is no longer necessary.
QEMU: 1.7 with 2 patches also provided in the link.

Note:
if you have an intel cpu with intel video gpu, you need to include the i915 video arbitration patches or you will have Code 43. If you don't want to patch then kernel 3.13 is the first working kernel...
But in my case it is something different, I don't think there is arbitration patch avaliable for AST chips. The only workaround is to set AST as primary video output in bios.

Note 2:
if you have intel CPU and want to use ATI card, I think it still needs some time to make them work with vfio. Black Screen is what I am facing right now. However, ATI cards works perfectly with PCI-STUB. But the last working build I have is qemu 1.5.3 with seabios 1.6.3. Any higher version results in a black screen.

Performance:
I just use windows index to check, everything looks normal except mem access. Qemu with Q35 simulation has some mem access regression. It is much slower than with 440fx：(5.5 vs 7.1). As a result, my 760 GTX doesn't play games faster than my 7790. 250 dollar is wasted...

I would say for ATI card users, stay with PCI-stub, you ready get the best performance. Everything is perfect except the reboot problem. By the way, My via usb 3 controller works flawless with 440fx emulation. With Q35 chipset, I always receive code 10...

For nvidia card users, well, no choices. I never make PCI-stub work for nvidia cards.

Last edited by doubledr (2013-12-27 01:35:07)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

EDIT: One thing I have noticed is there has been a marked decrease in my host (i915) graphics performance since using mainline-rc3 + i915 patch.  I'm presuming it's the kernel rather than the patch...?

This is why the i915 patches were reverted upstream, X wants to mmap the VGA mmio space and will disable DRI if the arbiter reports that more than one VGA device is present.  Therefore i915 has gone back to incorrectly reporting the VGA ranges the hardware decodes.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

dismuter

Member

Registered: 2013-12-27

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've been trying to get passthrough working for a Radeon on an Intel platform, but it seems that it just doesn't work right now (black screens or code 10 in windows).
If I may... Forums are nice to discuss subjects but if anyone is accustomed to creating wikis, one would be nice to summarize information for various cases. 38 pages is starting to be critical in terms of finding the right information, and it's never good to rely solely on the OP to update his post.

if you have intel CPU and want to use ATI card, I think it still needs some time to make them work with vfio. Black Screen is what I am facing right now. However, ATI cards works perfectly with PCI-STUB. But the last working build I have is qemu 1.5.3 with seabios 1.6.3. Any higher version results in a black screen.
[...]
I would say for ATI card users, stay with PCI-stub, you ready get the best performance.

You mean legacy pci-assign? It's a shame that it doesn't work with the latest qemu+seabios then.

xj4reg

Member

Registered: 2013-12-27

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have two Radeon cards on a intel i7 machine, the vga pass through worked, but I meet BSoD on gust VM after install the newest ATI Catalyst driver. The BSoD told the error is on atikmpag.sys.
My HW configuration are - i7 3820, MSI x79a-gd45(8d), ATI Radeon 5400(for host),MSI Hawk ATI Radeon 6850(for gust windows).

I strictly follow the instructions, using the PKGBUILD to seabios, qemu and linux-mainline listed in page 1, but still run into this issue. Do you have any idea?

lspci outputs

02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series]
03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Barts PRO [Radeon HD 6850]
03:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Barts HDMI Audio [Radeon HD 6800 Series]

lspci -n

03:00.0 0300: 1002:6739
03:00.1 0403: 1002:aa88

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=a6859dd0-b64e-4c62-82ac-cbef89dce8ae rw intel_iommu=on vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream pci-stubs.ids=1002:6739,1002:aa88 quiet

vfio-bind.cfg

DEVICES="0000:03:00.0 0000:03:00.1"

qemu script

qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
-smp 2,sockets=1,cores=2,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-usb -usbdevice "host:7.4" -usbdevice "host:7.3" \
-net nic -net user \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=03:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=03:00.1,bus=root.1,addr=00.1 \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/home/tianxj/temp/windows.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk \
-drive file=/home/tianxj/temp/cn_windows_7_ultimate_with_sp1_x64_dvd_u_677408.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-boot menu=on

Last edited by xj4reg (2013-12-27 17:05:43)

techdude300

Member

Registered: 2012-04-09

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Here's my directory from which you can test that memory leak issues.
linux-mainline.tar.gz

Provide me with feedback if you have the same problem with seabios after reverting those patches.

Thanks so much! I can confirm that this does in fact seem to fix the issue. For some reason I was having trouble building it when i tried the first two times, but I got it to work on the 3rd try. All the memory the VM allocates is now returned back to the machine, so I no longer have to reboot the host. A very strange issue. Now all that's left on my end is to figure out sound.

sbd

Member

Registered: 2013-11-21

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

A.W. recommended using kernel 3.13rc + qemu.git OR kernel 3.12.5 + qemu 1.7.0 with the following commits patched in: https://lists.fedoraproject.org/piperma … 03916.html

I can confirm that the 3.12.5 kernel series will work with the method described above, if anyone would prefer not to use 3.13rc. I applied the patches to kernel 3.12.5-302 and 1.7.0-3 on Fedora 20 (qemu package is from rawhide), and I've installed the Catalyst drivers in the Windows 7 guest with no special options (control centre is included). It seems to be working normally.

I did use different i915 patches though, since the ones being linked in this thread are for the 3.13 series and don't compile on 3.12.5. I used the following instead:

81b5c7bc8de3e6f63419139c2fc91bf81dea8a7d
6e1b4fdad5157bb9e88777d525704aba24389bee

Thanks very much for all the advice guys, this thread has been great.

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The windows VM now works perfectly, except if i'm running the open source radeon driver on the host, my entire system freezes when windows boots. So if I blacklist radeon, windows boots perfectly but then I can't use Linux XD Anyone experienced this? :s

Primary GPU for Linux: AMD X1300 (R500 arch)
Secondary GPU for Windows: AMD R7950

root@josh-desktop ~]# dmesg | grep AMD-Vi
[    1.079338] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[    1.079340] AMD-Vi: Interrupt remapping enabled
[    1.091501] AMD-Vi: Lazy IO/TLB flushing enabled
[    3.073444] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x0000000000009000 flags=0x0000]
[    3.073453] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x0000000000005000 flags=0x0020]
[    3.073461] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x0000000000009040 flags=0x0000]
[    3.073468] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x0000000000009080 flags=0x0000]
[    3.073474] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x00000000000090c0 flags=0x0000]
[    3.426471] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0019 address=0x0000000000005400 flags=0x0020]

[root@josh-desktop ~]# dmesg | grep pci-stub
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=UUID=7af60c5d-239a-4347-ae94-424f99fe644d rw vfio_iommu_type1.allow_unsafe_interrupts=1 pci-stub.ids=1002:679a,1002:aaa0 quiet
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=UUID=7af60c5d-239a-4347-ae94-424f99fe644d rw vfio_iommu_type1.allow_unsafe_interrupts=1 pci-stub.ids=1002:679a,1002:aaa0 quiet
[    1.227222] pci-stub: add 1002:679A sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    1.227232] pci-stub 0000:09:00.0: claimed by stub
[    1.227236] pci-stub: add 1002:AAA0 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    1.227242] pci-stub 0000:09:00.1: claimed by stub

[root@josh-desktop ~]# lspci 
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port E)
00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port F)
00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port G)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port A)
00:0b.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (NB-SB link)
00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port B)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] RV516 [Radeon X1300/X1550 Series]
01:00.1 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] RV516 [Radeon X1300/X1550 Series] (Secondary)
02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
05:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
07:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
09:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti PRO [Radeon HD 7950]
09:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]

EDIT: Fixed it by updating BIOS, moving the 2nd CPU to a different PCI port, enabling a 64MB IOMMU mode in the BIOS, adding "iommu=pt iommu=1 amd_iommu=on" to kernel boot and compiling the kernel to have static modules. Not sure which one actually fixed it but it works!

By the way, the linux-mainline package provided in this topic doesn't have the kernel modules statically loaded by default. I had to change the config files manually and skip makepkg integrity check. I know you can enable them in the kernel config dialog but I can never find anything there. I don't understand why they wouldn't be enabled by default in the package since it's possible to do so?

Last edited by BulliteShot (2013-12-28 12:08:08)

noobman

Member

Registered: 2013-12-13

Posts: 21

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@ xj4reg :
I have quite same setup, i7-3820 and dx79to (same x79 chipset), my primary is also ati (hd6570 with open source drivers), passing secondary gtx770. Perhaps not the best for this, but nonetheless i think it should work also for you.
About kernel, i had 'bus options \ pci-stub' built-in (not module) - if its module then see OP. Also ' device drivers \ iommu \ enable intel dma remmapping default ' built-in. And i did built in all virtio (except mem balooning). Perhaps not all this are necesarily like that but i am  just describing mine. For kernel, i had the memory issue with OP ones, so now i use the first link from xani (the link with fixmemory patch, not the second without). About config i used options kvm ignore_msrs=1 and about all the rest from OP. Also i am passing an entire usb controller instead of just devices, that is because i use a kvm switch. So i made the system service from OP and upon startup is good to check lspci -vvv and see kernel driver in use: vfio-pci for devices card, sound, and usb controller if passed like so. My sound works from guest to host with just the first two lines from OP, no other kinky quemu start params.

Good luck!

Last edited by noobman (2013-12-28 13:44:54)