Arch Linux

mafferri

Member

Registered: 2013-11-11

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just wanted to let you know I was able to get the thing working on following HW configurations:

AMD Platform:

• Motherboard: SABERTOOTH 990FX R20 (BIOS rev. 2005)

• CPU: AMD FX-8350 @4Ghz

• Primary VGA: Radeon HD 5850

• Secondary (passed through) VGA: Radeon HD 7870

Intel Platform:

• Motherboard: P9X79 DELUXE (BIOS rev. 4502)

• CPU: Intel i7-3930K

• Primary VGA: Radeon HD 6670

• Secondary (passed through) VGA: Radeon HD 7850

Both platform have also passed one internal USB HUB for keyboard / mouse and internal Intel HDA audio for sound. Network and disk use virtio interface.

Host OS (on both machines) is Archlinux with recent stock 3.12 kernel, qemu (1.7.r29424.g964668b) and seabios (1.7.3.2) from nbhs' first post.

AMD platform worked basically out of the box, because nbhs has very similar setup (chipset is the same).  Intel, however, was a bit tougher to get working because of Catalyst 13.9 atikmpag.sys' code 116 BSOD which was finally solved thanks to recent posts from aw and gentoo-nut by simply ommiting the HDMI Audio completely (or adding it to pcie.0 bus works as well).

I would like to Thank nbhs, aw and all you guys who share the information about setups, related issues and their solutions on this thread.  I wouldn't be able to get the thing working without you :-).

hello hda-intel will work well on the guest? to me makes strange noises and echoes

How did you pass the usb hub? because I can not even spend an entire controller usb3 -. - '
and I have to pass individually keyboard and mouse

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hey guys,

Thanks for the awesome writeup. My basic hardware is:

cpu: amd FX-6300
gpu: amd 7970
mb: ASUS M5A99FX PRO R2.0 (2005 bios)

I previously used this walkthrough to set up fully working passthrough using two graphics cards (using the 7970 as secondary and a really basic ati card as primary), but I just managed to get it working on my primary card alone, using pci_stub. I do see boot graphics and console output from the gfx card (I run all my monitors off of it), but when I run qemu it seems to switch over fine.
I hadn't seen much success with this/had heard a lot about primary passthrough not being possible, but since I'm looking at it, I thought I'd throw my experience into the ring. Now I have more space in my box so I can avoid the Realtek ethernet card!

The problem with primary graphics is detaching the host drivers or making sure the host driver uses the vga arbiter.  Typically if you free the primary device by blacklisting the framebuffer driver or using pci-stub then you're left with vgacon on the primary VGA.  vgacon is completely unaware of VGA arbitration and writes to whatever device owns VGA routing at the time.  That means you could see host console messages on the guest if the guest is in VGA mode, but even in graphics mode those writes could be corrupting the state of the device in the guest.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

juhe

Member

Registered: 2013-08-10

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,

hello hda-intel will work well on the guest? to me makes strange noises and echoes

I'm not sure to which system (AMD/Intel) I maintain you refer to. On Intel, it seems working fine. I didn't do extensive testing/game playing since it is a production machine at work to which I added another VGA for vfio testing (well, I couldn't resit to try it ). It means I didn't play a game on it, just run the "Kalimba test", i.e. run the Kalimba.mp3 from Music/ folder on windows 7 and did some internet browsing while listening to the music. From the feeling and comparison with the other AMD system I'm maintaining (and on which I run couple of games), the sound rarely stutters, even initial login sound seems clean. The stuttering seems more often happening on the AMD system, where I rarely get clean logon sound. However, after some time (probably when everything important gets cached in memory) it runs fine also on AMD system. BTW, thanks nbhs for the npt=0 parameter to kvm-amd kernel module, it really makes difference.

DPC Latency Checker seems to have lower red bars on the Intel platform, but all this is really just my current feeling (YMMV), more elaborate tests (possibly having both systems side by side, which is not that easily possible) would be necessary to compare the systems properly.

How did you pass the usb hub? because I can not even spend an entire controller usb3 -. - '
and I have to pass individually keyboard and mouse

I pass only USB2 controller (I really meant controller in my previous post, not HUB, I am sorry for the confusion). I use these options in my qemu startup command line for Intel system (which I have currently at hand):

	-device vfio-pci,host=00:1a.0,bus=pcie.0,id=usb \
	-device vfio-pci,host=00:1b.0,bus=pcie.0,id=snd \

$ lspci | grep 1[ab]\.0
00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06)
00:1b.0 Audio device: Intel Corporation C600/X79 series chipset High Definition Audio Controller (rev 06)

And both devices get to their own group, thanksfully, even without ACS patch. The lsgroup.sh used below is aw's script stolen from one of his previous post and added ".sh" extension.

$ ./lsgroup.sh | grep -C1 1[ab]\.0
### Group 9 ###
    00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06)
### Group 10 ###
    00:1b.0 Audio device: Intel Corporation C600/X79 series chipset High Definition Audio Controller (rev 06)
### Group 11 ###

I use a wireless keyboard and mouse with a single USB transmitter/receiver, which occupies one USB port.

HTH

Chetyre

Member

Registered: 2013-03-27

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hey guys. With the latest kernel and patches I finally managed to get it working! ...kinda.

I can't pass my usb 3.0 controller for some reason. It simply won't work. I can try to find another controller that works, but I really wanted it to be this one. For now I passed my usb keyboard and mouse only, but I need either the controller or my HUB (which I also can't pass).

The second problem is that it can't stand a reboot. If I try to reboot it will get stuck on a screen with the seabios version.

sbd

Member

Registered: 2013-11-21

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone,

Long time lurker, first time poster here. Thanks very much for the interesting and informative thread, it's been a great help. I've been trying to get VGA passthrough working for some time, and while I've succeeded in getting a secondary passthrough working with Xen I've yet to succeed with KVM. I'm experiencing the same stop 0x00000116 error seen by several others with Catalyst drivers installed, but haven't had any joy solving it with the suggested methods.

Unfortunately at the moment I'm having to do my testing on a Fedora machine (my other machine is indisposed right now, although I'm hoping to have it available again soon, please excuse the foreign OS). It has the following specs:
Asus P8H61M-LE (B3 revision)
i5-2500
HD5770
I was previously using version 1.6.1 of qemu to test, however I am currently using qemu compiled from Alex's git repo with the tag referenced in post 1:
git://github.com/awilliam/qemu-vfio.git tags/vfio-pci-for-qemu-20131003.0

Kernel is 3.11.8 (no special config) but I have checked and VFIO support is compiled in (as modules).

I have succeeded in getting the 5770 passed through as primary adapter to a Windows 7 guest, however when I install the Catalyst drivers and reboot, I get the stop 0x00000116 error, followed by a hard lockup of the machine (screen goes black, machine goes unresponsive, and I have to hard reboot. Before the catalyst drivers are installed everything is fine, but I am really after accelerated graphics. I have passed through a USB controller as well and this appears to function normally, and the virtio drivers for the network card also work fine.

When the stop error occurs, the machine goes completely unresponsive. No messages show on my other monitor (running off the onboard GPU), and the machine stops responding to keyboard and ping. There's nothing in the logs afterwards either.

I have done/tried the following without success:
- Blacklisting the radeon driver and assigning the graphics card to pci-stub in the grub command line.
- Setting options vfio_iommu_type1 allow_unsafe_interrupts=1 in /etc/modprobe.d/vfio_iommu_type1.conf
- Setting clear_emulator_capabilities = 0 in qemu.conf, and setting user and group to root.
- Adding the various /dev/vfio/x devices to cgroup_device_acl in qemu.conf
- Moving the HDMI audio component (01:00.1) of the 5770 to pcie.0, instead of being attached to the pcie root port.
- Attaching 01:00.1 to a pci port.
- Removing the 01:00.1 from the qemu config completely so it isn't passed through.
- Upgrading my motherboard BIOS

I have the guest running off of a ZFS zpool, with snapshots, so I've been able to rollback and reinstall the catalyst drivers under different configurations (for example, installing the drivers without the HDMI audio component). In each case I've encountered the same stop error.

I've checked my dmesg output and the IOMMU appears to be being detected properly (and I have intel_iommu=on set in /etc/default/grub):

[    0.000000] Intel-IOMMU: enabled
[    0.016716] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.016787] IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
[    0.443112] IOMMU 0 0xfed90000: using Queued invalidation
[    0.443114] IOMMU: Setting RMRR:
[    0.443121] IOMMU: Setting identity map for device 0000:00:1d.0 [0xdf0d6000 - 0xdf0e4fff]
[    0.443139] IOMMU: Setting identity map for device 0000:00:1a.0 [0xdf0d6000 - 0xdf0e4fff]
[    0.443148] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.443154] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]

However after the entry for device 00:1f.0 (LPC controller) I get this:

[    0.443154] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.443233] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.443290] ------------[ cut here ]------------
[    0.443295] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x77/0x90()
[    0.443296] Modules linked in:
[    0.443299] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.11.8-200.fc19.x86_64 #1
[    0.443300] Hardware name: System manufacturer System Product Name/P8H61-M LE, BIOS 4501 05/10/2013
[    0.443301]  0000000000000009 ffff8801f3eabce0 ffffffff8164781b 0000000000000000
[    0.443303]  ffff8801f3eabd18 ffffffff8106715d ffff8801f3b04098 ffff8801f3b04000
[    0.443305]  ffff8801f3b04098 ffffffff81cbb0a0 ffff8801f68095a0 ffff8801f3eabd28
[    0.443306] Call Trace:
[    0.443311]  [<ffffffff8164781b>] dump_stack+0x45/0x56
[    0.443313]  [<ffffffff8106715d>] warn_slowpath_common+0x7d/0xa0
[    0.443315]  [<ffffffff8106723a>] warn_slowpath_null+0x1a/0x20
[    0.443317]  [<ffffffff8132b547>] pci_find_upstream_pcie_bridge+0x77/0x90
[    0.443319]  [<ffffffff815262ad>] intel_iommu_add_device+0x4d/0x220
[    0.443322]  [<ffffffff8151d5a0>] ? bus_set_iommu+0x50/0x50
[    0.443323]  [<ffffffff8151d5ca>] add_iommu_group+0x2a/0x50
[    0.443325]  [<ffffffff813effe3>] bus_for_each_dev+0x63/0xa0
[    0.443327]  [<ffffffff8151d598>] bus_set_iommu+0x48/0x50
[    0.443330]  [<ffffffff81d5f4f0>] intel_iommu_init+0xa55/0xb4d
[    0.443332]  [<ffffffff8104f5e5>] ? free_init_pages+0x75/0xa0
[    0.443335]  [<ffffffff81d10694>] ? maybe_link.part.2+0x110/0x110
[    0.443337]  [<ffffffff81d17da9>] ? memblock_find_dma_reserve+0x124/0x124
[    0.443339]  [<ffffffff81d17dbb>] pci_iommu_init+0x12/0x3c
[    0.443341]  [<ffffffff810020fa>] do_one_initcall+0xfa/0x1b0
[    0.443344]  [<ffffffff810866a8>] ? parse_args+0x138/0x400
[    0.443346]  [<ffffffff81d0f078>] kernel_init_freeable+0x177/0x1ff
[    0.443347]  [<ffffffff81d0e898>] ? do_early_param+0x88/0x88
[    0.443350]  [<ffffffff8163db10>] ? rest_init+0x80/0x80
[    0.443352]  [<ffffffff8163db1e>] kernel_init+0xe/0x190
[    0.443354]  [<ffffffff8165696c>] ret_from_fork+0x7c/0xb0
[    0.443356]  [<ffffffff8163db10>] ? rest_init+0x80/0x80
[    0.443361] ---[ end trace 2d03b7eaf7f34bc3 ]---

I don't think it's related though, since I'm not trying to pass through any PCI devices. Here's my lspci for reference:

00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:16.0 Communication controller: Intel Corporation 6 Series/C200 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #2 (rev 05)
00:1b.0 Audio device: Intel Corporation 6 Series/C200 Series Chipset Family High Definition Audio Controller (rev 05)
00:1c.0 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 1 (rev b5)
00:1c.1 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 2 (rev b5)
00:1c.2 PCI bridge: Intel Corporation 6 Series/C200 Series Chipset Family PCI Express Root Port 3 (rev b5)
00:1c.4 PCI bridge: Intel Corporation 82801 PCI Bridge (rev b5)
00:1d.0 USB controller: Intel Corporation 6 Series/C200 Series Chipset Family USB Enhanced Host Controller #1 (rev 05)
00:1f.0 ISA bridge: Intel Corporation H61 Express Chipset Family LPC Controller (rev 05)
00:1f.2 SATA controller: Intel Corporation 6 Series/C200 Series Chipset Family SATA AHCI Controller (rev 05)
00:1f.3 SMBus: Intel Corporation 6 Series/C200 Series Chipset Family SMBus Controller (rev 05)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 5770]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]
03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
05:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8169 PCI Gigabit Ethernet Controller (rev 10)

I know a few other posters have encountered stop 0x00000116 but from what I can tell those issues were resolved for most people by placing 01:00.1 under pcie.0 (or disabling it entirely). Has anyone else had no luck with this method? I'm hoping I've just missed something obvious, but I'm guessing it could also be an issue with my Asus motherboard. My other machine has an Asrock board that I'm hoping to be able to test with soon.

Any advice appreciated. Thanks.

Norcoen

Member

From: Norway

Registered: 2013-11-02

Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Was the solution with deveject.exe at shutdown deleted from the initial post?
I can't find anything about it anymore.
I used it in an older VM and wanted to add my new PCIe Controller, which seems to have he same problem. But I don't remember where I put the entry to do so. Wasn't the taskplaner or mmc (w/e that is)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

...Any advice appreciated. Thanks...

You need the 3.12 kernel, and if you're using the i915 driver you'll also need some fixes

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Was the solution with deveject.exe at shutdown deleted from the initial post?
I can't find anything about it anymore.
I used it in an older VM and wanted to add my new PCIe Controller, which seems to have he same problem. But I don't remember where I put the entry to do so. Wasn't the taskplaner or mmc (w/e that is)

To automate this process three's a tool called "deveject" from http://www.withopf.com/tools/deveject/, extract it somewhere then run it from cmd.
You should see an output like this:

...
'@oem4.inf,%amd6719.1%;AMD Radeon HD 6900 Series' 'PCI\VEN_1002&DEV_6719&SUBSYS_31221682&REV_00\4&184E0D10&0&00E0' [REMOVEABLE]
...

Now press the start button and type gpedit.msc
Go to "Computer Configuration"-->"Windows Settings"-->"Scripts"-->"Shutdown"
Click Properties, then "Add.."
On Script Name put the full path to deveject.exe (C:\Windows\System32\deveject.exe in my case)
On Script Parameters -EjectId:"PCI\VEN_1002&DEV_6719&SUBSYS_31221682&REV_00\4&184E0D10&0&00E0" <--change this acording to your case
If its working you should see windows changing the resolution while shutting down.

sbd

Member

Registered: 2013-11-21

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks for the tips guys, I'll give those suggestions a try.

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys! Can anyone who got this working check which PCI bus does GPU end up on when booting Windows 7? Its in the device manager.

For me its PCI Bus 28, while on bare metal its PCI bus 0.

Last edited by ilya80 (2013-11-23 13:40:22)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys! Can anyone who got this working check which PCI bus does GPU end up on when booting Windows 7? Its in the device manager.

For me its PCI Bus 28, while on bare metal its PCI bus 0.

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Cool this must be something else then.

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys!

Today I successfully hardmodded my GTX 680 to report itself as GRID K2.

When I try to passthrough the card, I get no output on either display outputs though. Nothing suspicious in the logs, just:

Nov 24 20:19:42 cave-lin kernel: [  368.001538] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Nov 24 20:19:42 cave-lin kernel: [  368.001719] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Nov 24 20:20:20 cave-lin kernel: [  405.590323] kvm: zapping shadow pages for mmio generation wraparound

I havent changed anything since last time I run my config, except of device ID to assign to pci-stub during boot. Prior to that passthrough worked and I just wasnt able to install Nvidia drivers in guest.

Any advice?

Last edited by ilya80 (2013-11-24 19:39:38)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys!

Today I successfully hardmodded my GTX 680 to report itself as GRID K2.

When I try to passthrough the card, I get no output on either display outputs though. Nothing suspicious in the logs, just:

Nov 24 20:19:42 cave-lin kernel: [  368.001538] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Nov 24 20:19:42 cave-lin kernel: [  368.001719] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Nov 24 20:20:20 cave-lin kernel: [  405.590323] kvm: zapping shadow pages for mmio generation wraparound

I havent changed anything since last time I run my config, except of device ID to assign to pci-stub during boot. Prior to that passthrough worked and I just wasnt able to install Nvidia drivers in guest.

Any advice?

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.  However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.

With VBIOS you mean the bios on the video card? No I haven't updated that. I`d rather keep the original bios b/c as far as I understand it manages clocks/fans in a way I want it to - as a GTX 680. Is there a way to force seabios to run GPU BIOS even though there is a mismatch?

However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

Thanks for advice! Ill try that.

Update: for some reason WIndows is not able to boot - BSOD with PAGE_FAULT_IN_NONPAGED_AREA code 0x00000050. Sometimes BSOD has different code 0x0000007E ( SYSTEM_THREAD_EXCEPTION_NOT_HANDLED )

I might look into seabios at see if it can be forced to load BIOS even though there is a mismatch.

Last edited by ilya80 (2013-11-25 07:41:30)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.

With VBIOS you mean the bios on the video card? No I haven't updated that. I`d rather keep the original bios b/c as far as I understand it manages clocks/fans in a way I want it to - as a GTX 680. Is there a way to force seabios to run GPU BIOS even though there is a mismatch?

However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

Thanks for advice! Ill try that.

Update: for some reason WIndows is not able to boot - BSOD with PAGE_FAULT_IN_NONPAGED_AREA code 0x00000050. Sometimes BSOD has different code 0x0000007E ( SYSTEM_THREAD_EXCEPTION_NOT_HANDLED )

I might look into seabios at see if it can be forced to load BIOS even though there is a mismatch.

See my first post on how to load your vbios from a dump, but you`ll still need to modify the devid if you`re dumping it from your card

Last edited by nbhs (2013-11-25 10:26:20)

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

See my first post on how to load your vbios from a dump, but you`ll still need to modify the devid if you`re dumping it from your card

Yes, I read it again and see what you mean. I will also have to update checksum after I patch the device ID i think. Thanks for the hint.

Chetyre

Member

Registered: 2013-03-27

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm trying to understand cpuset. nbhs' script uses the number of physical cores, but I have 4 physical and 4 logical (I7 3770); so if I want to have 6 vcpus in my guest what do I do? Do I use -smp 3,sockets=1,cores=3,threads=2 or something like that?

Sorry for the dumb question.

Last edited by Chetyre (2013-11-25 13:26:32)

ilya80

Member

Registered: 2013-11-12

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

do I use -smp 3,sockets=1,cores=3,threads=2

I think this would work. I think cpuset is used to pin CPU's to VM, is it what you want to do?

Chetyre

Member

Registered: 2013-03-27

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

do I use -smp 3,sockets=1,cores=3,threads=2

I think this would work. I think cpuset is used to pin CPU's to VM, is it what you want to do?

Yeah, I want to squeeze as much performance from the processor as possible for the VM.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm trying to understand cpuset. nbhs' script uses the number of physical cores, but I have 4 physical and 4 logical (I7 3770); so if I want to have 6 vcpus in my guest what do I do? Do I use -smp 3,sockets=1,cores=3,threads=2 or something like that?

Sorry for the dumb question.

My script is broken on intel, i didnt take into consideration ht.

Chetyre

Member

Registered: 2013-03-27

Posts: 34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm trying to understand cpuset. nbhs' script uses the number of physical cores, but I have 4 physical and 4 logical (I7 3770); so if I want to have 6 vcpus in my guest what do I do? Do I use -smp 3,sockets=1,cores=3,threads=2 or something like that?

Sorry for the dumb question.

My script is broken on intel, i didnt take into consideration ht.

In that case is there a big difference between taskset and cpuset?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm trying to understand cpuset. nbhs' script uses the number of physical cores, but I have 4 physical and 4 logical (I7 3770); so if I want to have 6 vcpus in my guest what do I do? Do I use -smp 3,sockets=1,cores=3,threads=2 or something like that?

Sorry for the dumb question.

My script is broken on intel, i didnt take into consideration ht.

In that case is there a big difference between taskset and cpuset?

cpuset is a tool for controlling cgroups, what a did is create cpu cgroup with x cores, then move everything to the system cgroup (in my case 2 cores) and run qemu on the user cgroup

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did you also modify the VBIOS for the new device ID?  If the VBIOS doesn't match the card seabios won't run it.

With VBIOS you mean the bios on the video card? No I haven't updated that. I`d rather keep the original bios b/c as far as I understand it manages clocks/fans in a way I want it to - as a GTX 680. Is there a way to force seabios to run GPU BIOS even though there is a mismatch?

You won't have VGA on bare metal without a modified VBIOS either, so our emulation is accurate   The only way to force seabios to use the VBIOS would be to modify the code.

However, with a K2 you can avoid all the craziness of VGA and pass the card to the guest as a secondary display.  Pre-boot and early guest boot will happen on the emulated VGA.  When the Nvidia driver is loaded the emulated VGA is disabled and the assigned device will take over.  Use one of the standard emulated VGA devices and drop x-vga=on from the assigned device in your guest config.

Thanks for advice! Ill try that.

Update: for some reason WIndows is not able to boot - BSOD with PAGE_FAULT_IN_NONPAGED_AREA code 0x00000050. Sometimes BSOD has different code 0x0000007E ( SYSTEM_THREAD_EXCEPTION_NOT_HANDLED )

I might look into seabios at see if it can be forced to load BIOS even though there is a mismatch.

IIRC the PAGE_FAULT_IN_NONPAGED_AREA is the manifestation of the -cpu host problem on Nvidia.  Try -cpu qemu64 or one of the CPU specific types.  Note that you can dump the VBIOS, modify it, then load it with romfile= to avoid flashing it back to hardware.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Val532

Member

Registered: 2013-11-13

Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I need your help,

In my setup i added a sata3 controler on my VM but when i whant to reboot my VM the sata3 controler break the boot.

My sata3 controler is an Asmedia 1061.

I think the probleme is to reset the controler when the VM reboot.

Any idea ?