Arch Linux

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thank you and I am sorry for my inattention.
I have started VM via command bellow, but get black monitor - no signal... could you please help to investigate the issue?

qemu-system-x86_64 -M q35  -m 4096  -enable-kvm -cpu host  -vga none \
-smp 1,sockets=1,cores=2,threads=1 \
-device ahci,bus=pcie.0,id=ahci \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,x-vga=on,addr=0.0,multifunction=on,bus=root.1 \
-device vfio-pci,host=01:00.1,bus=pcie.0 \
-drive file=/run/media/abrilevskiy/CopiedData/Soft/saucy-desktop-amd64.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-drive if=none,id=drive0,cache=none,aio=native,format=raw,file=/home/abrilevskiy/windows.img \
-device virtio-blk,drive=drive0,scsi=off,config-wce=off,x-data-plane=on

I have tried:
modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0
but it did not help

post your hardware specs, and any message you get.

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

i5-3470
AsRock Z77 pro4
Nvidia GT640 - Grid K1 (01:00.0)
ATI HD 7750 (02:00.0)

[    0.368249] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.368333] ------------[ cut here ]------------
[    0.368338] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x77/0x90()
[    0.368339] Modules linked in:
[    0.368341] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.12.0-1-mainline #1
[    0.368342] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77 Pro4, BIOS P1.70 01/17/2013
[    0.368344]  0000000000000009 ffff880243811d20 ffffffff814eb993 0000000000000000
[    0.368346]  ffff880243811d58 ffffffff81061b5d ffff880243560098 ffff880243560000
[    0.368347]  ffff880243560098 0000000000000000 ffff880246c097e0 ffff880243811d68
[    0.368349] Call Trace:
[    0.368354]  [<ffffffff814eb993>] dump_stack+0x54/0x8d
[    0.368357]  [<ffffffff81061b5d>] warn_slowpath_common+0x7d/0xa0
[    0.368359]  [<ffffffff81061c3a>] warn_slowpath_null+0x1a/0x20
[    0.368361]  [<ffffffff812c3e47>] pci_find_upstream_pcie_bridge+0x77/0x90
[    0.368364]  [<ffffffff813d9d8d>] intel_iommu_add_device+0x4d/0x220
[    0.368366]  [<ffffffff813d0fb0>] ? bus_set_iommu+0x50/0x50
[    0.368368]  [<ffffffff813d0fda>] add_iommu_group+0x2a/0x50
[    0.368371]  [<ffffffff81385c93>] bus_for_each_dev+0x63/0xa0
[    0.368372]  [<ffffffff813d0fa8>] bus_set_iommu+0x48/0x50
[    0.368375]  [<ffffffff819279bc>] intel_iommu_init+0xaad/0xba2
[    0.368377]  [<ffffffff810523e5>] ? free_init_pages+0x75/0xa0
[    0.368380]  [<ffffffff818e2156>] ? maybe_link.part.1+0x10f/0x10f
[    0.368382]  [<ffffffff818ea9a3>] ? memblock_find_dma_reserve+0x124/0x124
[    0.368384]  [<ffffffff818ea9b5>] pci_iommu_init+0x12/0x3c
[    0.368386]  [<ffffffff810020fa>] do_one_initcall+0xfa/0x1b0
[    0.368389]  [<ffffffff81081f05>] ? parse_args+0x225/0x3f0
[    0.368391]  [<ffffffff818e1067>] kernel_init_freeable+0x15a/0x1e0
[    0.368393]  [<ffffffff818e089a>] ? do_early_param+0x88/0x88
[    0.368396]  [<ffffffff814de1a0>] ? rest_init+0x90/0x90
[    0.368398]  [<ffffffff814de1ae>] kernel_init+0xe/0x190
[    0.368400]  [<ffffffff814fa4bc>] ret_from_fork+0x7c/0xb0
[    0.368402]  [<ffffffff814de1a0>] ? rest_init+0x90/0x90
[    0.368405] ---[ end trace 8e9a9c119d6070f7 ]---

sudo journalctl --no-pager | grep vfio
Oct 23 00:41:47 abrilevskiy-pc.lan sudo[1078]: root : TTY=pts/1 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:01:00.0 0000:01:00.1
Oct 23 00:41:49 abrilevskiy-pc.lan sudo[1094]: root : TTY=pts/1 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:02:00.0 0000:02:00.1
Oct 23 00:43:38 abrilevskiy-pc.lan sudo[1168]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:00:1a.0
Oct 23 00:43:42 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 00:44:02 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 00:44:02 abrilevskiy-pc.lan kernel: vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Oct 23 00:44:02 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 00:54:58 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 00:54:58 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 00:55:29 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 00:55:29 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 00:56:23 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 00:56:23 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 01:04:07 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:05:13 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:05:57 abrilevskiy-pc.lan kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:09:36 abrilevskiy-pc.lan sudo[1671]: abrilevskiy : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:02:00.0 0000:02:00.1
Oct 23 01:09:36 abrilevskiy-pc.lan kernel:  usb_storage vfio_pci vfio_iommu_type1 vfio fuse isl6423 stv6110x stv090x x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd iTCO_wdt iTCO_vendor_support microcode joydev btusb bluetooth rfkill psmouse pcspkr serio_raw budget budget_core ttpci_eeprom saa7146 snd_hda_codec_hdmi dvb_core r8169 mii snd_hda_codec_realtek i915 snd_hda_intel snd_hda_codec intel_agp intel_gtt snd_hwdep snd_pcm drm_kms_helper video snd_page_alloc i2c_i801 drm snd_timer acpi_cpufreq snd button evdev mei_me i2c_algo_bit processor mei soundcore i2c_core shpchp lpc_ich hid_generic ext4 crc16 mbcache jbd2 usbhid hid sd_mod ahci libahci libata scsi_mod ehci_pci ehci_hcd xhci_hcd usbcore usb_common
Oct 23 01:09:36 abrilevskiy-pc.lan kernel: Oct 23 01:09:36 abrilevskiy-pc.lan kernel: CPU: 2 PID: 1672 Comm: vfio-bind Tainted: G        W    3.12.0-1-mainline #1
Oct 23 01:09:43 localhost.localdomain sudo[1767]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:02:00.0 0000:02:00.1
Oct 23 01:09:51 localhost.localdomain kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:09:51 localhost.localdomain kernel: vfio-pci 0000:02:00.0: enabling device (0000 -> 0003)
Oct 23 01:09:51 localhost.localdomain kernel: vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
Oct 23 01:10:19 localhost.localdomain kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:10:19 localhost.localdomain kernel: vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
Oct 23 01:10:53 localhost.localdomain kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:10:53 localhost.localdomain kernel: vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
Oct 23 01:11:06 localhost.localdomain kernel: vfio-pci 0000:02:00.0: irq 45 for MSI/MSI-X
Oct 23 01:11:12 localhost.localdomain kernel: vfio-pci 0000:02:00.1: irq 48 for MSI/MSI-X
Oct 23 01:14:12 localhost.localdomain kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:14:12 localhost.localdomain kernel: vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
Oct 23 01:14:21 localhost.localdomain kernel: vfio_cap_init: 0000:00:1a.0 hiding cap 0xa
Oct 23 01:14:21 localhost.localdomain kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:20:17 abrilevskiy-pc.lan sudo[1325]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:01:00.0 0000:01:00.1
Oct 23 20:20:21 abrilevskiy-pc.lan sudo[1344]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:02:00.0 0000:02:00.1
Oct 23 20:23:04 abrilevskiy-pc.lan kernel: vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Oct 23 20:23:04 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:23:53 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:24:32 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:26:51 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:30:30 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:38:28 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:40:02 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:40:48 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 23 20:41:43 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 24 00:24:33 abrilevskiy-pc.lan sudo[3886]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:01:00.0 0000:01:00.1
Oct 24 00:24:40 abrilevskiy-pc.lan sudo[4036]: root : TTY=pts/2 ; PWD=/home/abrilevskiy ; USER=root ; COMMAND=./vfio-bind 0000:02:00.0 0000:02:00.1
Oct 24 00:25:12 abrilevskiy-pc.lan kernel: vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
Oct 24 00:25:12 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 24 00:26:17 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 24 00:28:20 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
Oct 24 00:32:28 abrilevskiy-pc.lan kernel: vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900

Last edited by myweb (2013-10-23 21:50:16)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

i5-3470
AsRock Z77 pro4
Nvidia GT640 - Grid K1 (01:00.0)
ATI HD 7750 (02:00.0)

[    0.368249] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.368333] ------------[ cut here ]------------
[    0.368338] WARNING: CPU: 0 PID: 1 at drivers/pci/search.c:46 pci_find_upstream_pcie_bridge+0x77/0x90()
[    0.368339] Modules linked in:
[    0.368341] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.12.0-1-mainline #1
[    0.368342] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77 Pro4, BIOS P1.70 01/17/2013
[    0.368344]  0000000000000009 ffff880243811d20 ffffffff814eb993 0000000000000000
[    0.368346]  ffff880243811d58 ffffffff81061b5d ffff880243560098 ffff880243560000
[    0.368347]  ffff880243560098 0000000000000000 ffff880246c097e0 ffff880243811d68
[    0.368349] Call Trace:
[    0.368354]  [<ffffffff814eb993>] dump_stack+0x54/0x8d
[    0.368357]  [<ffffffff81061b5d>] warn_slowpath_common+0x7d/0xa0
[    0.368359]  [<ffffffff81061c3a>] warn_slowpath_null+0x1a/0x20
[    0.368361]  [<ffffffff812c3e47>] pci_find_upstream_pcie_bridge+0x77/0x90
[    0.368364]  [<ffffffff813d9d8d>] intel_iommu_add_device+0x4d/0x220
[    0.368366]  [<ffffffff813d0fb0>] ? bus_set_iommu+0x50/0x50
[    0.368368]  [<ffffffff813d0fda>] add_iommu_group+0x2a/0x50
[    0.368371]  [<ffffffff81385c93>] bus_for_each_dev+0x63/0xa0
[    0.368372]  [<ffffffff813d0fa8>] bus_set_iommu+0x48/0x50
[    0.368375]  [<ffffffff819279bc>] intel_iommu_init+0xaad/0xba2
[    0.368377]  [<ffffffff810523e5>] ? free_init_pages+0x75/0xa0
[    0.368380]  [<ffffffff818e2156>] ? maybe_link.part.1+0x10f/0x10f
[    0.368382]  [<ffffffff818ea9a3>] ? memblock_find_dma_reserve+0x124/0x124
[    0.368384]  [<ffffffff818ea9b5>] pci_iommu_init+0x12/0x3c
[    0.368386]  [<ffffffff810020fa>] do_one_initcall+0xfa/0x1b0
[    0.368389]  [<ffffffff81081f05>] ? parse_args+0x225/0x3f0
[    0.368391]  [<ffffffff818e1067>] kernel_init_freeable+0x15a/0x1e0
[    0.368393]  [<ffffffff818e089a>] ? do_early_param+0x88/0x88
[    0.368396]  [<ffffffff814de1a0>] ? rest_init+0x90/0x90
[    0.368398]  [<ffffffff814de1ae>] kernel_init+0xe/0x190
[    0.368400]  [<ffffffff814fa4bc>] ret_from_fork+0x7c/0xb0
[    0.368402]  [<ffffffff814de1a0>] ? rest_init+0x90/0x90
[    0.368405] ---[ end trace 8e9a9c119d6070f7 ]---

Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on,igfx_on,pass-through pcie_acs_override=downstream quiet

Boot with intel_iommu=on and remove the other stuff, also nvidia drivers on the host requires a patch

...
-device vfio-pci,host=01:00.0,x-vga=on,addr=0.0,multifunction=on,bus=root.1 \
-device vfio-pci,host=01:00.1,bus=pcie.0 \
...

Are you trying to passthrough your primary card?

Last edited by nbhs (2013-10-23 21:57:30)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Are you trying to passthrough your primary card?

No, my primary card is Integrated video.
I compiled kernel with pcie_acs_override support, booted with:
Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream quiet and got working Live Ubuntu on ATI HD 7750. Also, Windows 8 is installed and working!!! Thank you very much!!! Graphics performance is 7.3 for 3d/2d.

But I can not do this for Nvidia...

Last edited by myweb (2013-10-23 22:49:53)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Are you trying to passthrough your primary card?

No, my primary card is Integrated video.
I compiled kernel with pcie_acs_override support, booted with:
Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream quiet and got working Live Ubuntu on ATI HD 7750. Also, Windows 8 is installed and working!!! Thank you very much!!! Graphics performance is 7.3 for 3d/2d.

But I can not do this for Nvidia...

nice!

SpacePirate

Member

Registered: 2013-09-16

Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I finally got my setup working! What caused the trouble was apparently a buggy IOMMU implementation on my Mainboard (Gigabyte 970A-UD3). I switched to an Asrock 990FX Extreme3 since that was the only affordable Mainboard which was confirmed to be working in this topic i could find.

Thanks for the help!

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Are you trying to passthrough your primary card?

No, my primary card is Integrated video.
I compiled kernel with pcie_acs_override support, booted with:
Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream quiet and got working Live Ubuntu on ATI HD 7750. Also, Windows 8 is installed and working!!! Thank you very much!!! Graphics performance is 7.3 for 3d/2d.

But I can not do this for Nvidia...

nice!

Could you please specify what steps I should perform in order to investigate why my Nvidia card does not work?

teekay

Member

Registered: 2011-10-26

Posts: 271

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

No, my primary card is Integrated video.
I compiled kernel with pcie_acs_override support, booted with:
Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream quiet and got working Live Ubuntu on ATI HD 7750. Also, Windows 8 is installed and working!!! Thank you very much!!! Graphics performance is 7.3 for 3d/2d.

But I can not do this for Nvidia...

nice!

Could you please specify what steps I should perform in order to investigate why my Nvidia card does not work?

If you are trying to passthrough your nvidia card (and use your ATI for the host), either try to get rid of the nvidia drivers on the host, or simply pci-stub it.
Or are you trying to pass through both? (that's a bit unclear..)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I would like to setup and run simultaneously 2 virtual PC:
#1 Windows with ATI 7750 passthrough - works
#2 Linux with Nvidia GT 640 passthrough - not works
Host Linux should use Intel video card embeded in CPU

with -vga none I always get "no signal" on Nvidia GT 640 passthrough.
If boot to Kubuntu 12.10 live without specified "-vga none" monitor detecs signal and show black screen or some console messages from boot.
Newer version of Kubuntu does not boot to live: KDE crashed somehow (before dialog asking boot type: Live or Install), shows GUI that asking to enter password

nouveau and radeon modules are blacklisted

Seabios log:

Start bios (version ?-20130921_165137-mnt-storage-buildroots-extra-x86_64-eric)                                                                                                                                                                                       
No Xen hypervisor found.                                                                                                                                                                                                                                              
Running on KVM
Ram Size=0xb0000000 (0x0000000050000000 high)
Relocating init from 0x000e22d1 to 0xaffe0950 (size 62952)
Found QEMU fw_cfg
CPU Mhz=3200
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
PCI: pci_bios_init_bus_rec bdf = 0xe0
PCI: primary bus = 0x0
PCI: secondary bus = 0xff -> 0x1
PCI: pci_bios_init_bus_rec bus = 0x1
PCI: subordinate bus = 0x0 -> 0x1
=== PCI device probing ===
Found 10 PCI devices (max PCI bus is 01)
=== PCI new allocation pass #1 ===
PCI: check devices
PCI: secondary bus 1 size 00001000 type io
PCI: secondary bus 1 size 02000000 type mem
PCI: secondary bus 1 size 20000000 type prefmem
=== PCI new allocation pass #2 ===
PCI: map device bdf=00:01.0  bar 1, addr 0000d000, size 00000040 [io]
PCI: map device bdf=00:1f.3  bar 4, addr 0000d040, size 00000040 [io]
PCI: map device bdf=00:03.0  bar 4, addr 0000d080, size 00000020 [io]
PCI: map device bdf=00:1f.2  bar 4, addr 0000d0a0, size 00000020 [io]
PCI: map device bdf=00:01.0  bar 6, addr fe000000, size 00040000 [mem]
PCI: map device bdf=00:01.0  bar 0, addr fe040000, size 00020000 [mem]
PCI: map device bdf=00:04.0  bar 0, addr fe060000, size 00004000 [mem]
PCI: map device bdf=00:02.0  bar 0, addr fe064000, size 00001000 [mem]
PCI: map device bdf=00:03.0  bar 5, addr fe065000, size 00001000 [mem]
PCI: map device bdf=00:1f.2  bar 5, addr fe066000, size 00001000 [mem]
PCI: map device bdf=01:00.0  bar 5, addr 0000c000, size 00000080 [io]
PCI: map device bdf=01:00.0  bar 0, addr fc000000, size 01000000 [mem]
PCI: map device bdf=01:00.0  bar 6, addr fd000000, size 00080000 [mem]
PCI: map device bdf=01:00.0  bar 1, addr d0000000, size 10000000 [prefmem]
PCI: map device bdf=01:00.0  bar 3, addr e0000000, size 02000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:29c0
PCI: init bdf=00:01.0 id=8086:100e
PCI: init bdf=00:02.0 id=8086:1e2d
PCI: init bdf=00:03.0 id=8086:2922
PCI: init bdf=00:04.0 id=10de:0e1b
PCI: init bdf=00:1c.0 id=8086:3420
PCI: init bdf=00:1f.0 id=8086:2918
Q35 LPC init: elcr=00 0c
Using pmtimer, ioport 0xb008, freq 3579 kHz
PCI: init bdf=00:1f.2 id=8086:2922
PCI: init bdf=00:1f.3 id=8086:2930
PCI: init bdf=01:00.0 id=10de:0fc1
PCI: Enabling 01:00.0 for primary VGA
PCI: Setting VGA enable on bridge 00:1c.0
Found 1 cpu(s) max supported 1 cpu(s)
MP table addr=0x000f17c0 MPC table addr=0x000f17d0 size=232
SMBIOS ptr=0x000f17a0 table=0x000f1670 size=301
ACPI DSDT=0xafffd8e0
ACPI tables: RSDP=0x000f1640 RSDT=0xafffd8a0
Scan for VGA option rom
Running option rom at c000:0003

Last edited by myweb (2013-10-24 22:19:12)

SpacePirate

Member

Registered: 2013-09-16

Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I`ve got some trouble getting the sound working for my setup (everything else works fine). When I use your script from the first post i hear some scratches when the VM starts but windows doesnt have any soundoutput.
The device manager has an "AMD High Definition Audio device" and an "Unknown device". Do you know what I could be doing wrong?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I`ve got some trouble getting the sound working for my setup (everything else works fine). When I use your script from the first post i hear some scratches when the VM starts but windows doesnt have any soundoutput.
The device manager has an "AMD High Definition Audio device" and an "Unknown device". Do you know what I could be doing wrong?

What OS are you using? 32bit or 64bit? there should be a "high definition audio device" somewhere on the device manager thats the emulated sound card

EDIT: i updated my script to use cpuset (https://code.google.com/p/cpuset/)

Last edited by nbhs (2013-10-24 23:11:20)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have enabled vfio debug, please find Debug output via link: https://dl.dropboxusercontent.com/u/526 … ug.log.zip
and stubbed Nvidia card:

dmesg | grep stub
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream pci-stub.ids=10de:0fc1,10de:0e1b quiet
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=51388cdb-7737-469b-80db-268b2f41e16c ro intel_iommu=on pcie_acs_override=downstream pci-stub.ids=10de:0fc1,10de:0e1b quiet
[    0.418370] pci-stub: add 10DE:0FC1 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.418383] pci-stub 0000:01:00.0: claimed by stub
[    0.418392] pci-stub: add 10DE:0E1B sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.418399] pci-stub 0000:01:00.1: claimed by stub

Last edited by myweb (2013-10-25 06:18:15)

SpacePirate

Member

Registered: 2013-09-16

Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I`ve got some trouble getting the sound working for my setup (everything else works fine). When I use your script from the first post i hear some scratches when the VM starts but windows doesnt have any soundoutput.
The device manager has an "AMD High Definition Audio device" and an "Unknown device". Do you know what I could be doing wrong?

What OS are you using? 32bit or 64bit? there should be a "high definition audio device" somewhere on the device manager thats the emulated sound card

EDIT: i updated my script to use cpuset (https://code.google.com/p/cpuset/)

I use Windows and Arch 64Bit.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I`ve got some trouble getting the sound working for my setup (everything else works fine). When I use your script from the first post i hear some scratches when the VM starts but windows doesnt have any soundoutput.
The device manager has an "AMD High Definition Audio device" and an "Unknown device". Do you know what I could be doing wrong?

What OS are you using? 32bit or 64bit? there should be a "high definition audio device" somewhere on the device manager thats the emulated sound card

EDIT: i updated my script to use cpuset (https://code.google.com/p/cpuset/)

I use Windows and Arch 64Bit.

Well unless you're using an old version of windows XP the audio drivers should be automatically installed.

alphahere

Member

Registered: 2013-10-12

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Recently upgrade packages to the latest version.

I had to apply the two patches list below for vga-passthrough for i915 (built in graphics card). The i915 was is the host and not the guest. This was recommend in a previous post and works well.

http://o.cs.uvic.ca:20810/perl/cid.pl?c … ba24389bee
http://o.cs.uvic.ca:20810/perl/cid.pl?c … f81dea8a7d

Anyway will there be a more permanent solution for the Intel GPU in the future?

This may not be the best place to post this question but I thought I give it a shot here anyway.

Thanks.

SpacePirate

Member

Registered: 2013-09-16

Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What OS are you using? 32bit or 64bit? there should be a "high definition audio device" somewhere on the device manager thats the emulated sound card

EDIT: i updated my script to use cpuset (https://code.google.com/p/cpuset/)

I use Windows and Arch 64Bit.

Well unless you're using an old version of windows XP the audio drivers should be automatically installed.

http://i.imgur.com/UMzAKXW.jpg

I use Windows 7. For me it looks almost the same like your screenshot, but instead of the "High Definition Audio Device" I`ve got an "Unkown Device". When I uninstalled it and searched for plug`n`play devices in the Devicemanager, it took a while, and then I`ve got an BSOD.

Is there another Device that I could use?
Could the Error be produces by the fact, that the iso i used for installation is fairly old and that I dont have every update for Win7 yet, since it only updates some things per reboot (-.-) ?

EDIT: i switched the device to "intel-hda" (ich6) and now it works perfectly

Last edited by SpacePirate (2013-10-27 21:31:11)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dear All,

Could you please help to disable i915 driver right after boot?
I do not need GUI on IGD, I just need that all PCIE videocards are available for passthrough. As far as I understand (Alex, thank you for explanation) i915 driver does not allow to execute PCIE passthrough videocard in guest VM.
Thank you.

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

AAA!!! I blacklisted i915 module, booted to multi-user.target and Nvidia GT640 is successfully displayed Seabios messages and booted to Kubutu Live!!!

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

AAA!!! I blacklisted i915 module, booted to multi-user.target and Nvidia GT640 is successfully displayed Seabios messages and booted to Kubutu Live!!!

Gratz, have you tried applying the patches from the previous alphahere post?

Recently upgrade packages to the latest version.

I had to apply the two patches list below for vga-passthrough for i915 (built in graphics card). The i915 was is the host and not the guest. This was recommend in a previous post and works well.

http://o.cs.uvic.ca:20810/perl/cid.pl?c … ba24389bee
http://o.cs.uvic.ca:20810/perl/cid.pl?c … f81dea8a7d

Anyway will there be a more permanent solution for the Intel GPU in the future?

This may not be the best place to post this question but I thought I give it a shot here anyway.

Thanks.

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

AAA!!! I blacklisted i915 module, booted to multi-user.target and Nvidia GT640 is successfully displayed Seabios messages and booted to Kubutu Live!!!

Gratz, have you tried applying the patches from the previous alphahere post?

Recently upgrade packages to the latest version.

I had to apply the two patches list below for vga-passthrough for i915 (built in graphics card). The i915 was is the host and not the guest. This was recommend in a previous post and works well.

http://o.cs.uvic.ca:20810/perl/cid.pl?c … ba24389bee
http://o.cs.uvic.ca:20810/perl/cid.pl?c … f81dea8a7d

Anyway will there be a more permanent solution for the Intel GPU in the future?

This may not be the best place to post this question but I thought I give it a shot here anyway.

Thanks.

I did not apply the patches - I can not open the links...

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

AAA!!! I blacklisted i915 module, booted to multi-user.target and Nvidia GT640 is successfully displayed Seabios messages and booted to Kubutu Live!!!

Gratz, have you tried applying the patches from the previous alphahere post?

Recently upgrade packages to the latest version.

I had to apply the two patches list below for vga-passthrough for i915 (built in graphics card). The i915 was is the host and not the guest. This was recommend in a previous post and works well.

http://o.cs.uvic.ca:20810/perl/cid.pl?c … ba24389bee
http://o.cs.uvic.ca:20810/perl/cid.pl?c … f81dea8a7d

Anyway will there be a more permanent solution for the Intel GPU in the future?

This may not be the best place to post this question but I thought I give it a shot here anyway.

Thanks.

I did not apply the patches - I can not open the links...

First link

Second link

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

AAA!!! I blacklisted i915 module, booted to multi-user.target and Nvidia GT640 is successfully displayed Seabios messages and booted to Kubutu Live!!!

Gratz, have you tried applying the patches from the previous alphahere post?

Recently upgrade packages to the latest version.

I had to apply the two patches list below for vga-passthrough for i915 (built in graphics card). The i915 was is the host and not the guest. This was recommend in a previous post and works well.

http://o.cs.uvic.ca:20810/perl/cid.pl?c … ba24389bee
http://o.cs.uvic.ca:20810/perl/cid.pl?c … f81dea8a7d

Anyway will there be a more permanent solution for the Intel GPU in the future?

This may not be the best place to post this question but I thought I give it a shot here anyway.

Thanks.

I did not apply the patches - I can not open the links...

Glad you got it working, I suppose blacklisting i915 is another way to work around it's broken-ness with VGA arbiter, but vgacon doesn't play nicely with arbitration either, it just doesn't opt-out.  Here are (hopefully) working links to the i915 patches:

http://git.kernel.org/cgit/linux/kernel … f81dea8a7d
http://git.kernel.org/cgit/linux/kernel … ba24389bee

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Gratz, have you tried applying the patches from the previous alphahere post?

I did not apply the patches - I can not open the links...

Glad you got it working, I suppose blacklisting i915 is another way to work around it's broken-ness with VGA arbiter, but vgacon doesn't play nicely with arbitration either, it just doesn't opt-out.  Here are (hopefully) working links to the i915 patches:

http://git.kernel.org/cgit/linux/kernel … f81dea8a7d
http://git.kernel.org/cgit/linux/kernel … ba24389bee

And how did the 7750 work?  Did you happen to be using a pre-rc5 kernel that already included these?

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

coolper22@gmail.com

Member

From: Shanghai,China

Registered: 2013-10-18

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

first, thanks for the greatest post.

I got i915 (built in graphics card) and GeForce GTX 650 working on windows 7 64bit, the hardware list:
Mainboard: GIGABYTE Z87M-D3H
CPU: i5-4430
Graphic Card: i915 (built in graphics card)
Graphic Card: GeForce GTX 650
I use kernel 3.12 rc6 with two patches about i915 that previous post, and qemu from git master with NoSnoop patch.

But on windows 7 32bit, in device manager,the "Intel(R) 5520/5500/X58 I/O 集线器 PCI Express 根端口 0 - 3420" driver show error code 12,This device cannot find enough free resources that it can use.
I try boot windows 7 32bit using q35(or only -readconfig ../docs/q35-chipset.cfg) without graphics card passthrough,it has the same problem.
I also try seabios rel-1.7.3.2,rel-1.7.3.1,rel-1.7.3,rel-1.7.2.2 and qemu 1.6.1,but no success.

Can somebody give me some suggestions?
I build qemu,seabios on Ubuntu 12.04 64bit.

Anybody passthrough graphics card to windows 7 32 bit successfully?

Anybody passthrough graphics card to windows xp 32 bit successfully?

Last edited by coolper22@gmail.com (2013-10-29 08:15:34)

empie

Member

From: The Netherlands

Registered: 2013-06-15

Posts: 9

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Will try blacklisting the i915 again. Up till now only Code 43 with Nvidia with i915 not on blacklist.
Got my 6870 working with 13.9 driver. Had to do the WorkaroundInstall=true trick to get it working even when it's missing a file. Played Skyrim and Tombraider for a few minutes and looked good. Used Intel audio for sound.
Using Kernel 3.12.0-rc4 on my ASrock Z77 WS and i7 3770.

@myweb: How do you do the vfio binding? Per device, or using the script from nbhs?

Is the pcie_acs_override useful? Haven't used it up till now.