You are not logged in.

#1551 2014-04-14 14:24:57

FruitieX
Member
Registered: 2008-07-03
Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Val532 wrote:

And i have not any problem.

Qemu 2.0.0-RC2 + original kernel 3.14 from kernel.org + i915, acs (and other litle patch on KVM side but without it, it works the same).

Strange. I tried compiling the 3.14 kernel with the i915 and acs patches from the first post but am still having this issue. I'm using qemu from latest git.

I investigated this a bit more and found that this error message is generated when I have both compiled my kernel with pre-emption on and am using the hv-time qemu cpu flag. You didn't happen to change the kernel configuration from the one in the first post?

FruitieX

Offline

#1552 2014-04-14 16:20:55

scoobydog
Member
Registered: 2014-04-14
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Torello wrote:

Hello,
Finally, vga-passthrough work. ( Using kernel with debug registers patch, qemu and seabios from first post ).
But I have a strange issue, which makes keyboard and mouse in guest os - unusable.

My specs:
Core i7 4771, GA-Z87X-OC , Radeon 6950@6970 for guest and integrated intel video for host.
Keyboard Microsoft Natural 4000 ( usb ), mouse Logitech Mx510 ( usb ).

My startup script is:
qemu-system-x86_64 -enable-kvm -M q35 -m 16384 -cpu host \
-smp 4,sockets=1,cores=4,threads=1 \
-vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-drive file=/home/torello/virt_images/windows7.img,id=disk,format=raw -device ide-hd,bus=ide.0,drive=disk \
-drive file=/home/torello/distr/win7_msdn.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd \
-boot menu=on

If I boot vm without "-vga none".
I can switch to qemu window, and it grabs my keyboard and mouse input.

If I boot with "-vga none", I can't use keyboard in guest at all, and with my mouse I can do some operations like scrolling the window using the thumb, or change window place ( using mouse ), but If I want to click a button I must click 20-30 times, trying different corners, before I was able to press any onscreen button.

Do you have any suggestions, how I can debug/solve that issue?

Use a dummy console with -vga none -device qxl

So you don't need to passthru USB devices for testings.

Scoobydog

Offline

#1553 2014-04-14 16:42:53

Torello
Member
Registered: 2014-04-01
Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

novist wrote:

@Torello: pass-through usb ports too.

Thanks, but I want to use one keyboard and mouse on my system. I have already done that with some old version of qemu.

scoobydog wrote:

@Torello: Use a dummy console with -vga none -device qxl
So you don't need to passthru USB devices for testings.
Scoobydog

Thanks! That's exactly what I need, now mouse and keyboard fully functional. Hurray!

Offline

#1554 2014-04-14 17:14:31

pereczes
Member
Registered: 2014-03-08
Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

after my precious time went wasted with a crap PowerColor card, that turned out to be faulty (lot of freezes also before with opencl/darktable), received my GTX TI 750, works like charm on the guest with nvidia drivers! finally an old dream became true... can do photography on virtual machine with dedicated graphic card!

Offline

#1555 2014-04-14 18:36:44

Torello
Member
Registered: 2014-04-01
Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Now there is only one problem left - how to pass my 2tb gpt disk to qemu?
If I use that command:
-drive file=/dev/sdc2,id=diske,format=raw -device ide-hd,bus=ide.2,drive=diske
I got 5 or 6 partitions in windows disk manager( but actually there is only one partition ).
That's how I see my disk using lsblk -f

sdc    promise_fasttrack_raid_member                                           
├─sdc1 promise_fasttrack_raid_member                                           
└─sdc2 ntfs                          2 Tb

If I mount sdc2 under linux, using ntfs-3g, all works fine.

Offline

#1556 2014-04-14 20:19:21

scoobydog
Member
Registered: 2014-04-14
Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Torello wrote:

Now there is only one problem left - how to pass my 2tb gpt disk to qemu?
If I use that command:
-drive file=/dev/sdc2,id=diske,format=raw -device ide-hd,bus=ide.2,drive=diske
I got 5 or 6 partitions in windows disk manager( but actually there is only one partition ).
That's how I see my disk using lsblk -f

sdc    promise_fasttrack_raid_member                                           
├─sdc1 promise_fasttrack_raid_member                                           
└─sdc2 ntfs                          2 Tb

If I mount sdc2 under linux, using ntfs-3g, all works fine.

The sdc2 is a patition, in your gast PC it's a HDD with Partition Table, the best way is to use LVM oder Image Files, not single Partions.

User -- qemu-img crate -f raw fileneme.img sizeG -- to create a new file in the filesystem, but keep in mind, ntfs mountet to Linux is very slow.

ScoobyDog

Offline

#1557 2014-04-14 22:13:17

FruitieX
Member
Registered: 2008-07-03
Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

anickname wrote:

Okay I managed to restore VM performance to what it was on 3.13. As a temporary fix to get rid of the dmesg spam, here's a very ugly hack (the warning is probably there for a reason):

Just comment out the line that generates these warnings, in the kernel source file arch/x86/kvm/x86.c, line 1120:

Change:

	WARN_ON(preemptible());

into:

	//WARN_ON(preemptible());

and recompile your kernel.

Now the dmesg spam is gone, but I found performance is still terrible especially in the VM, but I noticed it's not as good as it used to be in the host OS either. Turns out that for some reason, even though I used the default "powersave" governor in both kernels, in 3.14 the governor would actually *do* something and thus my CPU frequencies were crazy low all the time. (in 3.13 the CPU frequency was maxed out all the time)

To fix this I set the default governor to "performance" in /etc/default/cpupower, and enabled the cpupower systemd service. (install the cpupower package first!) Now my VM performance is as good as on 3.13!

Hope this helps!
FruitieX

Offline

#1558 2014-04-15 00:02:41

Cubex
Member
Registered: 2014-04-06
Posts: 24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Those who have problems with keyboard missing key press/releases and similiar, try:

-usbdevice mouse -usbdevice keyboard

This worked as good as passing usb device, with the advantage that I can release/grab anytime

Offline

#1559 2014-04-16 03:06:37

rabcor
Banned
Registered: 2013-02-09
Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Can this work with the current stable arch kernel (3.14.0-5-ARCH) without recompiling it?

Those are it's default settings

$ zcat /proc/config.gz | grep VFIO
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_KVM_VFIO=y

$ zcat /proc/config.gz | grep STUB
CONFIG_EFI_STUB=y
CONFIG_PCI_STUB=m

Offline

#1560 2014-04-16 03:57:44

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

rabcor wrote:

Can this work with the current stable arch kernel (3.14.0-5-ARCH) without recompiling it?

Those are it's default settings

$ zcat /proc/config.gz | grep VFIO
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_KVM_VFIO=y

$ zcat /proc/config.gz | grep STUB
CONFIG_EFI_STUB=y
CONFIG_PCI_STUB=m

Depends on your hardware, if you're planning on using the intel IGP then no

Offline

#1561 2014-04-16 06:58:33

rabcor
Banned
Registered: 2013-02-09
Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Offline

#1562 2014-04-16 14:46:18

Eb
Member
Registered: 2014-04-08
Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

Have you tried swapping card to check if you get the same result? Have you tried to passthrough only the gpu without the audio device?

I've removed the audio device, and it works! Been playing around with it for a few hours and no issues! Thanks a bunch.

I really appreciate all the help everyone has given me, keep being cool guys. =]

Last edited by Eb (2014-04-16 14:46:54)

Offline

#1563 2014-04-16 17:27:43

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

rabcor wrote:

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Yeah that will work, though you'll need to patch the nvidia drivers

Offline

#1564 2014-04-16 17:46:45

andy123
Member
Registered: 2011-11-04
Posts: 169
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
rabcor wrote:

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Yeah that will work, though you'll need to patch the nvidia drivers

Or use that other unsupported method, if you can. Yes, I'm talking about UEFI boot/efifb.


i'm sorry for my poor english wirting skills…

Offline

#1565 2014-04-17 15:32:58

pereczes
Member
Registered: 2014-03-08
Posts: 14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

andy123 wrote:
nbhs wrote:
rabcor wrote:

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Yeah that will work, though you'll need to patch the nvidia drivers

Or use that other unsupported method, if you can. Yes, I'm talking about UEFI boot/efifb.


donno if helps, as stated before for me GTX 750 works on guest (windows + original nvidia driver) and with GeForce 7900 on host (nouveau)

Offline

#1566 2014-04-18 11:24:47

Krobar
Member
Registered: 2014-04-12
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi All,

I'm using Vanilla packages and found I can always start a VM with a Geforce 620 in VFIO pass through but never restart it, the VM looks to have restarted on the host but no display appears. Libvirt gave stability while running the VM but did not help with the restart problem. Any ideas or do I have to run a custom kernel / packages?

Hardware:
Opteron 4256EE
Asus KCMR-D12

Software:
Kernel 3.14.0-4-ARCH
Qemu V1.7.0

Offline

#1567 2014-04-18 16:45:29

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Krobar wrote:

Hi All,

I'm using Vanilla packages and found I can always start a VM with a Geforce 620 in VFIO pass through but never restart it, the VM looks to have restarted on the host but no display appears. Libvirt gave stability while running the VM but did not help with the restart problem. Any ideas or do I have to run a custom kernel / packages?

Hardware:
Opteron 4256EE
Asus KCMR-D12

Software:
Kernel 3.14.0-4-ARCH
Qemu V1.7.0

You need qemu =>2.0 or qemu-git

Offline

#1568 2014-04-18 18:39:53

spam
Member
Registered: 2013-06-15
Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I've been trying adding acs to my config but still had no luck my guest works only as a basic video card, I get a code 43 in windows. I am gonna try a linux guest to get traces. When I try to load a vbios with the romfile option my guest has no outputs at all. Is someone has an idea of what else could I try ? My gtx 660 is not UEFI compatible and I start my computer in UEFI mode, could it be a problem ?

Last edited by spam (2014-04-18 18:40:39)

Offline

#1569 2014-04-19 11:40:15

Krobar
Member
Registered: 2014-04-12
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
Krobar wrote:

Hi All,

I'm using Vanilla packages and found I can always start a VM with a Geforce 620 in VFIO pass through but never restart it, the VM looks to have restarted on the host but no display appears. Libvirt gave stability while running the VM but did not help with the restart problem. Any ideas or do I have to run a custom kernel / packages?

Hardware:
Opteron 4256EE
Asus KCMR-D12

Software:
Kernel 3.14.0-4-ARCH
Qemu V1.7.0

You need qemu =>2.0 or qemu-git

Thanks for the reply. I enabled the testing repo and now have this QEMU version:

pacman -Q qemu
qemu 2.0.0-2
qemu-x86_64 -version
qemu-x86_64 version 2.0.0, Copyright (c) 2003-2008 Fabrice Bellard

I'm still having the same problem though. VM destroy and create again or reboot of the VM means no display. Its strange because otherwise it works well, XBMC Gotham is running with HD video decode and HDMI audio passthrough.

Offline

#1570 2014-04-19 19:18:04

rabcor
Banned
Registered: 2013-02-09
Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
rabcor wrote:

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Yeah that will work, though you'll need to patch the nvidia drivers

Cool! patching the drivers is a lot less of a pain than patching the kernel i think, since I would need to reconfigure the kernel every time i update it but with nvidia drivers i can always just patch them again. I may want to start by trying what pereczes said and just use noveau on the linux host at first. I guess this means I'll give this whole thing a third attempt pretty soon smile

Did anyone try asking around on the nvidia forums yet?

Last edited by rabcor (2014-04-19 19:25:34)

Offline

#1571 2014-04-19 19:28:03

andy123
Member
Registered: 2011-11-04
Posts: 169
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

rabcor wrote:
nbhs wrote:
rabcor wrote:

What about Nvidia GTX-670(VM) and Nvidia GTX-550-Ti(Host)?

Yeah that will work, though you'll need to patch the nvidia drivers

Cool! patching the drivers is a lot less of a pain than patching the kernel i think, since I would need to reconfigure the kernel every time i update it but with nvidia drivers i can always just patch them again. I may want to start by trying what pereczes said and just use noveau on the linux host at first. I guess this means I'll give this whole thing a third attempt pretty soon smile

Did anyone try asking around on the nvidia forums yet?

There's a thread on there asking about the inclusion of the patch: https://devtalk.nvidia.com/default/topi … tion-lock/


i'm sorry for my poor english wirting skills…

Offline

#1572 2014-04-19 20:13:52

dismuter
Member
Registered: 2013-12-27
Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I'm trying to get VGA passthrough working on the following setup:
- Asus Q87 motherboard
- Intel Core i7 4771
- For host: Intel HD
- For guest: AMD Radeon 7770 OR NVIDIA GeForce GTX 660 (not both, whichever I can get to work first)

IOMMU is enabled through kernel switch.
I disabled all the modules which would interfere with guest graphics cards (radeon, nouveau, nvidia)
I did the vfio-bind.

Here's my command line:

qemu-kvm \
-machine pc-q35-2.0 -enable-kvm -cpu host -m 4096 \
-smp 2,sockets=1,cores=2,threads=2 \
-bios /usr/share/qemu/bios.bin -vga none -display none \
-usb -usbdevice host:046d:c52b \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device virtio-scsi-pci,id=scsi0,bus=pcie.0,addr=0x3 \
-drive file=/var/lib/libvirt/images/vm.img,if=none,id=drive-scsi0-0-0-0,format=raw,discard=unmap,aio=threads \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

(-display none is because there is no X server; otherwise I get "Gtk-WARNING **: cannot open display")

Results:
- Radeon on guest guests monitor out of standby, displays fine until guest drivers are loaded under Windows
- GeForce throttles fan, but monitor stays in standby, nothing gets displayed (see edit)

On the software side, I'm running Fedora 20 with kernel 3.15-rc1 (from rawhide) and qemu 2.0.
The kernel was compiled with the following config:

CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_KVM_VFIO=y

I read that those versions contained sufficient logic for getting this to work. But is that really the case, do they require any patch, must I go through recompiling them?
Does using the integrated graphics for the host have any influence?
What am I missing?

Thanks for any help


EDIT: I got the NVIDIA card working completely apparently by blacklisting i915.
I can even use libvirt by adding this:

  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='vfio-pci,host=01:00.1,bus=root.1,addr=00.1'/>
  </qemu:commandline>

Replace "01:00.x" with your output from lspci:

01:00.0 VGA compatible controller: NVIDIA Corporation GK106 [GeForce GTX 660] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK106 HDMI Audio Controller (rev a1)

(for those interested, don't forget to add the namespace at the beginning of the domain xml: <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>)
I did also face this error:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=pcie.0,addr=07.0,multifunction=on,x-vga=on: vfio: error opening /dev/vfio/1: Operation not permitted

and fixed it by adding /dev/vfio/1 to cgroup_device_acl in /etc/libvirt/qemu.conf, setting both user and group to "root" and clear_emulator_capabilities to 0

EDIT 2: Well, thanks to this post from MacCoaster, who had a similar configuration (Intel Haswell + Asus MB + R7770), I added the following to the kernel command line:

vfio_iommu_type1.allow_unsafe_interrupts=1 kvm_intel.emulate_invalid_guest_state=0 nohz=off

I don't know which of these got it working with the Radeon HD 7770, but it seems quite stable, I can even run the latest 3D Mark smoothly (BSOD with the NVIDIA, but I should try with the new kernel command line).

Last edited by dismuter (2014-04-20 01:01:58)

Offline

#1573 2014-04-20 01:09:38

Krobar
Member
Registered: 2014-04-12
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Dismuter, does your VM reboot OK? I had to disable an Intel HDMI Audio driver to get my GF620 to passthrough, might be related to I915.

Offline

#1574 2014-04-20 01:27:19

dismuter
Member
Registered: 2013-12-27
Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Krobar wrote:

Dismuter, does your VM reboot OK? I had to disable an Intel HDMI Audio driver to get my GF620 to passthrough, might be related to I915.

Both with the Radeon and the GeForce, it reboots without any problem.

Offline

#1575 2014-04-21 08:58:29

Krobar
Member
Registered: 2014-04-12
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Anyone know if VFIO can be used with SR-IOV/VNICs? Have an Intel 1000ET dual card and it looks like Intel Virtual Function NICs might be easier to setup in Arch than a bridge.

Offline

Board footer

Powered by FluxBB