Arch Linux

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I am trying to use Virtio as a standard for network. I followed the libvirt wiki page (http://wiki.libvirt.org/page/Virtio), but I couldn't make it work.

this is how my network configuration looks right now:

<interface type='direct'>
      <mac address='52:54:00:f3:27:c5'/>
      <source dev='eno1' mode='bridge'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>

can someone please help me use Virtio for networking? Thanks a lot!

Last edited by 4kGamer (2014-11-25 19:42:23)

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I am having a reproducable race-condition resulting in multiple kernel panics/oopses under the following condition:

- Use 3.17.2 kernel from OP (need it due to i915 arbiter mess)
- use systemd service to assign VGA card to vfio:

# /etc/systemd/system/vfio.service
[Unit]
Description=Binds devices to vfio-pci
After=syslog.target

[Service]
EnvironmentFile=-/etc/vfio-pci.cfg
Type=oneshot
RemainAfterExit=yes
ExecStart=-/usr/bin/vfio-bind $DEVICES

[Install]
WantedBy=multi-user.target

Contents of /etc/vfio-pci.cfg:

DEVICES="0000:01:00.0 0000:01:00.1"

vfio-bind script:

#!/bin/bash

modprobe vfio-pci

for dev in "$@"; do
    vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
    device=$(cat /sys/bus/pci/devices/$dev/device)
    if [ -e /sys/bus/pci/devices/$dev/driver ]; then
        echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
    fi
    echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id
done

Kernel freaks out when trying to start kdm service (probably the underlying X is causing the problem) in parallel with the vfio service. If i disable them and start them manually one after the other everything is fine, no matter what sequence.

Now I know there are ways to workaround this, and it is not really a big deal for me.
My actual question is rather if this indicates a real/serious bug in the kernel or if this is somehow expected behaviour with the current setup which just should be accepted/ignored? Should I try to track this down in more detail and open a bug somewhere?

dakabali

Member

Registered: 2014-11-11

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

I've made the next step in improving my configuration. After setting up a working PCI passthrough I connected a USB sound card (Creative OMNI Surround 5.1) to the virtual Windows 8.1 machine. It works, however the quality of the sound is not satisfying. It is noisy and seems to depend on the CPU because as the games start the quality decreases. Before going into the details in this forum: can it have something to do with the PCI and/or USB hardware delegations at all? Unfortunately I've got no other PC where I could try a non-virtualized setup. Any experience? Thx.

EDIT:  long story short --> the USB passthrough seems to be very noisy which is ok for mouse and keyboard connections but disturbing for HDA. The solution was to send back the OMNI Surround 5.1 and purchase a PCI Creative Sound Blaster Zx. Having used pci-stub I managed to hide the sound card in Arch and delegate it to Win8.1 in KVM. Without stub the front speakers won't work as Linux reserves the stereo channels by default. Sorry for this off-topic because - as it came out at the end - this has nothing to do with VGA PCI passthrough at all.

Last edited by dakabali (2014-12-10 17:55:33)

snarfies

Member

Registered: 2014-11-05

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

How does one install the linux-mainline-3.17.2.tar.gz on the front page?

I tried pacman -U, but it gave me:

error: missing package metadata in /home/erik/linux-mainline-3.17.2.tar.gz
error: '/home/erik/linux-mainline-3.17.2.tar.gz': invalid or corrupted package

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

How does one install the linux-mainline-3.17.2.tar.gz on the front page?

I tried pacman -U, but it gave me:

error: missing package metadata in /home/erik/linux-mainline-3.17.2.tar.gz
error: '/home/erik/linux-mainline-3.17.2.tar.gz': invalid or corrupted package

It's source package, not an actual one. You need to build it with makepkg.

Hi,

I am trying to use Virtio as a standard for network. I followed the libvirt wiki page (http://wiki.libvirt.org/page/Virtio), but I couldn't make it work.

this is how my network configuration looks right now:

    <interface type='direct'>
          <mac address='52:54:00:f3:27:c5'/>
          <source dev='eno1' mode='bridge'/>
          <model type='rtl8139'/>
          <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
        </interface>

can someone please help me use Virtio for networking? Thanks a lot!

Change rtl8139 to virtio. Done.

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah I tried that. I got no connection then ...

Kironide

Member

Registered: 2014-11-26

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

I know that VGA passthrough using Xen has some issues when AMD graphics cards are used, namely, the guest OS cannot be rebooted.

My curiosity lies in the question of whether the same issue is present when using QEMU for VGA passthrough. I would imagine not, but I felt it best to make sure.

Best regards,

Kironide

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

go here: http://blog.ktz.me/?p=219

reboots do work under Xen

Bronek

Member

From: London

Registered: 2014-02-14

Posts: 123

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I am having a reproducable race-condition resulting in multiple kernel panics/oopses under the following condition:

Contents of /etc/vfio-pci.cfg:

DEVICES="0000:01:00.0 0000:01:00.1"

This looks funny, are you sure these are correct device IDs for your graphics card and its audio when you run lspci -l on host ? Are they bound to pci-stub on system startup ?

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

mine looks exactly the same

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I am trying to use Virtio as a standard for network. I followed the libvirt wiki page (http://wiki.libvirt.org/page/Virtio), but I couldn't make it work.

this is how my network configuration looks right now:

    <interface type='direct'>
          <mac address='52:54:00:f3:27:c5'/>
          <source dev='eno1' mode='bridge'/>
          <model type='rtl8139'/>
          <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
        </interface>

can someone please help me use Virtio for networking? Thanks a lot!

Change rtl8139 to virtio. Done.

Just to be more precise: Windows 8.1 DOES start, but I got no internet connection then.

Last edited by 4kGamer (2014-11-26 10:45:09)

Nesousx

Member

Registered: 2012-03-27

Posts: 46

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

I have a working setup based on Intel CPU and Z77 chipset + GTX 970 as graphic cards. However I have one question, and one issue.

First, here is my question: whats the point using a bios dump (from graphic card) in the qemu command-line? I am not doing it yet and everything seems ok. Good performance, VM reboots ok, etc. FIXED

Then, my little issue. I have an USB sound card : Asus Xonar U7. If I use it on a USB hub (shared with my mouse + keyboard), then I have some sound issues. It stutters sometimes, and causes lag. Only happens during games; never while watching a movie / youtube / listening to music.
I decided to go with emulated audio (cf. command line below). Now i non longer have those sound / lags issue. However I have a little latency in sound. All the time. The sound occurs half a second (or even less) after the real action. I tried with various PA samples from 128 to 16384 (cf. command line). 128 to 512 is unusable, audio lag and video lag. And high values (above 4096) make the latency higher. So far, it seems that the best value is 1024.

Do you have any idea what to do?

Let me know if you need additional information.
Thanks in advance.

Edit ; HDMI audio is not a solution since my screen as no audio jack output (and on screen's speaker are just horrible).

Command line:

QEMU_PA_SAMPLES=1024 QEMU_AUDIO_DRV=pa \
qemu-system-x86_64 -name game -M q35 -enable-kvm \
-mem-path /dev/hugepages \
-balloon virtio \
-nographic \
-m 16384 \
-cpu host,kvm=off -smp 4,sockets=1,cores=4,threads=1 \
-bios /usr/share/qemu/bios.bin -realtime mlock=on -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=pcie.0 \
-device virtio-scsi-pci,id=scsi \
-drive file=/dev/guest/Windaube_7,id=sys,format=raw -device scsi-hd,drive=sys \
-drive file=/dev/guest/win7,id=data,format=raw -device scsi-hd,drive=data \
-device vfio-pci,host=00:1d.0,bus=pcie.0 \
-device virtio-scsi-pci,id=scsi2 \
-net nic,model=virtio,macaddr=52:54:00:7c:3c:42 -net bridge,br=virbr0 \
-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

Sorry to bump, but if anyone has any idea on the sound part, I'll be very happy.

Last edited by Nesousx (2014-11-26 10:55:53)

bpbastos

Member

Registered: 2014-11-21

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

First I'd like thank you for this great post and the valuable information posted here.

I'm trying to get vga passthrough work on ubuntu 14.10, I know here is a Arch Linux forum sorry if I'm being rude or something.

My setup is the follow:

Core i7 4770S
Gigabyte GA-B85M-D3PH
Nvidia Geforce GT630 1GB
AMD R7 250 2GB

I've tried to passthrough both gpus, but I can't get video signal from DVI output connected to VGAs.

I think the problem is related to missing i915 arbiter patch, I haven't found the 915 patch for 3.16 kernel which is the ubuntu 14.10 default kernel.

Does someone has the i915 arbiter patch for 3.16 kernel?

Thank you

Last edited by bpbastos (2014-11-26 11:58:44)

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to be more precise: Windows 8.1 DOES start, but I got no internet connection then.

you need to install virtio network drivers form virtio drivers iso.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Do you have any idea what to do?

Sorry to bump, but if anyone has any idea on the sound part, I'll be very happy.

How do u pass usb devices? i have had same problem with -usb device, but passing whole INTEL usb controller with vifio worked flawlessly.

Last edited by slis (2014-11-26 12:08:24)

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

First I'd like thank you for this great post and the valuable information posted here.

I'm trying to get vga passthrough work on ubuntu 14.10, I know here is a Arch Linux forum sorry if I'm being rude or something.

My setup is the follow:

Core i7 4770S
Gigabyte GA-B85M-D3PH
Nvidia Geforce GT630 1GB
AMD R7 250 2GB

I've tried to passthrough both gpus, but I can't get video signal from DVI output connected to VGAs.

I think the problem is related to missing i915 arbiter patch, I haven't found the 915 patch for 3.16 kernel which is the ubuntu 14.10 default kernel.

Does someone has the i915 arbiter patch for 3.16 kernel?

Thank you

i used this patch for 3.15
http://paste.ubuntu.com/9248971/

if something fails apply manually.

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to be more precise: Windows 8.1 DOES start, but I got no internet connection then.

you need to install virtio network drivers form virtio drivers iso.

thank you so much, of course!

I got it working now

Nesousx

Member

Registered: 2012-03-27

Posts: 46

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Do you have any idea what to do?

Sorry to bump, but if anyone has any idea on the sound part, I'll be very happy.

How do u pass usb devices? i have had same problem with -usb device, but passing whole INTEL usb controller with vifio worked flawlessly.

Unfortunately, I already pass the whole controller:

-device vfio-pci,host=00:1d.0,bus=pcie.0 \

This controller controls 2 front USB ports on my case. From here, I plug an USB hub which contains mouse + keyboard (both wireless Logitech via 1 unify adapter), a micro USB cable (for Xbox gamepad), and the sound card.

PS: I just ordered a new USB soundcard from 20€ from Amazon (free return if it doesn't suits my needs). I'll give it a try to make sure it is not some stupid hardware issue, related to my sound card.

Last edited by Nesousx (2014-11-26 12:40:07)

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Do you have any idea what to do?

Sorry to bump, but if anyone has any idea on the sound part, I'll be very happy.

How do u pass usb devices? i have had same problem with -usb device, but passing whole INTEL usb controller with vifio worked flawlessly.

Unfortunately, I already pass the whole controller:

-device vfio-pci,host=00:1d.0,bus=pcie.0 \

is that intel controller? VIA caused problems for me..

Nesousx

Member

Registered: 2012-03-27

Posts: 46

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

How do u pass usb devices? i have had same problem with -usb device, but passing whole INTEL usb controller with vifio worked flawlessly.

Unfortunately, I already pass the whole controller:

-device vfio-pci,host=00:1d.0,bus=pcie.0 \

is that intel controller? VIA caused problems for me..

Yup:

lspci | grep 00:1d.0
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)

bpbastos

Member

Registered: 2014-11-21

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

First I'd like thank you for this great post and the valuable information posted here.

I'm trying to get vga passthrough work on ubuntu 14.10, I know here is a Arch Linux forum sorry if I'm being rude or something.

My setup is the follow:

Core i7 4770S
Gigabyte GA-B85M-D3PH
Nvidia Geforce GT630 1GB
AMD R7 250 2GB

I've tried to passthrough both gpus, but I can't get video signal from DVI output connected to VGAs.

I think the problem is related to missing i915 arbiter patch, I haven't found the 915 patch for 3.16 kernel which is the ubuntu 14.10 default kernel.

Does someone has the i915 arbiter patch for 3.16 kernel?

Thank you

i used this patch for 3.15
http://paste.ubuntu.com/9248971/

if something fails apply manually.

Thank you slis!

thearcherblog

Member

Registered: 2014-10-30

Posts: 165

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello all,

After reading a lot looking why sometimes my sound is not working on the KVM (Windows 8.1, NVIDIA sound from the GTX 770 card), I think I found the solution (needs to be checked more time but seems to work for now (2 hours testing)

First of all... when I was running:

[jgomez@myArch ~]$ sudo lspci -v -s 01:00.0
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 770] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: eVga.com. Corp. Device 3774
        Flags: bus master, fast devsel, latency 0, IRQ 49
        Memory at ee000000 (32-bit, non-prefetchable) [size=16M]
        Memory at e0000000 (64-bit, prefetchable) [size=128M]
        Memory at e8000000 (64-bit, prefetchable) [size=32M]
        I/O ports at e000 [size=128]
        Expansion ROM at ef000000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+   -------------------> Before was MSI: Enable-
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [b4] Vendor Specific Information: Len=14 <?>
        Capabilities: [100] Virtual Channel
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Capabilities: [900] #19
        Kernel driver in use: vfio-pci
        Kernel modules: nouveau

[jgomez@myArch ~]$ sudo lspci -v -s 01:00.1 //For the Sound part :)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
        Subsystem: eVga.com. Corp. Device 3774
        Flags: bus master, fast devsel, latency 0, IRQ 50
        Memory at ef080000 (32-bit, non-prefetchable) [size=16K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+ -------------------> Before was MSI: Enable-
        Capabilities: [78] Express Endpoint, MSI 00
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

So, reading here,  here and here that changed and seems to work for me

Reading all of that suggests that you try (making backup or whatever to cover your back ):

add pci=msi on your kernel parameters --------> That changes my enabled- to enabled+
and go to your Windows guest, Control Panel, System, Device Manager
View menu, Resources by Type
Check your device (VGA and Sound in my case)
Check if the IRQ it's positive (MSI not enabled) or negative (MSI Enabled) ---------> In my case were positive so... let's go to the next step... remember to make backup, maybe your guest will not boot again!!!
Check the properties of that device, details tab, Device Instance Path and copy that value (something like PCI\VEN_10DE&DEV_1184....
Open your regedit
Go to: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Enum\PCI\ ------------- Your copied value here -----------------\DeviceParameters\Interrupt Management\

Maybe exists maybe not a key named MessageSignaledInterruptProperties (if does not exists, create it)
Create a new DWORD inside named MSISupported (value should be 1 for enabled and 0 for disabled)

Close regedit and reboot your computer

Cross your fingers and start praying

In my case was working and after reboot no more problems with sound (at the moment).

If you go again to the Device Manager you can check that now your IRQ for that device are negative.

If that does not happen, remove the keys on regedit and... bad luck... at least according to the post I read).

This should give us less latency so that means... better performance

I hope it helps for all of you!

Best regards,

TheArcher

P.S.= Thanks a lot to the authors of these blogs / forum because they are genius!

dRaiser

Member

From: Poland

Registered: 2013-05-20

Posts: 51

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, could anyone tell me how exactly CPU assignment in KVM/QEMU works?
I have and i5 3550 CPU with 4 cores and no hyperthreading. I currently gave 2 cores for VM but, as I understand it, it doesn't give cores exclusively, so is it possible to pass all 4 cores and with  CPU load on host near to 0 should it perform like it has native 4 cores on guest? I hope my question is easy to understand.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, could anyone tell me how exactly CPU assignment in KVM/QEMU works?
I have and i5 3550 CPU with 4 cores and no hyperthreading. I currently gave 2 cores for VM but, as I understand it, it doesn't give cores exclusively, so is it possible to pass all 4 cores and with  CPU load on host near to 0 should it perform like it has native 4 cores on guest? I hope my question is easy to understand.

vCPUs are threads and by default are handled the same as any other thread on the host, scheduling on available physical CPUs.  You can pin them to help out the scheduler and improve cache locality.  You also need to realize that there are additional threads that run in the host for I/O, so if you have a 1:1 mapping of vCPU to pCPUs and don't have extra resources for the additional threads, then that time will get stolen from the vCPUs.  It's therefore often advisable to not fully consume the available host CPUs.  cgroups can be used to attempt to further isolate where processes can run and what can run on the same pCPUs as your guest.  There's also always some overhead to virtualization, so you're never going to get 100% of native performance.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, could anyone tell me how exactly CPU assignment in KVM/QEMU works?
I have and i5 3550 CPU with 4 cores and no hyperthreading. I currently gave 2 cores for VM but, as I understand it, it doesn't give cores exclusively, so is it possible to pass all 4 cores and with  CPU load on host near to 0 should it perform like it has native 4 cores on guest? I hope my question is easy to understand.

I use cset to do this, so i have 2 cores dedicated to the host, and 6 for the vm