KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist · 2014-12-14 17:05:45

oegat wrote:

Nvidia driver: 244.11 / 244.75 (same result with both, cannot try older since the gpu is too new)

You've meant 344.11 and 344.75, right?

oegat · 2014-12-14 17:16:15

Duelist wrote:

oegat wrote:
Nvidia driver: 244.11 / 244.75 (same result with both, cannot try older since the gpu is too new)
You've meant 344.11 and 344.75, right?

Yes, sorry.

thearcherblog · 2014-12-15 09:15:01

Hello,

Just to inform that with the new kernel (3.17.6-1 ARCH), I do not need to patch it, just works perfect

My kernel options are now:

rdblacklist=nouveau nouveau.modeset=0 pci-stub.ids=10de:1184,10de:030a,8086:15a1 intel_pstate=disable intel_iommu=on,igfx_off quiet

And performance is the same, or even a bit more now

Regards,

TheArcher

4kGamer · 2014-12-15 10:31:54

Thanks TheArcher, that is great news. I have something to share too:

Ok, everyone with the intention of buying a new motherboard and owners of a motherboard with DUAL LAN, listen up:

I found out something exciting and I want to share it with you. I realize this may be no news whatsoever for the majority here. It was a big one for me though.

I was never happy with the performance of the VM. Even having virtio for disk and network didn't seem to do the trick. Then I tried to passtrough one of the LAN ports as a pci device - and it worked wonders. Disk usage went down to 0% in idle and 4k gaming is finally possible for me in a virtualized Windows 8.1.

Like I said this might not be news and (it is kind of common sense) but I can only advise people taking full advantage of that second LAN port.

A big thanks to aw and nbhs for their help. I am not done configuring, so many more questions to come

btw: I am using direct block device as aw suggested. It has another huge advantage: When I need to I can boot directly into native Windows 8.1 - the same disk I use as my VM.

Hope it helps some of you

Last edited by 4kGamer (2014-12-15 12:25:16)

thearcherblog · 2014-12-15 12:51:09

I have 2 ethernet (Asrock Z97 Extreme 6) as well and I was passing the intel one without problems but with the realtek one... was not able to work fine... (drivers downloaded from the official website)... so, if you have two and passthrough one and is not working... try the other one XD

Regards,

TheArcher

4kGamer · 2014-12-15 13:10:28

yeah, I have the same motherboard! I sent you a mail, did you get it? Need uefi boot working, which still doesn't. Just have a look into your mail account.

thanks a lot!

thearcherblog · 2014-12-15 13:18:56

4kGamer wrote:

yeah, I have the same motherboard! I sent you a mail, did you get it? Need uefi boot working, which still doesn't. Just have a look into your mail account.
thanks a lot!

Hello

I've got you mail right now, but I will you reply here to benefit of all the community (please, moderators, move it to the proper place if necessary).

To boot with UEFI you just need to choose on the boot menu (F11) the UEFI option for your USB / DVD boot device.

Normally you will have the device two times (one with UEFI and one without ).

Please, let us know if solved

Anyway, this forum is plenty of this information on the newbie corner (I need to go there almost everyday to find solutions ).

Regards,

TheArcher

4kGamer · 2014-12-15 13:37:05

ok, thank you. The problem is, it is NOT showing in BIOS. So it must be something else. I use standard grub ...

I think my problem is the actual path of the bootx64.efi/grubx64.efi

thanks a lot!

Last edited by 4kGamer (2014-12-15 13:42:50)

thearcherblog · 2014-12-15 13:43:53

I'm not expert on this but I remember that I had a few problems with it when booting (booting from USB pen drive).

Just check that you don't have secureboot enabled or fastboot (gives a lot of problems).

I hope it helps

4kGamer · 2014-12-15 13:48:11

I don't have secure boot or fast boot enabled. What would really help: can you please tell me 2 things:

1. where is your grub.cfg located and
2. where is your .efi file located? (and what name does it have? grubx64.efi or what?)

That would solve my problem, I am sure

thearcherblog · 2014-12-15 13:59:48

Ok

I'm not using grub, I like gummibot, I just followed the instructions on the Wiki (https://wiki.archlinux.org/index.php/be … guide#GRUB) but:

My EFI file is on: /boot/EFI/Boot/BOOTX64.EFI

I love Gummiboot

Last edited by thearcherblog (2014-12-15 14:19:17)

4kGamer · 2014-12-15 14:04:46

all right, thanks a lot!. I will have a look into that later on. I think I can solve it now My bootx64.efi is located under /boot/efi/EFI/boot

so that really might be it

Last edited by 4kGamer (2014-12-15 14:05:17)

TripleSpeeder · 2014-12-15 14:23:57

thearcherblog wrote:

Hello,
My kernel options are now:
rdblacklist=nouveau nouveau.modeset=0 pci-stub.ids=10de:1184,10de:030a,8086:15a1 intel_pstate=disable intel_iommu=on,igfx_off quiet
And performance is the same, or even a bit more now

What is the benefit of "intel_pstate=disable"?

thearcherblog · 2014-12-15 14:25:54

TripleSpeeder wrote:

thearcherblog wrote:
Hello,
My kernel options are now:
rdblacklist=nouveau nouveau.modeset=0 pci-stub.ids=10de:1184,10de:030a,8086:15a1 intel_pstate=disable intel_iommu=on,igfx_off quiet
And performance is the same, or even a bit more now
What is the benefit of "intel_pstate=disable"?

To be honest, the problem I had was that my CPU (4790K) was all time running at 4.4GHz so I was looking for some information and when I disabled it, then works quite good (it's at 800MHz when it's in idle mode and changing to 4.4GHz when need it. Then my computer's temperature is back to normal I've checked it today with pstate enabled and temperatures and core speeds are back to normal I also run my gummibot with the microcode update before initram... maybe that made the trick

I hope it helps to you

Last edited by thearcherblog (2014-12-16 09:17:56)

Rholk · 2014-12-15 15:37:47

4kGamer:

Hi, is it possible that multiplayer games are slow down by network traffic?

I have 1 nic, a 4790k cpu and using 6 cores for the guest (cpu pinned, but not using cpuset shield, only taskset). While i have good performance in a couple of games (solo), i have really bad performance on "simple games" like Heroes of the storm. Where i barely reach 50fps (native windows was at 80...).
I use a tap bridge, my network card uses virtio.

Can you give some numbers of the gain between kvm network and the new mode?

I also thougt of performance problem with emulated sound, but it doesn't seems to affect this so much. (made a game with sound card disabled, performance wasn't better)

Last edited by Rholk (2014-12-15 15:38:11)

sinny · 2014-12-15 17:10:14

Rholk wrote:

4kGamer:
Hi, is it possible that multiplayer games are slow down by network traffic?
I have 1 nic, a 4790k cpu and using 6 cores for the guest (cpu pinned, but not using cpuset shield, only taskset). While i have good performance in a couple of games (solo), i have really bad performance on "simple games" like Heroes of the storm. Where i barely reach 50fps (native windows was at 80...).
I use a tap bridge, my network card uses virtio.
Can you give some numbers of the gain between kvm network and the new mode?
I also thougt of performance problem with emulated sound, but it doesn't seems to affect this so much. (made a game with sound card disabled, performance wasn't better)

anything might factor - it highly depends on the way application is designed/coded
80 to 50 fps is not that huge of a performance drop, i would say (significant though)

in my case, audio and memory allocations were main slowdowns (jumped to 200-300 fps in lol, from 50-60 fps)
oh, i'm using basic virtio network by the way

Last edited by sinny (2014-12-15 17:10:49)

4kGamer · 2014-12-15 23:30:08

@Rholk:

I can't give you an exact answer. I haven't had the chance to test thoroughly. I can tell you more in a few days, but based on my experiences so far, I would believe performance does depend on network usage. You can also give multi-queue virtio-scsi a try. aw wrote about it a page back.

Krobar · 2014-12-16 01:23:40

4kGamer wrote:

Thanks TheArcher, that is great news. I have something to share too:
Ok, everyone with the intention of buying a new motherboard and owners of a motherboard with DUAL LAN, listen up:
I found out something exciting and I want to share it with you. I realize this may be no news whatsoever for the majority here. It was a big one for me though.
I was never happy with the performance of the VM. Even having virtio for disk and network didn't seem to do the trick. Then I tried to passtrough one of the LAN ports as a pci device - and it worked wonders. Disk usage went down to 0% in idle and 4k gaming is finally possible for me in a virtualized Windows 8.1.
Like I said this might not be news and (it is kind of common sense) but I can only advise people taking full advantage of that second LAN port.
A big thanks to aw and nbhs for their help. I am not done configuring, so many more questions to come

btw: I am using direct block device as aw suggested. It has another huge advantage: When I need to I can boot directly into native Windows 8.1 - the same disk I use as my VM.

Hope it helps some of you

For those of you with supported Intel Nics I find using VF devices also performs better than bridge too (You can pass through the spawned VFs using VFIO).

Krobar · 2014-12-16 01:29:20

Can anyone help me with an example of using cpuset and cgroups to fix the host to cores 0-3, VM1 to cores 4-5, VM2 to core 6 and VM3 to core 7?

anthonym52 · 2014-12-16 04:34:12

4kGamer wrote:

Thanks TheArcher, that is great news. I have something to share too:
Ok, everyone with the intention of buying a new motherboard and owners of a motherboard with DUAL LAN, listen up:
I found out something exciting and I want to share it with you. I realize this may be no news whatsoever for the majority here. It was a big one for me though.
I was never happy with the performance of the VM. Even having virtio for disk and network didn't seem to do the trick. Then I tried to passtrough one of the LAN ports as a pci device - and it worked wonders. Disk usage went down to 0% in idle and 4k gaming is finally possible for me in a virtualized Windows 8.1.
Like I said this might not be news and (it is kind of common sense) but I can only advise people taking full advantage of that second LAN port.
A big thanks to aw and nbhs for their help. I am not done configuring, so many more questions to come

btw: I am using direct block device as aw suggested. It has another huge advantage: When I need to I can boot directly into native Windows 8.1 - the same disk I use as my VM.

Hope it helps some of you

How do you pass a single ethernet port as a PCI device? Do you have benchmarks ?

thearcherblog · 2014-12-16 06:19:28

anthonym52 wrote:

4kGamer wrote:
Thanks TheArcher, that is great news. I have something to share too:
Ok, everyone with the intention of buying a new motherboard and owners of a motherboard with DUAL LAN, listen up:
I found out something exciting and I want to share it with you. I realize this may be no news whatsoever for the majority here. It was a big one for me though.
I was never happy with the performance of the VM. Even having virtio for disk and network didn't seem to do the trick. Then I tried to passtrough one of the LAN ports as a pci device - and it worked wonders. Disk usage went down to 0% in idle and 4k gaming is finally possible for me in a virtualized Windows 8.1.
Like I said this might not be news and (it is kind of common sense) but I can only advise people taking full advantage of that second LAN port.
A big thanks to aw and nbhs for their help. I am not done configuring, so many more questions to come

btw: I am using direct block device as aw suggested. It has another huge advantage: When I need to I can boot directly into native Windows 8.1 - the same disk I use as my VM.

Hope it helps some of you
How do you pass a single ethernet port as a PCI device? Do you have benchmarks ?

Hello,

I do not have benchmark but is quite confortable to me using the second ethernet device as a native one on Windows because then both computers (windows and linux) are on the same LAN without bridge/NAT.

This is the command I use to start the qemu:

qemu-system-x86_64 -enable-kvm -rtc base=localtime -m 8192 -cpu host,kvm=off -drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/ovmf_x64.bin -drive if=pflash,format=raw,file=/home/jgomez/VM/ovmf_x64.bin -smp 4,sockets=1,cores=4 -serial none -parallel none -nodefconfig -drive file=/home/jgomez/VM/win7.img,cache=writeback,if=none,id=drive0,aio=native -device virtio-blk-pci,drive=drive0,ioeventfd=on,bootindex=1 -device virtio-scsi-pci,id=scsi -drive file=/home/jgomez/Downloads/win8.iso,id=iso_install,if=none -device scsi-cd,drive=iso_install -cdrom /home/jgomez/Downloads/virtio.iso -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on -device vfio-pci,host=01:00.1 -device vfio-pci,host=00:19.0 -usb -usbdevice host:046d:c52e -net none -vga none -nographic

Regards,

TheArcher

nbhs · 2014-12-16 11:09:01

Krobar wrote:

Can anyone help me with an example of using cpuset and cgroups to fix the host to cores 0-3, VM1 to cores 4-5, VM2 to core 6 and VM3 to core 7?

Just use cset

cset shield -c 4-5
cset shield --shield --pid $PID1
cset shield --shield --pid $PID2

Then pin with taskset:

taskset -pc 4 $PID1
taskset -pc 5 $PID2

You can get each VCPU pid by running qemu with:

-qmp unix:./qmp-sock,server,nowait

And getting the VCPU'S pids

echo "{ \"execute\": \"qmp_capabilities\" }" "{ \"execute\": \"query-cpus\" }" | socat UNIX-CONNECT:./qmp-sock STDIO

After you're done

cset shield --reset

And here's the cset build for arch cpuset.tar.gz

Last edited by nbhs (2014-12-16 11:25:30)

anthonym52 · 2014-12-16 15:35:05

thearcherblog wrote:

This is the command I use to start the qemu:

qemu-system-x86_64 -enable-kvm -rtc base=localtime -m 8192 -cpu host,kvm=off -drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/ovmf_x64.bin -drive if=pflash,format=raw,file=/home/jgomez/VM/ovmf_x64.bin -smp 4,sockets=1,cores=4 -serial none -parallel none -nodefconfig -drive file=/home/jgomez/VM/win7.img,cache=writeback,if=none,id=drive0,aio=native -device virtio-blk-pci,drive=drive0,ioeventfd=on,bootindex=1 -device virtio-scsi-pci,id=scsi -drive file=/home/jgomez/Downloads/win8.iso,id=iso_install,if=none -device scsi-cd,drive=iso_install -cdrom /home/jgomez/Downloads/virtio.iso -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on -device vfio-pci,host=01:00.1 -device vfio-pci,host=00:19.0 -usb -usbdevice host:046d:c52e -net none -vga none -nographic

Regards,

TheArcher

I can't seem to get the PCI stub ID claimed, both interfaces have the same ID, do your IDs match as well?

05:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
06:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]

[anthony:/home/anthony] 7s $ dmesg |grep -i claim
[ 0.332788] pci 0000:00:1f.0: quirk: [io 0x0500-0x053f] claimed by ICH6 GPIO
[ 4.332685] pci-stub 0000:03:00.0: claimed by stub
[ 4.332702] pci-stub 0000:03:00.1: claimed by stub

apostmax · 2014-12-16 15:51:00

Any news on passing through a Intel IGD?
Is it possible at all? I am interested in passing through a HD 4600 (Intel i7 4790)

magemaster · 2014-12-16 16:00:20

Someone else using with arch kernel?

uname -a

Linux 3.17.6-1-ARCH #1 SMP PREEMPT Sun Dec 7 23:43:32 UTC 2014 x86_64 GNU/Linux

cat /proc/cmdline

BOOT_IMAGE=/vmlinuz-linux root=UUID=e9f18698-c70a-41f6-b5c6-c84a8e7eac85 rw quiet rdblacklist=nouveau nouveau.modeset=0 pci-stub.ids=10de:10c3,10de:0be3 intel_pstate=disable i915.enable_hd_vgaarb=1 intel_iommu=on,igfx_off quiet

Tried this way, based on thearcherblog, gave me no error, but no screen

sudo qemu-system-x86_64 -enable-kvm -rtc base=localtime -cpu host,kvm=off -m 2048 -smp 2,sockets=1,cores=2 -serial none -parallel none -nodefconfig \
-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/ovmf_x64.bin \
-device virtio-blk-pci,drive=drive0,ioeventfd=on,bootindex=1 \
-device virtio-scsi-pci,id=scsi \
-drive file=/home/user/windows.img,cache=writeback,if=none,id=drive0,aio=native \
-drive file=/home/user/Win81.iso,id=iso_install,if=none \
-device scsi-cd,drive=iso_install \
-cdrom /home/user/Downloads/virtio.iso \
-device vfio-pci,host=01:00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1 \
-net none \
-vga none \
-boot menu=on \
-nographic

and this way, based on nbhs, gave me this error: KVM internal error. Suberror: 1 emulation failure

sudo qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
-smp 2,sockets=1,cores=2,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

Arch Linux

#3576 2014-12-14 17:05:45

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3577 2014-12-14 17:16:15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3578 2014-12-15 09:15:01

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3579 2014-12-15 10:31:54

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3580 2014-12-15 12:51:09

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3581 2014-12-15 13:10:28

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3582 2014-12-15 13:18:56

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3583 2014-12-15 13:37:05

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3584 2014-12-15 13:43:53

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3585 2014-12-15 13:48:11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3586 2014-12-15 13:59:48

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3587 2014-12-15 14:04:46

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3588 2014-12-15 14:23:57

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3589 2014-12-15 14:25:54

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3590 2014-12-15 15:37:47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3591 2014-12-15 17:10:14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3592 2014-12-15 23:30:08

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3593 2014-12-16 01:23:40

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3594 2014-12-16 01:29:20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3595 2014-12-16 04:34:12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3596 2014-12-16 06:19:28

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3597 2014-12-16 11:09:01

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3598 2014-12-16 15:35:05

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3599 2014-12-16 15:51:00

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#3600 2014-12-16 16:00:20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Board footer