Arch Linux

PrinzipDesSees

Member

Registered: 2015-06-04

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Another thing:

My host has no X installed.
Can I pass the output of a virtual machine to a second display?  As in completely not passing through any GPU but just having a virtual GPU throw the image on monitor 2 without installing X.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Another thing:

My host has no X installed.
Can I pass the output of a virtual machine to a second display?  As in completely not passing through any GPU but just having a virtual GPU throw the image on monitor 2 without installing X.

You want to make QXL output on some hardware screen, right?

You could make the host headless(without X or any other video output) by passing through your main GPU inside some VM and having no other GPU left for host.

Last edited by Duelist (2015-06-12 03:20:15)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I installed ovmf-svn from aur and now have the following files:

[michael@stingray ~]$ find /usr/share/ovmf/
/usr/share/ovmf/
/usr/share/ovmf/ia32
/usr/share/ovmf/ia32/ovmf_vars_ia32.bin
/usr/share/ovmf/ia32/ovmf_code_ia32.bin
/usr/share/ovmf/ia32/ovmf_ia32.bin
/usr/share/ovmf/x64
/usr/share/ovmf/x64/ovmf_vars_x64.bin
/usr/share/ovmf/x64/ovmf_code_x64.bin
/usr/share/ovmf/x64/ovmf_x64.bin

Trying to set up a new VM with virt-manager complains that libvirt did not detect any UEFI Bios, so i can't switch the firmware to UEFI. For which files is libvirt/virtmanager looking? Probably for some *.fd instead of .bin?

PrinzipDesSees

Member

Registered: 2015-06-04

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You want to make QXL output on some hardware screen, right?

You could make the host headless(without X or any other video output) by passing through your main GPU inside some VM and having no other GPU left for host.

Yes.

That seems excessive, cant I make it use just the monitor?

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

[facepalm]

EDIT: QEMU does support SDL and SDL does not necessarily require X.  But, if you're not doing VGA/GPU device assignment, it's not relevant to this thread.

Last edited by aw (2015-06-12 14:09:59)

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

supertrampx

Member

Registered: 2015-06-07

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Vga passthrough working with gtx 970 on sr-2 classified and dual xeon 5680s. The only issue I have noticed is a freeze on first boot of a new  game. I reboot qemu / wondows and the issue is gone and game loads. Any ideas? Also sound stutters slightly.

Thanks in advance.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I installed ovmf-svn from aur and now have the following files:

[michael@stingray ~]$ find /usr/share/ovmf/
/usr/share/ovmf/
/usr/share/ovmf/ia32
/usr/share/ovmf/ia32/ovmf_vars_ia32.bin
/usr/share/ovmf/ia32/ovmf_code_ia32.bin
/usr/share/ovmf/ia32/ovmf_ia32.bin
/usr/share/ovmf/x64
/usr/share/ovmf/x64/ovmf_vars_x64.bin
/usr/share/ovmf/x64/ovmf_code_x64.bin
/usr/share/ovmf/x64/ovmf_x64.bin

Trying to set up a new VM with virt-manager complains that libvirt did not detect any UEFI Bios, so i can't switch the firmware to UEFI. For which files is libvirt/virtmanager looking? Probably for some *.fd instead of .bin?

The old way

The right way:
/etc/libvirt/qemu.conf
# Location of master nvram file
#
# When a domain is configured to use UEFI instead of standard
# BIOS it may use a separate storage for UEFI variables. If
# that's the case libvirt creates the variable store per domain
# using this master file as image. Each UEFI firmware can,
# however, have different variables store. Therefore the nvram is
# a list of strings when a single item is in form of:
#   ${PATH_TO_UEFI_FW}:${PATH_TO_UEFI_VARS}.
# Later, when libvirt creates per domain variable store, this
# list is searched for the master image.
nvram = [ "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd" ]

BUT! As noted in aw's blog:

If UEFI is not available, your libvirt and virt manager tools may be too old.

So the above lines are needed only for old versions.

Vga passthrough working with gtx 970 on sr-2 classified and dual xeon 5680s. The only issue I have noticed is a freeze on first boot of a new  game. I reboot qemu / wondows and the issue is gone and game loads. Any ideas? Also sound stutters slightly.

Thanks in advance.

Maybe your "game" tries to detect your GPU and as we know - all "auto-detecting" software like driver-detect stuff generate BSODs or some other weird behaviour. A strange example is GeForce Experience not allowing you to capture video if you don't have a "real" CPUID.
A workaround would be to use host-passthrough CPU model and ignore_msrs=1 in kvm module options for some weird software to work right.
You can see CPU-z query the CPU by doing rdmsr calls and dmesg tells you that that msrs are unknown.
Another reason for that would be... Maan, dual xeon? Socket 771, am i right? I'd crash the first time i'd see that system, not just a game. In other words - you have a very, very, very WEIRD config from VM's view.

@aw, does your face hurt?.. If so - what was there before the edit?..

Last edited by Duelist (2015-06-13 20:11:30)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Vga passthrough working with gtx 970 on sr-2 classified and dual xeon 5680s. The only issue I have noticed is a freeze on first boot of a new  game. I reboot qemu / wondows and the issue is gone and game loads. Any ideas? Also sound stutters slightly.

Thanks in advance.

http://vfio.blogspot.com/2014/09/vfio-i … ndows.html

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

supertrampx

Member

Registered: 2015-06-07

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Awesome thanks guys for the leads. yes this setup is weird and fast "wink", and to make it even more fun, my pci-e ports are behind an Nvidia NF200 controller.
"Socket 771, am i right?"  actually socket : 1366
So from what I understand, specify a cpu in my vm file so that special software calls wont detect that its in a vm and cause a BSOD?
I am unsure of host-passthrough cpu model and what you mean.

Current VM Config
----------------------------------------------------------------------------------------------------------------------------------------
#!/bin/bash

configfile=/etc/vfio-pci1.cfg

vfiobind() {
    dev="$1"
        vendor=$(cat /sys/bus/pci/devices/$dev/vendor)
        device=$(cat /sys/bus/pci/devices/$dev/device)
        if [ -e /sys/bus/pci/devices/$dev/driver ]; then
                echo $dev > /sys/bus/pci/devices/$dev/driver/unbind
        fi
        echo $vendor $device > /sys/bus/pci/drivers/vfio-pci/new_id

}

modprobe vfio-pci

cat $configfile | while read line;do
    echo $line | grep ^# >/dev/null 2>&1 && continue
        vfiobind $line
done

#Startup Windows 7 Guest with VGA-Passthrough
qemu-system-x86_64 -name win7 -enable-kvm -m 12000 -cpu host,kvm=off \
-smp 6,sockets=1,cores=4,threads=2 \
-rtc base=localtime -realtime mlock=off \
-bios /home/coyote/seabios/out/bios.bin \
-device vfio-pci,host=04:00.0,x-vga=on -device vfio-pci,host=04:00.1 \
-usb -usbdevice host:046d:c068 -usbdevice host:046d:c22d -usbdevice host:047f:c010 \
-device virtio-scsi-pci,id=scsi \
-drive file=/home/coyote/win7.img,id=win,format=raw,if=none -device scsi-hd,drive=win \
-drive file=/home/coyote/virtio.iso,id=virtiocd,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
-drive file=/home/coyote/windows7.iso,id=isocd,if=none -device scsi-cd,drive=isocd \
-vga none
exit 0

----------------------------------------------------------------------------------------------------------------------------------------------
Info on my machine

Motherboard : http://www.evga.com/articles/00537/
Processors : http://ark.intel.com/products/47916/Int … -Intel-QPI
RAM : 24 GB ecc 1600 "would be using 48 but if you fill all slots, the ram is forced to run at 1300"
Graphics : http://us.msi.com/product/vga/GTX-970-G … o-overview "run this card in passthrough to windows kvm"
Graphics Host : An old GTX 260 with Nouveau drivers
HD : PNY SSD 250 GB and Crucial c300 250GB
6x  1TB Hard drives for backup in hot swap bays
1x Dell 30 inch ultra sharp for windows
1x Samsung 24 inch HD monitor Linux
OS openSUSE Tumbleweed 4.0.4-3
OS passthrough Windows 7

Last edited by supertrampx (2015-06-14 08:41:52)

supertrampx

Member

Registered: 2015-06-07

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If I can get this running, I may try with 4 graphics cards, as I have the slots for it.

I am also getting some warnings when I run the vm

WARNING: Image format was not specified for '/home/coyote/virtio.iso' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

WARNING: Image format was not specified for '/home/coyote/windows7.iso' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

Last edited by supertrampx (2015-06-14 08:38:02)

supertrampx

Member

Registered: 2015-06-07

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you wish here are some project photos Little bragging here sorry But I am proud and it has been a rocking project!

http://s560.photobucket.com/user/joecay … ibrary/sr2

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks to this thread, for making this possible

Here is my project:
6 hosts in lan for gaming
gigabyte z77, i7 2600, 20gb ram, 120gb ssd, 3tb sata, 2x gtx680 = 12 players/seats
Debian jessie with 3.19 kernel, custom win7 for guests. Running steam instead explorer, with everything locked down hard.

https://www.youtube.com/watch?v=YoLLHKzXbC8

It's been running for 6 months, No problems so far, everything is remote managed. And hosts happen to be on 10user 100mbps local wifi network, so they all can remote play over steam in-home streaming.

TripleSpeeder

Member

Registered: 2011-05-02

Posts: 47

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,

I installed ovmf-svn from aur and now have the following files:

[michael@stingray ~]$ find /usr/share/ovmf/
/usr/share/ovmf/
/usr/share/ovmf/ia32
/usr/share/ovmf/ia32/ovmf_vars_ia32.bin
/usr/share/ovmf/ia32/ovmf_code_ia32.bin
/usr/share/ovmf/ia32/ovmf_ia32.bin
/usr/share/ovmf/x64
/usr/share/ovmf/x64/ovmf_vars_x64.bin
/usr/share/ovmf/x64/ovmf_code_x64.bin
/usr/share/ovmf/x64/ovmf_x64.bin

Trying to set up a new VM with virt-manager complains that libvirt did not detect any UEFI Bios, so i can't switch the firmware to UEFI. For which files is libvirt/virtmanager looking? Probably for some *.fd instead of .bin?

The right way:
/etc/libvirt/qemu.conf
# Location of master nvram file
#
# When a domain is configured to use UEFI instead of standard
# BIOS it may use a separate storage for UEFI variables. If
# that's the case libvirt creates the variable store per domain
# using this master file as image. Each UEFI firmware can,
# however, have different variables store. Therefore the nvram is
# a list of strings when a single item is in form of:
#   ${PATH_TO_UEFI_FW}:${PATH_TO_UEFI_VARS}.
# Later, when libvirt creates per domain variable store, this
# list is searched for the master image.
nvram = [ "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd" ]

Hmm, but these are .fd files, while ovmf-svn builds only .bin files as shown above. I suppose they're not interchangeable, or are they?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If I can get this running, I may try with 4 graphics cards, as I have the slots for it.

I am also getting some warnings when I run the vm

WARNING: Image format was not specified for '/home/coyote/virtio.iso' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

WARNING: Image format was not specified for '/home/coyote/windows7.iso' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

The warnings tell that you should specify the format of your images.

-drive file=/home/coyote/win7.img,id=win,format=raw,if=none -device scsi-hd,drive=win \
-drive file=/home/coyote/virtio.iso,id=virtiocd,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
-drive file=/home/coyote/windows7.iso,id=isocd,if=none -device scsi-cd,drive=isocd

You've only specified the format for win7.img. So it should be like this:

-drive file=/home/coyote/virtio.iso,id=virtiocd,,format=raw,if=none -device ide-cd,bus=ide.1,drive=virtiocd \
-drive file=/home/coyote/windows7.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd

And you're using cpu host model already, so try following aw's advice and make everything use MSI(interrupts, not the vendor).

MultiGPU setup... SLI on nvidia hasn't been observed working yet. I've seen(and using) only CrossFire multiGPU configs. Maybe slis have a different opinion on this.
I've suggested one guy to use HyperSLI to trick nvidia's driver, but it didn't work.

Thanks to this thread, for making this possible

Here is my project:
6 hosts in lan for gaming
gigabyte z77, i7 2600, 20gb ram, 120gb ssd, 3tb sata, 2x gtx680 = 12 players/seats
Debian jessie with 3.19 kernel, custom win7 for guests. Running steam instead explorer, with everything locked down hard.

https://www.youtube.com/watch?v=YoLLHKzXbC8

It's been running for 6 months, No problems so far, everything is remote managed. And hosts happen to be on 10user 100mbps local wifi network, so they all can remote play over steam in-home streaming.

Yeah, that's the right usecase for that tech. But you have two 680s... How does it work? One gpu per VM? Quadrified them and split into many VMs? Or what?

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah it's one gpu per vm, splitting doesn't work, Quadrified (to grid k2) them just for hyper-v and latest driver.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah it's one gpu per vm, splitting doesn't work, Quadrified (to grid k2) them just for hyper-v and latest driver.

So how does it gets 12 players?..

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah it's one gpu per vm, splitting doesn't work, Quadrified (to grid k2) them just for hyper-v and latest driver.

So how does it gets 12 players?..

6 hosts with 2x 680, 2vms per host, one gpu each

Last edited by slis (2015-06-14 22:26:42)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah it's one gpu per vm, splitting doesn't work, Quadrified (to grid k2) them just for hyper-v and latest driver.

So how does it gets 12 players?..

6 hosts with 2x 680, 2vms per host, one gpu each

Aww, come on! That's not so fun. But fun anyway.

Why it doesn't split? It'd be extra awesome to do four VMs per one GPU, making 8 guests out of one host.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yeah it would be nice, but I tried with citrix xen and it wouldn't work, passing worked.

mqddb

Member

Registered: 2015-04-24

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

8.1, AIUI 10 has numerous issues running as a VM

@Punkbob

What build of Windows 10 are you using and did you need to do anything special to get it installed?  I see one report that installing build 10041 requires at least one hyper-v extension to be enabled, which isn't going to sit well with Nvidia drivers.

I am using build 9926 on my most recent VM, and I didn't change anything or enable anything different then I would with a normal Windows 10 install. When I first switched over to Windows 10 a few months ago it initially didn't have any specific display drivers from Nvidia, so I just used the ones from Windows 8.1. However once I try drivers that are specifically released for Windows 10, the screen goes black, and when I reboot it just spits the code 43 error back at me.

Edit: Let me try a later build real quick, I thought I was using one that was more recent, but I think I confused which iso I grabbed off my NAS.

Hi All:
    Is there an method to solve this code 43 issue? I also face this boring problem.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

For me Windows10_InsiderPreview_x64_EN-US_10074.iso, is working fine with 352.84 on gtx680.

WildyLion

Member

Registered: 2015-06-07

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi guys,

Could anybody advise me on how to solve subtle timing issues with a 8.1 guest?
When I'm playing music in foobar, the EQ would stop for a second and the sound will stutter, and when I'm playing games, I still get sudden FPS dropouts.

My hardware
* FX-8350 CPU
* M5A99FX PRO 2.0 M/B
* Sapphire Vapor-X Radeon R9 290 as passthrough GPU (VFIO mode)
* USB controller with k/b, mouse and Xonar U7 passed to the VM via PCI passthrough
* I switch k/b and mouse between the motherboard USB ports with an ATEN CS22U KVM

What I've already done:

* Using i440FX vChipset at the moment (with OVMF)
* kvm-amd nested pages disabled
* Hyper-V extensions enabled for the VM at install time
* Host CPU (FX-8350) cpufreq set to performance
* Hugepages memory backing enabled, with hugetlbfs
* 6 VCPUs pinned to 6 real cores (0-5 respectively)

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi

So my X99-Deluxe exploded out of sudden and took my 5930k with it !

Been a month and I finally replaced it with a X99-E WS + 5820k .

Everything is working fine , but I couldn't get VFIO to bind to my NVIDIA using modprobe.d . Binding my devices using the vfio-bind script is fine .

Modules are loaded early in initramfs , and "options vfio-pci ids=" are specified correctly . My other devices get bound correctly using this method , only NVIDIA cards don't .

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@Denso

If vfio-pci is a module, you need to do something to make sure it gets loaded before the native driver has already claimed the device.  Using the modprobe.d softdep option might be a way to handle this, or you can use the techniques I provide in part 3 of my howto series

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Denso

Member

Registered: 2014-08-30

Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@aw :

Thanks , but I am blacklisting nouveau too . In fact when I issue lspci -k , there is no driver claiming the devices at all . Not the native drivers nor vfio-pci . Changing from universal NVIDIA's PCI IDs to GPU's specific doesn't solve the issue either .