Attempting VGA Passthrough, no video output

knknif · 2016-01-12 06:35:28

Hey guys, I am attempting to get VGA pass through working so I can have a "once-and-for-all" solution to gaming on Linux. I am following this guide:

https://www.reddit.com/r/linux_gaming/c … de_on_how/

He has helped me out a couple times but unfortunately I am still struggling. I will try to "prove" that various conditions have been met with the following snippits, but please let me know if there is more information you need. Thank you.

#lspci -nnk (after binding, to me this proves that pci-stub is configured correctly, as well as the actual binding of the vfio-pci drivers)

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 680] [10de:1180] (rev a1)
	Subsystem: eVga.com. Corp. Device [3842:2686]
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)
	Subsystem: eVga.com. Corp. Device [3842:2686]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

#dmesg | grep -i iommu

[    0.000000] DMAR: IOMMU enabled
[    0.025431] DMAR-IR: IOAPIC id 8 under DRHD base  0xfed91000 IOMMU 1
[    0.412440] iommu: Adding device 0000:00:00.0 to group 0
[    0.412449] iommu: Adding device 0000:00:01.0 to group 1
[    0.412454] iommu: Adding device 0000:00:02.0 to group 2
blah blah blah, no errors

#more/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="resume=UUID=7270807a-c77a-4eb0-a0e0-21642bdb3c61 qui
et intel_iommu=on pci-stub.ids=10de:1180,10de:0e0a"

#more /etc/mkinitcpio.conf

MODULES="dm_mod dm_crypt ext4 aes_x86_64 sha256 sha512 pci-stub"

#more /usr/bin/windows

#!/bin/bash

cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
qemu-system-x86_64 \
  -enable-kvm \
  -m 2048 \
  -cpu host,kvm=off \
  -vga none \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd

Paths in the "windows" script are correct, files exist.

When I start the windows script, the QEMU "compat_monitor0 console" comes up, which indicates to me that the VM started.

I verified the VM started by looking at used memory with "free -m" and could see that when I start the VM my memory jumps the amount assigned in the script. Not surprisingly, my memory use drops that amount when I quit the VM.

So the VM appears to have started but I don't have video. I managed to find the "info" command QEMU has, which indicates to me that the VM is being passed the video card:

(qemu) info pci (obviously cant list everything here, let me know if you need me to list more, but I cant figure out how to scroll up anyway)

Bus 0, device 3, function 0:
  VGA controller: PCI device 10de:1180
  IRQ 11.
  blah blah (let me know if the rest of this matter, dont feel like transposing)

So, to sum up what I am seeing here. Pre-reqs are met, vm starts and has the video card assiged, but I dont have any output from the card. I have tried hooking the monitor up to all 4 ports this card has (DVIx2,DP,HDMI) and do not get any output. I feel like based on what I've done I should be. What am I missing folks?

I think one thing I wasn't sure of is if the PCI bus itself:

00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
    Kernel driver in use: pcieport
    Kernel modules: shpchp

needs to be stubbed and assigned? I'm pretty sure I remember reading that you shouldn't do that,but I don't remember for sure. I did try assigning that as well on one of the 6 different attempts I've made at this so far and it didnt seem to help, but I cant imagine why else this wouldnt be working. Every guide I've read gives me the impression that I should have video output right now.

Thanks!

Last edited by knknif (2016-01-12 06:55:08)

jasonwryan · 2016-01-12 06:41:10

Please use code tags when pasting to the boards: https://wiki.archlinux.org/index.php/Fo … s_and_code

Not a Sysadmin issue, moving to NC...

tom.ty89 · 2016-01-12 06:43:39

You do NOT use the pci-stub module for -device vfio-pci, but the vfio-pci module. pci-stub is for -device (kvm-)pci-assign.

knknif · 2016-01-12 06:59:46

Sorry for the "code" thing, I thought it might be expected, but didnt realize it was a rule, nor did I know how. We'll chalk that one up to lazyness.

As for the pci-stub vs. vfio-pci thing - Are you saying that is wrong, or are you saying it isnt the ideal method? I set the driver to pci-stub on boot then run a "vfio-bind" command which binds them to vfio-pci.

Immediately after posting I realized I should take advantage of the fact that I have a spare video card laying around and try using that one. It worked.

So now the question becomes, why wouldn't the 680 work? I know that using certain emulation platforms the 680 needs to be hard-modded, but I was under the impression that with qemu the 680 should work without modification.

tom.ty89 · 2016-01-12 07:10:38

It's wrong, afaik. different methods require different kernel modules, end of story.

I have no idea what's a "vfio-bind" command. In any case if you bind it to vfio-pci after boot then that means you aren't using pci-stub anyway, so booting with it is just silly.

knknif · 2016-01-12 09:07:55

"vfio-bind" command was a short little script in the guide to allow easy binding of devices.

I'm probably going to switch back to manjaro now that I've got this "working" with the 750. Right now I'm unable to get it to boot to a drive (hard drive or cd iso) but I think I should be able to figure that out in a bit. Here is the VM script if you can notice anything wrong with it:

#!/bin/bash

cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
qemu-system-x86_64 \
  -enable-kvm \
  -m 2048 \
  -cpu host,kvm=off \
  -vga none \
  -usb -usbdevice host:1b1c:1b09 \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -device virtio-scsi-pci,id=scsi \
  -drive file=/windows/win7ult64.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/windows/windows1.img,id=disk,format=raw,if=none -device scsi-hd,drive=disk \
  -drive file=/windows/virt.iso,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd

Trying to keep it on the basic side until I get things off to a good start.

When i re-install, ill try skipping the pci-stub steps and see how the system handles it.

tom.ty89 · 2016-01-12 09:29:33

probably because you use ide for "virt.iso" and it got prioritised by ovmf. fwiw you know that you can press any key to get into ovmf's uefi setting / boot menu?

knknif · 2016-01-12 10:55:26

Well I know how to get into the settings but I wasn't aware that an immediate 'any key' would do the trick as well.

I have tried changing my start script to the following with no luck:

#more /usr/bin/windows

#!/bin/bash

cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd

qemu-system-x86_64 \
  -enable-kvm \
  -m 6969 \
  -smp cores=2,threads=2 \
  -cpu host,kvm=off \
  -vga none \
  -usb -usbdevice host:046d:c22d \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -device virtio-scsi-pci,id=scsi \
  -cdrom /windows/win7ult64.iso \
  -boot order=d \
  -drive file=/windows/windows1.img,format=raw

For some reason it keeps failing to boot to cd or the hdd image. I tried reading through some qemu/arch documentation but it seemed pretty complicated. Maybe im just too tired/high. Does this configuration make sense?

knknif · 2016-01-12 11:11:59

Re-reading your original statement about virtio being on ide, I dont think that has to do with it.

When I use this configuration:

#!/bin/bash

cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
qemu-system-x86_64 \
  -enable-kvm \
  -m 2048 \
  -cpu host,kvm=off \
  -vga none \
  -usb -usbdevice host:1b1c:1b09 \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -device virtio-scsi-pci,id=scsi \
  -drive file=/windows/win7ult64.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/windows/windows1.img,id=disk,format=raw,if=none -device scsi-hd,drive=disk \
  -drive file=/windows/virt.iso,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd

I can see during boot it tries booting to EFI SCSI as well as EFI SCSI 1, presumably my windows1.img as well as the windows 7 iso. It fails to boot to these two devices though.

Is this something that could be a permissions issue?

EDIT: I should mention that I have all of these relevant files on a separate HDD from my linux install. I mount /dev/sdb1 on /windows which is where I am storing these files.

Last edited by knknif (2016-01-12 11:16:06)

knknif · 2016-01-13 00:36:00

I hope this isnt considered bumping, I've since gotten much further and my problems have changed.

I can get the guest started, outputting video (after switching to 750ti from my gtx680), and booting to the windows installation image.

However, I need to install the virtio storage drivers in order to use the hard drive at better speeds. The guide states that I should be able to browse to the attached ISO of the virtio drivers during the windows install, but I am unable to do that. When selecting browse to choose the location of the drivers, nothing appears but the local installation image/windows cd.

Here is my qemu config:

#!/bin/bash

cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
qemu-system-x86_64 \
  -enable-kvm \
  -m 6970 \
  -cpu host,kvm=off \
  -vga none \
  -usb -usbdevice host:046d:c22d -usbdevice host:046d:c051 \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -device virtio-scsi-pci,id=scsi \
  -drive file=/home/jeff/win10.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/windows/windows8.img,id=disk,format=raw,if=none,cache=writeback -device scsi-hd,drive=disk \
  -drive file=/home/jeff/virt.iso,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd

tom.ty89 · 2016-01-13 11:54:35

Try to get rid of these TO TEST:

  -vga none \
  -usb -usbdevice host:046d:c22d -usbdevice host:046d:c051 \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \

I have experiences that passing through too many items could break things randomly.

Last edited by tom.ty89 (2016-01-13 17:22:19)

knknif · 2016-01-15 01:09:13

I was able to get things working (for the most part). I ended up booting the windows installer from a thumb drive and including the storage drivers in the boot dir. This allowed me to boot everything up and install windows properly, although TBH im not sure if I'm even utilizing the virtio storage drivers in my configuration.

Now I'm trying to figure out why the Windows VM is only detecting 1 core/thread. This is my current config if anyone could point out why I only get 1 CPU in windows.

#!/bin/bash

xrandr --output HDMI1 --off
cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
qemu-system-x86_64 \
  -enable-kvm \
  -m 6970 \
  -smp sockets=1,cores=2,threads=2 \
  -cpu host,kvm=off \
  -vga none \
  -usb -usbdevice host:046d:c22d -usbdevice host:046d:c051 \
  -device vfio-pci,host=01:00.0,multifunction=on \
  -device vfio-pci,host=01:00.1 \
  -drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
  -drive if=pflash,format=raw,file=/tmp/my_vars.fd \
  -device virtio-scsi-pci,id=scsi \
  -hda /dev/sdb

sl1pkn07 · 2016-01-15 18:44:50

try remove the all cpu cores form device manager in control panel and reboot the guest

Last edited by sl1pkn07 (2016-01-16 03:52:05)

knknif · 2016-01-15 23:55:47

That solution was way easier than I expected, gracias!

dominicm · 2016-03-21 14:17:03

I am having an almost identical issue (no video output). Care to share how you got past the original issue in your question?

Byter · 2016-04-09 17:07:43

dominicm wrote:

I am having an almost identical issue (no video output). Care to share how you got past the original issue in your question?

I'm in the same boat. Unfortunately he got through it by switching cards...

I can get the guest started, outputting video (after switching to 750ti from my gtx680), and booting to the windows installation image.

The only difference for me is I've got a 670 instead of a 680 (same card underneath though).

dominicm · 2016-04-09 17:39:59

I am trying to do the same now, with R9 Nano since radeon 7970 / 280x didn't work. Really annoying since plenty of people got 7970 to work and no one had same issues with it. What card are you using?

headkase · 2016-04-09 18:09:50

For new and unresolved people: you do need to check if your CPU supports vt-d and that it is enabled in the BIOS. If so then that is a point to put on your checklist that you report here, "what I have done." Without vt-d you are not going to be able to use a passthrough.

Last edited by headkase (2016-04-09 18:11:29)

dominicm · 2016-04-09 18:17:09

I don't know about others but I have exhausted every option and I certainly could not have gotten past even the very early attempts without vt-d. Also if using virt-manager you will get an error message if no vt-d support is present or IOMMU is not enabled.

headkase · 2016-04-09 18:24:24

For an Arch-specific guide, does this help: https://wiki.archlinux.org/index.php/PC … h_via_OVMF ?

dominicm · 2016-04-09 18:31:51

I used this guide and a bunch of others with no difference. I believe it's hardware related. Arch guide is also a lot more complicated that it needs to be, so I wouldn't recommend following it fully especially when trying to troubleshoot.

Last edited by dominicm (2016-04-09 18:35:48)

headkase · 2016-04-09 18:33:55

As an Arch user you should correct errors you are certain of in the Wiki: it's what you give back. Arch's Wiki is one of the best out there because people keep it in shape.

dominicm · 2016-04-09 18:42:56

I edited the post just before yours removing the errors part as I couldn't spot any looking back at the wiki, it is overcomplicated though and that's not something you can correct as some may need the complicated parts.

headkase · 2016-04-09 19:00:17

You can also improve it from a clarity perspective. Going over everything to improve it would also make the content more concrete in your own mind.

Sorry I have no further suggestions, never set up passthrough myself.

dominicm · 2016-09-04 18:26:38

I got it working in the end, in my case it was a configuration issue not hardware. I did a write up but posting blog links is against the rules.

Last edited by dominicm (2016-09-04 19:07:34)

Arch Linux

#1 2016-01-12 06:35:28

Attempting VGA Passthrough, no video output

#2 2016-01-12 06:41:10

Re: Attempting VGA Passthrough, no video output

#3 2016-01-12 06:43:39

Re: Attempting VGA Passthrough, no video output

#4 2016-01-12 06:59:46

Re: Attempting VGA Passthrough, no video output

#5 2016-01-12 07:10:38

Re: Attempting VGA Passthrough, no video output

#6 2016-01-12 09:07:55

Re: Attempting VGA Passthrough, no video output

#7 2016-01-12 09:29:33

Re: Attempting VGA Passthrough, no video output

#8 2016-01-12 10:55:26

Re: Attempting VGA Passthrough, no video output

#9 2016-01-12 11:11:59

Re: Attempting VGA Passthrough, no video output

#10 2016-01-13 00:36:00

Re: Attempting VGA Passthrough, no video output

#11 2016-01-13 11:54:35

Re: Attempting VGA Passthrough, no video output

#12 2016-01-15 01:09:13

Re: Attempting VGA Passthrough, no video output

#13 2016-01-15 18:44:50

Re: Attempting VGA Passthrough, no video output

#14 2016-01-15 23:55:47

Re: Attempting VGA Passthrough, no video output

#15 2016-03-21 14:17:03

Re: Attempting VGA Passthrough, no video output

#16 2016-04-09 17:07:43

Re: Attempting VGA Passthrough, no video output

#17 2016-04-09 17:39:59

Re: Attempting VGA Passthrough, no video output

#18 2016-04-09 18:09:50

Re: Attempting VGA Passthrough, no video output

#19 2016-04-09 18:17:09

Re: Attempting VGA Passthrough, no video output

#20 2016-04-09 18:24:24

Re: Attempting VGA Passthrough, no video output

#21 2016-04-09 18:31:51

Re: Attempting VGA Passthrough, no video output

#22 2016-04-09 18:33:55

Re: Attempting VGA Passthrough, no video output

#23 2016-04-09 18:42:56

Re: Attempting VGA Passthrough, no video output

#24 2016-04-09 19:00:17

Re: Attempting VGA Passthrough, no video output

#25 2016-09-04 18:26:38

Re: Attempting VGA Passthrough, no video output

Board footer