[resolved] signal from monitor w/ qemu, two GPUs, one monitor

Sara · 2016-06-23 18:19:18

I'm trying to get the second HDMI input detected on a single LG 4K UHD 27UD68-P monitor so I can do GPU passthrough with the second card. After a lot of headscratching post setting the VM I realized my issue was that the second input did not appeared to be recognized in Linux. I have two Nvidia GeForce 1080s:

lspci -nnk
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] 
(rev a1)
        Subsystem: eVga.com. Corp. Device [3842:6180]
        Kernel driver in use: nvidia
        Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)
        Subsystem: eVga.com. Corp. Device [3842:6180]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] 
(rev a1)
        Subsystem: NVIDIA Corporation Device [10de:119e]
        Kernel driver in use: vfio-pci
        Kernel modules: nouveau, nvidia_drm, nvidia
02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)
        Subsystem: NVIDIA Corporation Device [10de:119e]
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel

and I have them plugged into both hdmi slots on my monitor (it also supports a single displayport). I am running the linux-vfio kernel, and have already bound the second card to the vfio-pci driver (for GPU passthrough using this guide). So I tried adding the following lines (to my autogenerated xorg.conf from nvidia-xconfig) to my xorg.conf:

Section "ServerLayout"
  Identifier "Layout0"
  Screen 0 "Screen0"
  Screen 1 "Screen1"
  ...
EndSection

Section "Device"
  Identifier "Device1"
  Driver "vfio-pci"
  BusID "PCI:2:0:0"
EndSection

Section "Screen"
  Identifier "Screen1"
  Device "Device1"
  Monitor "Monitor0"
  DefaultDepth 24
  SubSection "Display"
    Depth 24
  EndSubSection
EndSection

But upon rebooting I still don't see input on the second monitor when I tested running an efi shell with qemu. Also, the second HDMI does not appear in xrandr:

Screen 0: minimum 8 x 8, current 3840 x 2160, maximum 32767 x 32767
DVI-D-0 disconnected (normal left inverted right x axis y axis)
HDMI-0 connected primary 3840x2160+0+0 (normal left inverted right x axis y axis) 600mm x 340mm
   3840x2160     30.00*+  29.97    25.00    23.98  
   1920x1080     60.00    59.94    29.97    60.00  
   1600x900      60.00  
   1280x1024     60.02  
   1280x800      59.81  
   1280x720      60.00    59.94  
   1152x864      60.00  
   1024x768      60.00  
   800x600       60.32  
   720x480       59.94  
   640x480       59.94    59.93  
DP-0 disconnected (normal left inverted right x axis y axis)
DP-1 disconnected (normal left inverted right x axis y axis)
DP-2 disconnected (normal left inverted right x axis y axis)
DP-3 disconnected (normal left inverted right x axis y axis)
DP-4 disconnected (normal left inverted right x axis y axis)
DP-5 disconnected (normal left inverted right x axis y axis)

HDMI-0 is my currently connected display. I tried switching the inputs, connecting my 1080 located at 1:0:0 to the second input and the one located at 2:0:0 to the first, with no difference in xrandr (it still calls it HDMI-0) and the Linux input is still detected while my EFI shell test is not.

This is the script (running as root) for my EFI shell test by the way (with ovmf-git installed):

#!/bin/bash

cp /usr/share/ovmf/x64/ovmf_vars_x64.bin /tmp/my_vars.fd
qemu-system-x86_64 \
-enable-kvm \
-m 2048 \
-cpu host,kvm=off \
-vga none \
-device vfio-pci,host=02:00.0,multifunction=on \
-device vfio-pci,host=02:00.1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/ovmf_code_x64.bin \
-drive if=pflash,format=raw,file=/tmp/my_vars.fd

Also

(qemu) info pci

does show the card in qemu (which opens as a pop-up window instead of in the second input):

Bus 0, device 3, function 0:
  VGA controller: PCI device 10de:1b80
    IRQ 11.
    BAR0: 32 bit memory at 0x80000000 [0x80ffffff]
    BAR1: 64 bit prefetchable memory at 0x800000000 [0x80fffffff].
    BAR3: 64 bit prefetchable memory at 0x810000000 [0x811ffffff].
    BAR5: I/O at 0xc000 [0xc07f].
    BAR6: 32 bit memory at 0xffffffffffffffff [0x0007fffe].
    id ""
Bar 0, device 4, function 0:
  Audio controller: PCI device 10de:10f0
    IRQ 10.
    BAR0: 32 bit memory at 0x81020000 [0x81023fff].
    id ""

I see this error in the terminal after it opens but I thought it only related to my gtk theme (maybe it's to blame?):

(qemu-system-x86_64:5131): Gtk-WARNING **: Theme parsing error: gtk-contained.css:1700:22: Not a valid image

I unfortunately don't have my VM with to fully test it as I deleted it after the second input repeatedly failed to load thinking I'd done something wrong, but then realized I needed to add my second card to my xorg.conf (which I've done), and didn't want to go about setting up the VM again until I've entirely cleared away whatever other issues there may have been. Unfortunately, I don't know what that entails as most guides I find with multiple gpus are for multiple monitors, and I have only a single monitor whose second input does not appear detected. I think I might still be missing something elementary in my xorg.conf, but if it's a more deeper issue I would appreciate a pointer. Thanks in advance.

If it also helps to know, I am running version 4.6.2-1 of linux-vfio, 64-bit, and my motherboard is an MSI Z170-A XPOWER TITANIUM, and my CPU is an Intel Skylake 6600k. First PC I've ever built and it was pretty much for the purpose of GPU passthrough. XD Using a second monitor isn't really an option as I don't have the space. (Plus, 4K is still plenty expensive.) Thanks again!

Edit: Thanks to helpful posters, I realize it had nothing to do with my xorg, as the host should not be seeing the input connected to the guest GPU. However, my issue still remains post removing the lines mentioned in my xorg.conf.

Edit Final: So I plugged the HDMI input into a second monitor instead, and immediately it WORKED. Got to the EFI shell with the test script. Really unfortunate that the second input on my main monitor failed to work properly, as nvidia-settings clearly showed; I feel it has to be a Linux-centric issue, as the same monitor did detect when an HDMI cable was plugged in, it simply never got the signal from the other graphics card. (Nothing wrong with the second HDMI port in and of itself, as I swapped the first graphics card's HDMI into the second port and vice versa, and the first graphics card continued to power the Linux display without a hitch.) I could do further testing of the second port to try to figure it out, but for now I'm happy; qemu will be more handy with dual monitors anyhow. (Playing in one monitor, chatting in another.) So psyched now about getting the rest of GPU passthrough working, thanks again to everyone!

Last edited by Sara (2016-06-24 07:36:37)

Lone_Wolf · 2016-06-23 18:35:14

1 hdmi-input gets its signal from the passed through videocard if i understand your setup correctly?

I'd be very surprised to see that 2nd device detected in your host environment .

You did switch the 4k screen to display the input from the 2nd hdmi connection after starting the VM ?

Slithery · 2016-06-23 18:37:45

Why do you expect Arch to see your 2nd monitor?

Surely the whole point of VGA passthrough is that your host OS (and therefore xorg) is no longer aware of the 2nd graphics card, it can only be used by the guest OS.
What happens if you remove everything you added to your xorg.conf and try the shell test again

Sara · 2016-06-23 18:38:46

Lone_Wolf wrote:

1 hdmi-input gets its signal from the passed through videocard if i understand your setup correctly?
I'd be very surprised to see that 2nd device detected in your host environment .
You did switch the 4k screen to display the input from the 2nd hdmi connection after starting the VM ?

Yup, I switched to the second input via my monitor's toggle, but it said "No signal detected" and switched off thanks to its aggressive power savings. So the HDMI-0 gets its input from the non-passed through video card and the second HDMI gets it through the passed through videocard, yes.

Ah, so it's normal that the second input doesn't show in the host environment? That's good to know.

Sara · 2016-06-23 18:44:44

slithery wrote:

Why do you expect Arch to see your 2nd monitor?
Surely the whole point of VGA passthrough is that your host OS (and therefore xorg) is no longer aware of the 2nd graphics card, it can only be used by the guest OS.
What happens if you remove everything you added to your xorg.conf and try the shell test again

Ahh, that makes sense, it wouldn't be right for the host to see it if it's supposed to be exclusive to the guest OS. XD Is it strange though that the xorg.conf that nvidia generated did not include my second card before binding it to vfio-pci (when I had simply just installed Arch to this PC), or was that to be expected? That's why I thought I'd missed something there.

I get it now that I should remove the device from my xorg.conf, which I'll do straightaway. However, I was having this issue before trying that, so something else must be to blame.

Edit: Edited my xorg.conf and rebooted, and I still don't get a signal from my second input when I switch. I changed the GTK to Adwaita (the gnome default) and the error that was appearing before in the terminal did not appear, so that is in fact unrelated. I would like to try setting up the VM again, but if it's failing at like the first test that doesn't seem like a good sign... is there anything else I might be missing? I tried using the script as is from TurluCode and manually downloading the rpm file and extracting it as the guide mentioned (in my script, I renamed the relevant files as the Arch AUR package does), but no dice.

If it also helps to know, here is the relevant iommu grouping:

IOMMU group 12
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] (rev a1)
	01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)
IOMMU group 13
	02:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1b80] (rev a1)
	02:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f0] (rev a1)

so the two cards are in different groups (I had to use the linux-vfio kernel with the pcie_acs_override=downstream kernel option to get it that way, however). Also I don't think the second hdmi cable is faulty (it's also brand new) as the monitor detects I've added it when I remove and reinsert.

I've tried passing through a USB keyboard (I use a different, bluetooth keyboard on the host) to see if the issue was a input device not being detected; it is correctly passed through and no longer usable on the host, but when I switch inputs, hitting the keyboard doesn't trigger a signal.

(qemu) info usb
  Device 0.1, Port 1, Speed 1.5 Mb/s, Product USB Keyboard

Edit 2: So, to do further testing, I switched to the mainline linux kernel which does not have those vfio settings and iommu grouping enabled. I re-added the lines to my xorg.conf so my second card could be recognized, and opened nvidia-settings. It shows that my second card (GPU - (Geforce GTX 1080)) is running and operational, but not connected to any display. (Whereas my other one is DFP-1 - (LG Electronics LG Ultra HD)). So my issue is in fact the monitor not reading the second input. I'm going to try swapping cables to see if that yields a fix, but it's good to know it's (probably) not at all related to my qemu settings.

Edit Final: So I managed to wiggle a second monitor on my tiny, tiny desk, and immediately the nvidia-settings application detected it, as DFP-1 - (Ancor COmmunications Inc). I'm pretty sure this will resolve my issue now. Going to test again with qemu with the linux-vfio kernel to be absolutely sure, would be so happy if it works!

Last edited by Sara (2016-06-24 02:06:16)

Arch Linux

#1 2016-06-23 18:19:18

[resolved] signal from monitor w/ qemu, two GPUs, one monitor

#2 2016-06-23 18:35:14

Re: [resolved] signal from monitor w/ qemu, two GPUs, one monitor

#3 2016-06-23 18:37:45

Re: [resolved] signal from monitor w/ qemu, two GPUs, one monitor

#4 2016-06-23 18:38:46

Re: [resolved] signal from monitor w/ qemu, two GPUs, one monitor

#5 2016-06-23 18:44:44

Re: [resolved] signal from monitor w/ qemu, two GPUs, one monitor

Board footer