You are not logged in.

Pages: 1

#1 2017-04-24 15:45:08

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Problems with IOMMU and my graphics card

Hey guys,

I'm trying to passthrough my AMD graphics card (GPU1) to my virtual Windows machine, but this does not work properly with the current configuration. GPU1 is connected to the first PCIe slot because it's the fastest of all. The second graphics card is a Nvidia GTX 660 (GPU2) and it's connected to the second PCIe slot, which is much slower. So I want to use GPU2 for my host system.

The main problem is that I don't get any output of GPU1 if it's passthroughed to the virtual machine. If I swap those two GPUs, so that GPU2 is connected to the first PCIe slot and GPU1 is connected to the second one, everything  is working fine! I get no errors or warnings from Qemu and I also get a working display output from GPU1. But because of the lower bandwidth of the second PCIe slot I want to use the configuration written below.

Another strange thing: I have one monitor connected with GPU1 and another one with GPU2. For me it's like the system always tries to get the first PCIe slot as the primary graphics device. This is bad in combination with my configuration because I want to passthrough GPU1 and I am also disabling the AMD graphics driver by Grub kernel parameters so that GPU1 can be grabed by vfio. The result is that I have no display output while booting. I have to type blind to log in and to start X over startx. After that I get a working display output over GPU2 on my Linux host.

I'm doing my Windows installation like this:

  1. Installing Windows with windows-install script (see below)

  2. Installation done. Shutdown Windows.

  3. Restarting the virtual machine with windows-gpu script (see below)
    Qemu shows this warning:

    qemu-system-x86_64: -device vfio-pci,host=01:00.0,addr=0xa,multifunction=on,romfile=/home/thilo/virt/testwin/radeon.rom: Failed to mmap 0000:01:00.0 BAR 0. Performance may be slow

  4. So far, so good. Until now everything looks fine yet. I even get a second "Microsoft Basic Display Adapter" in the Windows devices manager.

  5. While installing the AMD graphics card driver I get a black screen on the display output of GPU1 and on the Qemu window.

  6. End

Thank you very much for your answers in advance!

Greetings,
avocadoom


My system configuration

  • Mainboard: Asus M5A97 R2.0 EVO
    -> IOMMU enabled in BIOS settings

  • CPU: AMD FX-8350

  • PCIe 2.0 x16 - GPU1 AMD Radeon R9 290
    -> Has UEFI enabled ROM

  • PCIe 2.0 x16 (x4 mode) - GPU2 Asus NVIDIA GTX 660

lspci

[root@pc ~]# lspci -nnk
00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge [1002:5a14] (rev 02)
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge [1002:5a14]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O Memory Management Unit (IOMMU) [1002:5a23]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] RD890S/RD990 I/O Memory Management Unit (IOMMU) [1002:5a23]
00:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GFX port 0) [1002:5a16]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 0) [1002:5a18]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 1) [1002:5a19]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 2) [1002:5a1a]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 3) [1002:5a1b]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] [1002:4391] (rev 40)
Subsystem: ASUSTeK Computer Inc. Device [1043:84dd]
Kernel driver in use: ahci
Kernel modules: ahci
00:12.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Kernel driver in use: ohci-pci
Kernel modules: ohci_pci
00:12.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Kernel driver in use: ehci-pci
Kernel modules: ehci_pci
00:13.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Kernel driver in use: ohci-pci
Kernel modules: ohci_pci
00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Kernel driver in use: ehci-pci
Kernel modules: ehci_pci
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller [1002:4385] (rev 42)
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller [1002:4385]
Kernel driver in use: piix4_smbus
Kernel modules: i2c_piix4, sp5100_tco
00:14.2 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) [1002:4383] (rev 40)
Subsystem: ASUSTeK Computer Inc. Device [1043:84fb]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d] (rev 40)
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d]
00:14.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge [1002:4384] (rev 40)
00:14.5 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller [1002:4399]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller [1002:4399]
Kernel driver in use: ohci-pci
Kernel modules: ohci_pci
00:15.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) [1002:43a0]
Kernel driver in use: pcieport
Kernel modules: shpchp
00:16.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
Kernel driver in use: ohci-pci
Kernel modules: ohci_pci
00:16.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
Kernel driver in use: ehci-pci
Kernel modules: ehci_pci
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0 [1022:1600]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1 [1022:1601]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2 [1022:1602]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3 [1022:1603]
Kernel driver in use: k10temp
Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4 [1022:1604]
Kernel driver in use: fam15h_power
Kernel modules: fam15h_power
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5 [1022:1605]
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii PRO [Radeon R9 290/390] [1002:67b1]
Subsystem: PC Partner Limited / Sapphire Technology Device [174b:e285]
Kernel driver in use: vfio-pci
Kernel modules: radeon, amdgpu
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]
Subsystem: PC Partner Limited / Sapphire Technology Device [174b:aac8]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 09)
Subsystem: ASUSTeK Computer Inc. P8 series motherboard [1043:8505]
Kernel driver in use: r8169
Kernel modules: r8169
03:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard [1043:8488]
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
Subsystem: ASUSTeK Computer Inc. Device [1043:84b7]
Kernel driver in use: ahci
Kernel modules: ahci
05:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
Subsystem: ASUSTeK Computer Inc. P8B WS Motherboard [1043:8488]
Kernel driver in use: xhci_hcd
Kernel modules: xhci_pci
06:07.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller [1106:3044] (rev c0)
Subsystem: ASUSTeK Computer Inc. Motherboard [1043:81fe]
Kernel driver in use: firewire_ohci
Kernel modules: firewire_ohci
07:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 [GeForce GTX 660] [10de:11c0] (rev a1)
Subsystem: ASUSTeK Computer Inc. Device [1043:8421]
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia
07:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio Controller [10de:0e0b] (rev a1)
Subsystem: ASUSTeK Computer Inc. Device [1043:8421]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel

IOMMU groups

IOMMU Group 0 00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge [1002:5a14] (rev 02)
IOMMU Group 10 00:14.2 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) [1002:4383] (rev 40)
IOMMU Group 11 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d] (rev 40)
IOMMU Group 12 00:14.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge [1002:4384] (rev 40)
IOMMU Group 12 06:07.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller [1106:3044] (rev c0)
IOMMU Group 13 00:14.5 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller [1002:4399]
IOMMU Group 14 00:15.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) [1002:43a0]
IOMMU Group 14 07:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK106 [GeForce GTX 660] [10de:11c0] (rev a1)
IOMMU Group 14 07:00.1 Audio device [0403]: NVIDIA Corporation GK106 HDMI Audio Controller [10de:0e0b] (rev a1)
IOMMU Group 15 00:16.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 15 00:16.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
IOMMU Group 16 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii PRO [Radeon R9 290/390] [1002:67b1]
IOMMU Group 16 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Hawaii HDMI Audio [Radeon R9 290/290X / 390/390X] [1002:aac8]
IOMMU Group 17 02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 09)
IOMMU Group 18 03:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU Group 19 04:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU Group 1 00:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GFX port 0) [1002:5a16]
IOMMU Group 20 05:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
IOMMU Group 2 00:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 0) [1002:5a18]
IOMMU Group 3 00:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 1) [1002:5a19]
IOMMU Group 4 00:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 2) [1002:5a1a]
IOMMU Group 5 00:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 3) [1002:5a1b]
IOMMU Group 6 00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] [1002:4391] (rev 40)
IOMMU Group 7 00:12.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 7 00:12.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
IOMMU Group 8 00:13.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 8 00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
IOMMU Group 9 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller [1002:4385] (rev 42)

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="modprobe.blacklist=amdgpu,radeon iommu=pt amd_iommu=on cryptdevice=UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx:mypc:allow-discards"

/etc/mkinitcpio.conf

MODULES="tun kvm kvm-amd vfio vfio_iommu_type1 vfio-pci vfio_virqfd"

/etc/modprobe.d/blacklist.conf

blacklist amdgpu
blacklist radeon

/etc/modprobe.d/vfio.conf

options vfio-pci ids=1002:67b1,1002:aac8

windows-install script

echo "Starting virtual machine"
export QEMU_AUDIO_DRV="pa"
qemu-system-x86_64 \
	-serial none \
	-parallel none \
	-nodefaults \
	-nodefconfig \
	-no-user-config \
	-monitor unix:/run/windows-gaming.sock,server,nowait \
	-daemonize \
	-k de \
	-enable-kvm \
	-name gaming \
	-machine pc,accel=kvm,kernel_irqchip=on,mem-merge=off \
	-cpu host,kvm=off,hv_vapic,hv_time,hv_relaxed,hv_spinlocks=0x1fff,hv_vendor_id=sugoidesu \
	-smp sockets=1,cores=4,threads=1 \
	-m 4096 \
	-mem-prealloc \
	-soundhw hda \
	-device ich9-usb-uhci3,id=uhci \
	-device usb-ehci,id=ehci \
	-device nec-usb-xhci,id=xhci \
	-device virtio-scsi-pci,id=scsi \
	-machine pc,accel=kvm,kernel_irqchip=on,mem-merge=off \
	-drive if=pflash,format=raw,readonly,file=/home/thilo/virt/gaming/ovmf_code.fd \
	-drive if=pflash,format=raw,file=/home/thilo/virt/gaming/ovmf_vars.fd \
	-rtc base=localtime,clock=host,driftfix=none \
	-device scsi-hd,drive=hd0 \
	-drive if=none,id=hd0,file=/dev/mapper/thilopc-disk2,format=raw,cache=none,aio=native \
	-boot order=d \
	-vga std \
	-display sdl \
	-usbdevice tablet \
	-drive file=/home/thilo/virt/Win10_1703_German_x64.iso,if=ide,id=cd0,media=cdrom,readonly \
	-drive file=/home/thilo/virt/virtio-win.iso,if=ide,id=cd1,media=cdrom,readonly

windows-gpu script

echo "Unbinding devices from host"
for dev in "0000:01:00.0" "0000:01:00.1"; do
	vendor=$(cat /sys/bus/pci/devices/${dev}/vendor)
	device=$(cat /sys/bus/pci/devices/${dev}/device)
	if [ -e /sys/bus/pci/devices/${dev}/driver ]; then
		echo "${dev}" | sudo /usr/bin/tee /sys/bus/pci/devices/${dev}/driver/unbind > /dev/null
		while [ -e /sys/bus/pci/devices/${dev}/driver ]; do
			sleep 0.1
		done
	fi
	echo "${vendor} ${device}" | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id > /dev/null
done

echo "Starting virtual machine"
QEMU_ALSA_DAC_BUFFER_SIZE=512 QEMU_ALSA_DAC_PERIOD_SIZE=170 QEMU_AUDIO_DRV=alsa qemu-system-x86_64 \
	-boot order=d \
	-serial none \
	-parallel none \
	-nodefaults \
	-nodefconfig \
	-enable-kvm \
	-daemonize \
	-name gaming \
	-rtc base=localtime,clock=host,driftfix=none \
	-machine pc,accel=kvm,kernel_irqchip=on,mem-merge=off \
	-cpu host,kvm=off,hv_vapic,hv_time,hv_relaxed,hv_spinlocks=0x1fff,hv_vendor_id=sugoidesu \
	-smp sockets=1,cores=6,threads=1 \
	-m 6144 \
	-mem-prealloc \
	-soundhw ac97 \
	-monitor unix:/run/windows-gaming.sock,server,nowait \
	-device virtio-balloon-pci,id=balloon0,bus=pci.0 \
	-device ich9-usb-uhci3,id=uhci \
	-device virtio-scsi-pci,id=scsi \
	-device scsi-hd,drive=hd0 \
	-drive if=none,id=hd0,file=/dev/mapper/thilopc-disk2,format=raw,cache=none,aio=native \
	-drive file=/dev/sda1,cache=none,aio=native,format=raw,if=virtio \
	-drive if=pflash,format=raw,readonly,file=/home/thilo/virt/gaming/ovmf_code.fd \
	-drive if=pflash,format=raw,file=/home/thilo/virt/gaming/ovmf_vars.fd \
	-net nic,vlan=0,macaddr=52:54:00:00:00:01,model=e1000,name=net0 \
	-net bridge,vlan=0,name=bridge0,br=br0 \
	-device vfio-pci,host=01:00.0,addr=0xa,multifunction=on,romfile=/home/thilo/virt/windows/radeon.rom \
	-device vfio-pci,host=01:00.1,addr=0xb \
	-nographic \
	-usbdevice host:28de:1142 \
	-usbdevice host:1e7d:2e22 \
	-usbdevice host:04b4:0101

Offline

#2 2017-04-25 20:37:00

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,911

Re: Problems with IOMMU and my graphics card

For me it's like the system always tries to get the first PCIe slot as the primary graphics device

That's exactly what happens, and it's arranged that way on hardware level.

Your "problems" are not due to software or OS limitations, but to hardware limitations.


Things may not be as bad as they seem though.
Most videocards don't need as much bandwidth on PCIe bus as the manufacturer claims.

Try running graphics heavy stuff on your fastest card in the x16 slot, then repeat with the fast card in the x16 slot with 4 lanes.
I think you'll be surprised by the differences between the numbers.

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#3 2017-04-25 20:54:23

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Re: Problems with IOMMU and my graphics card

Lone_Wolf wrote:

Most videocards don't need as much bandwidth on PCIe bus as the manufacturer claims.

Try running graphics heavy stuff on your fastest card in the x16 slot, then repeat with the fast card in the x16 slot with 4 lanes.
I think you'll be surprised by the differences between the numbers.

Yes, I already tried to use the Radeon card in the second slot, but that caused big performance issue and a bunch of lags. The graphics power was much weaker than the x16 slot with 16 lanes and with Windows installed as host system.. That's why I'm trying to use my Radeon GPU in the x16 slot.

And the graphics heavy stuff has to run on my Windows VM (gaming and video editing)

Last edited by avocadoom (2017-04-25 21:05:00)

Offline

#4 2017-04-25 21:18:53

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: Problems with IOMMU and my graphics card

Does your bios/fw have any option to set the primary card? Maybe that could help if there is such an option.

R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#5 2017-04-25 21:28:09

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Re: Problems with IOMMU and my graphics card

already searched for it, I can only switch between PCI / PEG, and that does not help in any way neither have I a PCI graphics card

Here is my mainboards manual for reference: http://dlcdnet.asus.com/pub/ASUS/mb/Soc … V2_WEB.pdf

Last edited by avocadoom (2017-04-25 21:30:11)

Offline

#6 2017-04-25 22:04:54

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: Problems with IOMMU and my graphics card

By hardware design, the first PCIe slot will basically always be the boot VGA device (ignoring iGPU systems).

If you set your system to boot in pure EFI mode (assuming it has an UEFI firmware, disable any form of CSM) you are most likely able to forward your primary GPU without any problem. CSM (or oldschool BIOS) prevent you from reassigning the primary/boot VGA device.
Aside from your motherboard, this also requires the GPU to be UEFI-Ready (or however you want to call it) otherwise your system will not boot; it can't initialize the GPU (note: the system may automatically re-enable CSM to allow booting again). The nVidia card should be fine on that part, I don't know about the AMD one. You'll have to check that.

Last edited by Omar007 (2017-04-25 22:10:05)

Offline

#7 2017-04-25 23:42:48

wulfspider
Member
From: Germany, Hesse, Darmstadt
Registered: 2015-05-05
Posts: 13

Re: Problems with IOMMU and my graphics card

I just want to join the discussion smile

Found this, that's what Omar007 said... https://wiki.archlinux.org/index.php/PC … _the_guest

Omar007 wrote:

If you set your system to boot in pure EFI mode (assuming it has an UEFI firmware, disable any form of CSM) you are most likely able to forward your primary GPU without any problem. [...] Aside from your motherboard, this also requires the GPU to be UEFI-Ready (or however you want to call it) otherwise your system will not boot; it can't initialize the GPU [...]

So it's not enough to boot in EFI mode but to disable CSM completely? What if the Nvidia graphics card isn't UEFI-Ready? Will it work for the host system though?

avocadoom wrote:

PCIe 2.0 x16 - GPU1 AMD Radeon R9 290
-> Has UEFI enabled ROM

How did you check that?

Last edited by wulfspider (2017-04-25 23:51:45)

Offline

#8 2017-04-26 13:29:49

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Re: Problems with IOMMU and my graphics card

I've checked that with GPU-Z and also flashed a UEFI compatible BIOS onto my R9 290.

Offline

#9 2017-05-01 21:44:09

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Re: Problems with IOMMU and my graphics card

Both of my cards support UEFI now, but if both are connected to the motherboard I get no display output and if I disable CSM it happens the same. Only if one card is connected I get an output.

Last edited by avocadoom (2017-05-02 21:34:55)

Offline

#10 2017-05-02 21:35:15

avocadoom
Member
Registered: 2017-04-24
Posts: 12

Re: Problems with IOMMU and my graphics card

any ideas?

Offline

#11 2018-12-25 12:32:42

paigethylamine
Member
Registered: 2018-12-25
Posts: 1

Re: Problems with IOMMU and my graphics card

For what this is worth to you, I managed to get my R9 390 working with the q35 configuration by adding pcie-switch-downstream-port and pcie-switch-upstream-ports as well as a pcie bridge to the configuration. Before the AMD drivers would always crash when it was connected directly to the root complex for me. Here's a copy of my domxml: https://gitlab.com/snippets/1788426#note_127191397

and a rough translation to the command line parameters for qemu here: https://gitlab.com/snippets/1788426#note_127194064

I read this is how it should be anyway, connecting the video card directly to the pci/e root complex isn't the correct topology layout. So it may explain the reason for why the drivers didn't work, still haven't been able to confirm for sure--I just know that it works now and didn't before.

Last edited by paigethylamine (2018-12-25 12:33:43)

Offline

Pages: 1

Board footer

Powered by FluxBB