Arch Linux

r0b0t

Member

From: /tmp

Registered: 2009-05-24

Posts: 505

[SOLVED]Unable to bind vfio-pci to RX 590 Radeon

Hi,

I'm trying to "isolate the GPU" https://wiki.archlinux.org/index.php/PC … ng_the_GPU , and so far I don't understand why it doesn't work...

The integrated chip is an UHD Graphics 630 (Desktop)

So far : IOMMU supported:

 cat /etc/default/grub | grep iomm
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"

$ dmesg | grep -e DMAR -e IOMMU
[    0.006884] ACPI: DMAR 0x0000000039542AF0 0000A8 (v01 ALASKA A M I    00000002      01000013)
[    0.173079] DMAR: IOMMU enabled
[    0.235277] DMAR: Host address width 39
[    0.235277] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.235280] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.235281] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.235282] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.235283] DMAR: RMRR base: 0x000000399d4000 end: 0x00000039c1dfff
[    0.235283] DMAR: RMRR base: 0x0000003b000000 end: 0x0000003f7fffff
[    0.235284] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.235285] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.235285] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.236759] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    1.573839] DMAR: No ATSR found
[    1.573871] DMAR: dmar0: Using Queued invalidation
[    1.573874] DMAR: dmar1: Using Queued invalidation
[    1.574104] DMAR: Setting RMRR:
[    1.574157] DMAR: Setting identity map for device 0000:00:02.0 [0x3b000000 - 0x3f7fffff]
[    1.574195] DMAR: Setting identity map for device 0000:00:14.0 [0x399d4000 - 0x39c1dfff]
[    1.574200] DMAR: Prepare 0-16MiB unity mapping for LPC
[    1.574229] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    1.574234] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    3.566892] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    3.566893] AMD-Vi: AMD IOMMUv2 functionality not available on this system

Currently detected as :

IOMMU Group 1 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e1)
IOMMU Group 1 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]

Binding set correctly:

 cat /etc/modprobe.d/vfio.conf 
options vfio-pci ids=1002:67df,1002:aaf0

and :

egrep -i 'modules|hooks' /etc/mkinitcpio.conf | grep -v "#"
MODULES=(nvme vfio_pci vfio vfio_iommu_type1 vfio_virqfd)
HOOKS=(base udev autodetect modconf block filesystems keyboard fsck)

Of course done as well the  mkinitcpio -g /boot/linux-custom.img and rebooted different times already...
I also tried to put amdgpu after vfio_virgfd, however didn't do the trick, the card is still owned by amdgpu...

01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e1)
	Subsystem: Sapphire Technology Limited Nitro+ Radeon RX 570/580 [1da2:e366]
	Kernel driver in use: amdgpu
	Kernel modules: amdgpu
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
	Subsystem: Sapphire Technology Limited Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1da2:aaf0]
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

Note, atm arch fully up-to-date:

uname -r
5.0.8-arch1-1-ARCH

======================
SOLUTION
======================

cat /etc/modprobe.d/amdgpu.conf 
softdep amdgpu pre: vfio vfio_pci
softdep snd_hda_intel pre: vfio vfio_pci

Last edited by r0b0t (2019-04-22 16:36:42)

r0b0t

Member

From: /tmp

Registered: 2009-05-24

Posts: 505

Re: [SOLVED]Unable to bind vfio-pci to RX 590 Radeon

anyone?

Lone_Wolf

Member

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 11,911

Re: [SOLVED]Unable to bind vfio-pci to RX 590 Radeon

Which processor ?

What card is set as primary videocard in uefi firmware ?

remove quiet from grub commandline, try adding the vfio options as kernel paramaters directly.
Another thing to try is blacklisting admgpu

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

r0b0t

Member

From: /tmp

Registered: 2009-05-24

Posts: 505

Re: [SOLVED]Unable to bind vfio-pci to RX 590 Radeon

Which processor ?

What card is set as primary videocard in uefi firmware ?

remove quiet from grub commandline, try adding the vfio options as kernel paramaters directly.
Another thing to try is blacklisting admgpu

- i7 8700
- When I installed arch the intel UHD card was disabled (on bios / gigabyte), after the installation intel's gpu wasn't showing at all, but I enabled it on bios and I'm able to use it now.
How can I verify which one is currently set as primary in uefi firmware?
EDIT: the firmware thingy throw me a bit off, if you are referring to the bios settings , it's set to IGFX so intel is set as primary, by default it was set into ATI / PCIe slot, but if we want to isolate it for windows I guess it's good the way it is, having intel as primary.

- I'll try setting intel as primary into the firmware before trying to force stuff via the bootline, maybe it will work that way.

Last edited by r0b0t (2019-04-22 13:12:27)

Lone_Wolf

Member

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 11,911

Re: [SOLVED]Unable to bind vfio-pci to RX 590 Radeon

Bios has been in use from the 1980's until approx 2013 , but 99.9% of new pcs / laptops use uefi firmware instead of bios.
I know people often use BIOS when they should use uefi .

since you want to passthrough the amd card setting the intel one as primary for host use is the logical setup.

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

r0b0t

Member

From: /tmp

Registered: 2009-05-24

Posts: 505

Re: [SOLVED]Unable to bind vfio-pci to RX 590 Radeon

Yeah,, this is a Gygabyte z390 pro, its pretty new but I still refer to the uefi firmware as bios... although it's uefi firmware...
The kernel parameters didn't help either, I did this tho:

cat /etc/modprobe.d/amdgpu.conf 
softdep amdgpu pre: vfio vfio_pci

which it seems to do the trick on the GPU:

01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e1)
	Subsystem: Sapphire Technology Limited Nitro+ Radeon RX 570/580 [1da2:e366]
	Kernel driver in use: vfio-pci
	Kernel modules: amdgpu

But not the sound card which is still sticking with snd_hda_intel, will try the same trick for it as well.