Using two AMD GPUs simultaneously

Zizilla · 2019-12-07 16:01:46

Hi All,

I'm new to the forum, since I could solve each issues I had with the help of our great wiki.

In my current desktop machine for work I have two AMD GPUs:
- AMD Raven Ridge 2400G IGP
- AMD RX580 dGPU

Current behaviour (without any tweak):
- IGP works without 3D acceleration (even the browser is slow as hell)
- dGPU works fine in all regard

Desired behaviour:
- IGP should work with 3D acceleration
- dGPU should be able to be turned off for power saving, and turned on for OpenCL render (Blender) and for GPU passthrough (virtual windows gaming).

I have tried the following:
- /usr/share/acpi_call/examples/turn_off_gpu.sh
Result: all lines ends with "failed".
- lshw: https://pastebin.com/wN7RYwF9
- $ glxinfo | grep "Device"
Device: Radeon RX 580 Series (POLARIS10, DRM 3.33.0, 5.3.8-arch1-1, LLVM 9.0.0) (0x67df)
- DRI_PRIME=0 and 1 glxinfo | grep OpenGL: https://pastebin.com/XF3V4dGb
- BIOS settings: init display first. IGP/Auto/PCI-E - all the same in linux
- native Windows 10: I could switch between the IGP and dGPU for games, both worked correctly

Could you please help me what I did wrong, and what can I do for the desired behaviour?

Thank you in advance!
BR,
Zizilla

Last edited by Zizilla (2019-12-07 16:03:08)

Lone_Wolf · 2019-12-08 11:55:43

acpi-call hasn't seen new upstream commits since 2013, your cpu/igp/gpu are way to new for it.

You have read the https://wiki.archlinux.org/index.php/PRIME page ?

- dGPU should be able to be turned off for power saving, and turned on for OpenCL render (Blender) and for GPU passthrough (virtual windows gaming).

GPU passthrough usually needs to be setup at boot time, turning on/off the dgpu without rebooting will be tricky (if possible, which I doubt ).

The glxinfo outputs don't show errrors.

For now I suggest you set the igp as init display first in firmware.

Post full dmesg and/or journalctl -b output, as well as xorg log .
lspci -k & xrandr --listproviders outputs could also be helpful .

Last edited by Lone_Wolf (2019-12-08 11:56:43)

Zizilla · 2019-12-09 20:40:21

Hi Lone_Wolf,

Thanks for the help in advance!
Now, I'm reading the linket wiki page.

Until then:

dmesg: https://pastebin.com/WZAbPg8s

lspci -k: https://pastebin.com/4ckJTVNE

xrandr --listproviders: https://pastebin.com/4r7nYWmV

Thanks!

Last edited by Zizilla (2019-12-09 20:40:35)

progandy · 2019-12-09 21:43:36

Lone_Wolf wrote:

GPU passthrough usually needs to be setup at boot time, turning on/off the dgpu without rebooting will be tricky (if possible, which I doubt ).

In theory you can bind/unbind pci devices from any driver and add them to the vfio-pci driver with command files in the sysfs tree. The GPU might not reset into a defined state without a power cycle, but I have no experience with this.

https://www.kernel.org/doc/Documentation/vfio.txt
https://old.reddit.com/r/VFIO/comments/ … from_host/

Last edited by progandy (2019-12-09 21:48:33)

Lone_Wolf · 2019-12-10 13:00:43

Thx, progandy.

Apart from a segfault that worries me (see further below) there are several things in the log that stand out to me.

kvm: disabled by bios

If you want to use a VM this should definitely be enabled

[    5.795545] sp5100_tco: SP5100/SB800 TCO WatchDog Timer Driver

[    5.797898] sp5100-tco sp5100-tco: Using 0xfeb00000 for watchdog MMIO address
[    5.801171] sp5100-tco sp5100-tco: initialized. heartbeat=60 sec (nowayout=0)

That Watchdog hardware is typically only useful/present on server boards, think amd EPYC systems. (My threadripper board also doesn't have it).
Best to disable it in firmware.

[    0.618847] pci 0000:00:00.2: AMD-Vi: Unable to write to IOMMU perf counter.

[    0.618969] pci 0000:00:00.2: can't derive routing for PCI INT A
[    0.618970] pci 0000:00:00.2: PCI INT A: not connected

Not sure if they are related, but they shouldn't be there at all.

Hardware name: Gigabyte Technology Co., Ltd. A320M-S2H/A320M-S2H-CF, BIOS F50 11/28/2019

Can you verify whether you have the A320M-S2H or A320M-S2H-CF motherboard ?

Are you booting into uefi ?
Is CSM disabled in firmware ?

Not sure what this means, but a segfault related to amdgpu is never a good sign.

[    6.997199] WARNING: CPU: 5 PID: 397 at drivers/gpu/drm/amd/amdgpu/../display/dc/calcs/dcn_calcs.c:1464 dcn_bw_update_from_pplib+0xaa/0x2f0 [amdgpu]
[    6.997200] Modules linked in: cfg80211 rfkill 8021q garp mrp stp llc amdgpu(+) edac_mce_amd nls_iso8859_1 ccp nls_cp437 rng_core vfat fat kvm snd_hda_codec_hdmi snd_hda_intel gpu_sched snd_intel_nhlt i2c_algo_bit irqbypass snd_hda_codec ttm crct10dif_pclmul crc32_pclmul snd_hda_core r8169 drm_kms_helper ghash_clmulni_intel wmi_bmof realtek snd_hwdep aesni_intel snd_pcm crypto_simd cryptd glue_helper libphy pcspkr snd_timer drm agpgart sp5100_tco snd uas syscopyarea input_leds joydev sysfillrect mousedev sysimgblt fb_sys_fops soundcore i2c_piix4 k10temp wmi pinctrl_amd evdev gpio_amdpt mac_hid acpi_cpufreq acpi_call(OE) ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 sd_mod usb_storage hid_logitech ff_memless hid_generic usbhid hid crc32c_intel ahci libahci xhci_pci libata xhci_hcd scsi_mod
[    6.997241] CPU: 5 PID: 397 Comm: systemd-udevd Tainted: G           OE     5.4.2-arch1-1 #1
[    6.997243] Hardware name: Gigabyte Technology Co., Ltd. A320M-S2H/A320M-S2H-CF, BIOS F50 11/28/2019
[    6.997386] RIP: 0010:dcn_bw_update_from_pplib+0xaa/0x2f0 [amdgpu]
[    6.997390] Code: 0c 24 85 c9 74 24 8d 71 ff 48 8d 44 24 04 48 8d 54 f4 0c eb 0d 48 83 c0 08 48 39 d0 0f 84 2e 01 00 00 44 8b 00 45 85 c0 75 eb <0f> 0b e8 8f d5 e7 df 4c 89 e2 be 04 00 00 00 4c 89 ef e8 7f 5c fe
[    6.997391] RSP: 0018:ffffb492c0b87630 EFLAGS: 00010246
[    6.997393] RAX: ffffb492c0b87634 RBX: ffff8b0498230000 RCX: 0000000000000004
[    6.997394] RDX: ffffb492c0b87654 RSI: 0000000000000003 RDI: ffffffffa1d64a42
[    6.997395] RBP: ffffb492c0b87778 R08: 0000000000000000 R09: 0000000000000004
[    6.997396] R10: 0000000000000000 R11: 0000000000000001 R12: ffffb492c0b876c0
[    6.997397] R13: ffff8b04ab9d3880 R14: 0000000000000001 R15: 0000000000000000
[    6.997399] FS:  00007faef01fd840(0000) GS:ffff8b04b0540000(0000) knlGS:0000000000000000
[    6.997401] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.997402] CR2: 00007f2c5b94e100 CR3: 00000003abce0000 CR4: 00000000003406e0
[    6.997403] Call Trace:
[    6.997554]  dcn10_create_resource_pool+0x835/0xb60 [amdgpu]
[    6.997561]  ? __kmalloc+0x196/0x270
[    6.997700]  dc_create_resource_pool+0xc3/0x130 [amdgpu]
[    6.997863]  dc_create+0x24d/0x760 [amdgpu]
[    6.997868]  ? _raw_spin_lock_irqsave+0x26/0x50
[    6.997873]  ? kmem_cache_alloc_trace+0x17b/0x220
[    6.998025]  ? amdgpu_cgs_create_device+0x23/0x50 [amdgpu]
[    6.998188]  amdgpu_dm_init+0x13d/0x1c0 [amdgpu]
[    6.998346]  ? phm_wait_for_register_unequal.part.0+0x50/0x80 [amdgpu]
[    6.998509]  dm_hw_init+0xe/0x20 [amdgpu]
[    6.998679]  amdgpu_device_init.cold+0x14b2/0x17cb [amdgpu]
[    6.998802]  amdgpu_driver_load_kms+0x53/0x1a0 [amdgpu]
[    6.998823]  drm_dev_register+0x111/0x150 [drm]
[    6.998944]  amdgpu_pci_probe+0xee/0x150 [amdgpu]
[    6.998949]  ? __pm_runtime_resume+0x49/0x60
[    6.998954]  local_pci_probe+0x42/0x80
[    6.998956]  ? pci_match_device+0xd7/0x100
[    6.998959]  pci_device_probe+0x104/0x1a0
[    6.998963]  really_probe+0x147/0x3c0
[    6.998967]  driver_probe_device+0xb6/0x100
[    6.998970]  device_driver_attach+0x53/0x60
[    6.998973]  __driver_attach+0x8a/0x150
[    6.998975]  ? device_driver_attach+0x60/0x60
[    6.998978]  bus_for_each_dev+0x89/0xd0
[    6.998981]  bus_add_driver+0x14d/0x1f0
[    6.998984]  driver_register+0x6c/0xc0
[    6.998987]  ? 0xffffffffc1112000
[    6.998991]  do_one_initcall+0x59/0x234
[    6.998996]  do_init_module+0x5c/0x230
[    6.999000]  load_module+0x2729/0x29e0
[    6.999008]  ? __do_sys_init_module+0x169/0x190
[    6.999010]  __do_sys_init_module+0x169/0x190
[    6.999016]  do_syscall_64+0x4e/0x140
[    6.999019]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[    6.999021] RIP: 0033:0x7faef0fb2d2e
[    6.999024] Code: 48 8b 0d 55 01 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 49 89 ca b8 af 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 22 01 0c 00 f7 d8 64 89 01 48
[    6.999026] RSP: 002b:00007fff03569168 EFLAGS: 00000246 ORIG_RAX: 00000000000000af
[    6.999028] RAX: ffffffffffffffda RBX: 0000562375da7cd0 RCX: 00007faef0fb2d2e
[    6.999029] RDX: 00007faef0c7984d RSI: 00000000008e1689 RDI: 00007faeee44d010
[    6.999030] RBP: 00007faeee44d010 R08: 0000562375d0301a R09: 0000000000000006
[    6.999032] R10: 0000562375d03010 R11: 0000000000000246 R12: 00007faef0c7984d
[    6.999033] R13: 0000000000000001 R14: 0000562375d06720 R15: 0000562375da7cd0
[    6.999037] ---[ end trace 2e9c80ba9e4e40e6 ]---

Zizilla · 2019-12-14 23:26:46

I could not make all changes in BIOS, you recommended.
I could not find any KVM related setting at all.

I could force IGP, and IOMMU.
The new dmesg: https://pastebin.com/Hgq9wST8
My board is: A320M-S2H

Yes, I am booting to UEFI.
CSM is disabled.

Last edited by Zizilla (2019-12-14 23:28:41)

Ropid · 2019-12-15 04:13:57

The virtualization thing is named "SVM" on AMD. That's what you have to look for in the BIOS.

Besides IOMMU there's also a thing named "SR-IOV" that you should look for and enable.

Zizilla · 2019-12-15 18:30:53

Ropid wrote:

The virtualization thing is named "SVM" on AMD. That's what you have to look for in the BIOS.
Besides IOMMU there's also a thing named "SR-IOV" that you should look for and enable.

Unfortunatelly neither of them can be found in my BIOS.
Manual says SVM should be under MIT->Advanced CPU Core Settings, but I dont have this manu item.
Maybe Gigabyte removed this in a recent BIOS?

But anyway, is this important for enabling my IGP instead of the dGPU?
First I would be happy, if the IGP could work fine, with acceleration.
Then I could go for the KVM thing.

Ropid · 2019-12-16 00:56:40

I tried looking for screenshots of Gigabyte's UEFI/BIOS of your board. Make sure you are not in the "easy" mode of the Gigabyte menus. Switch to the "classic" menu mode. Maybe that's why you can't find the advanced entries.

You need to enable SVM. You cannot use KVM without it.

I don't know why your IGP works slow in the browser and such. That glxinfo output you showed looks good to me, as if it's working right.

Last edited by Ropid (2019-12-16 01:05:08)

Lone_Wolf · 2019-12-16 11:06:26

I do think your firmware settings are the cause of your IGP problems.

While IOMMU / SVM / SR-IOV / AMD-Vi are often thought to be only relevant for virtualisation, this is a misunderstanding*.
AMD has used IOMMU v1 / AMD-V to manage x86_64 devices since approx 2008 .
Iommu v2 & AMD-Vi are merely expansions that add some things while still performing great for the original usage.

* possibly caused by Intel using different methods to manage 64-bit sytems

Zizilla · 2019-12-18 20:14:10

Hi Guys,

You were right.
There was a setting for SVM. It was very well hidden under CPU freq settings.

The new dmesg: https://pastebin.com/qrPm3s76

The IGP speed is still very slow.
GLXGears: 1fps on IGP, 60fps on dGPU.

I saw this in dmesg:

[ 6.764091] kfd kfd: Failed to resume IOMMU for device 1002:15dd
[ 6.764302] kfd kfd: device 1002:15dd NOT added due to errors

Last edited by Zizilla (2019-12-19 15:32:52)

Lone_Wolf · 2019-12-21 12:52:13

and again IOMMU is mentioned in the error message...

Time to check basics .

The latest bios from gigabyte for this motherboard is F50 from 2019/11/28 .
Are you using that version ?
If not, try it.

[    0.618969] pci 0000:00:00.2: can't derive routing for PCI INT A
[    0.618970] pci 0000:00:00.2: PCI INT A: not connected INTA

I also finally realised why INT A related messages worry me so :

The INTx signals are from the pci era and PCI express uses MSI / MSI-X instead .
https://electronics.stackexchange.com/a/343218/25469

Once you're sure you have the latest bios, post

# dmesg | grep MSI
# dmesg | grep INT

Last edited by Lone_Wolf (2019-12-21 12:52:33)

Zizilla · 2019-12-23 16:54:35

Hi Lone_Wolf,

Yes, I was already using the latest (F50) BIOS.

$ dmesg | grep MSI
[    0.500737] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]

$ dmesg | grep INT
[    0.000000] ACPI: DSDT 0x00000000BC22C218 006B6E (v02 ALASKA A M I    01072009 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC2385A0 003699 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC23BC80 006810 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC260060 0012BE (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC261320 00146E (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC262790 000373 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC262B08 001EC3 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x00000000BC2649D0 0000BF (v01 ALASKA A M I    00001000 INTL 20160930)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[    0.631952] pci 0000:00:00.2: can't derive routing for PCI INT A
[    0.631953] pci 0000:00:00.2: PCI INT A: not connected

Lone_Wolf · 2019-12-24 11:57:41

Check the hardware revision of your motherboard. It should be in the left bottom corner of you Mobo, manual page 2 shows a picture where to find it.

Firmware settings I'd like you to check (some may be greyed out)

Miscellaneous settings
PCIe slot configuration
3dMark01 Enhancement

Bios
Fast Boot
VGA Support
Other PCI Device ROM Priority

Peripherals
Initial Display output

Chipset
Integrated Graphics
UMA frame buffer size

Zizilla · 2019-12-25 11:32:37

Motherboard revision is 1.0

Miscellaneous settings
PCIe slot configuration: Auto
3dMark01 Enhancement: Disabled

Bios
Fast Boot: Disabled
VGA Support: Could not find this option
Other PCI Device ROM Priority: I had to enable CSM support to see this option. It was set to Legacy only. I have switched it to UEFI only.

Peripherals
Initial Display output: IGD Video

Chipset
Integrated Graphics: Forces (have tried all ot these before, without luck)
UMA frame buffer size: I had to switch UMA mode from Auto to UMA specified to see this option. It is the default 2G now.

Aaaaaand this two switch did the trick. glxgears now run 60fps on IGP also.

Thank you very much for the help!!!

_____________________

Now, I start searching for the solution for turning on/off the dVGA.
I will start with the links in progandy's previous post (Thanks for that!).

Last edited by Zizilla (2019-12-25 11:39:16)

Lone_Wolf · 2019-12-25 12:31:53

Other PCI Device ROM Priority: I had to enable CSM support to see this option. It was set to Legacy only. I have switched it to UEFI only.

This was probably the culprit.
I have encountered a few other ryzen systems where settings that could only be changed when CSM was enabled influenced behavior AFTER CSM was disabled.

Could you rerun (and post the output of) dmesg | grep MSI and dmesg | grep INT ?
I expect those will look very different now.

Oh, and if you're satisfied this is solved, preped [Solved] to the title (edit first post) .

Zizilla · 2019-12-25 21:12:28

 dmesg | grep MSI
[    0.498884] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]

$ dmesg | grep INT
[    0.000000] ACPI: DSDT 0x000000003C22C208 006B6E (v02 ALASKA A M I    01072009 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C238590 003699 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C23BC70 006810 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C243FB0 0012BE (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C245270 00146E (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C2466E0 000373 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C246A58 001EC3 (v01 ALASKA A M I    00000001 INTL 20160930)
[    0.000000] ACPI: SSDT 0x000000003C248920 0000BF (v01 ALASKA A M I    00001000 INTL 20160930)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[    0.630816] pci 0000:00:00.2: can't derive routing for PCI INT A
[    0.630817] pci 0000:00:00.2: PCI INT A: not connected

It look the same to me.

I still have the dGPU ON/OFF question pending. Should I open a new thread for that and mark this as solved, or should I keep this open?

Last edited by Zizilla (2019-12-25 21:15:13)

Ropid · 2019-12-25 21:31:28

Zizilla wrote:

[...]
- dGPU should be able to be turned off for power saving, and turned on for OpenCL render (Blender) and for GPU passthrough (virtual windows gaming).
[...]

What is the current power usage? You can find what the driver thinks about that in a file like this:

/sys/kernel/debug/dri/0/amdgpu_pm_info

I guess with two cards there will be a "0" and a "1" sub-folder in that "/sys/kernel/debug/dri/" location. You can find the right one by looking into the "name" file:

$ sudo sh -c 'tail /sys/kernel/debug/dri/?/name'
amdgpu dev=0000:0d:00.0 master=pci:0000:0d:00.0 unique=0000:0d:00.0

and checking "lspci -D".

There's also a tool named "rocm-smi" that might be useful. It's very small, pacman says just 136K disk usage for it. It prints a list like this:

========================ROCm System Management Interface========================
================================================================================
GPU  Temp   AvgPwr   SCLK    MCLK    Fan     Perf    PwrCap  VRAM%  GPU%  
0    32.0c  32.202W  608Mhz  300Mhz  23.92%  manual  142.0W   11%   0%    
================================================================================
==============================End of ROCm SMI Log ==============================

Maybe you are lucky about this problem and the driver perhaps by itself already does something like a shutdown of the card when there's absolutely zero use of it.

Last edited by Ropid (2019-12-25 21:32:46)

Zizilla · 2019-12-25 22:16:05

GFX Clocks and Power:
        300 MHz (MCLK)
        1344 MHz (SCLK)
        600 MHz (PSTATE_SCLK)
        1000 MHz (PSTATE_MCLK)
        1200 mV (VDDGFX)
        56.156 W (average GPU)

$ lspci -D
0000:00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Root Complex
0000:00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 IOMMU
0000:00:01.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:01.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0]
0000:00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 PCIe GPP Bridge [6:0]
0000:00:08.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge
0000:00:08.1 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus A
0000:00:08.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Internal PCIe GPP Bridge 0 to Bus B
0000:00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 61)
0000:00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 51)
0000:00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 0
0000:00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 1
0000:00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 2
0000:00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 3
0000:00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 4
0000:00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 5
0000:00:18.6 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 6
0000:00:18.7 Host bridge: Advanced Micro Devices, Inc. [AMD] Raven/Raven2 Device 24: Function 7
0000:01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] (rev e7)
0000:01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590]
0000:02:00.0 USB controller: Advanced Micro Devices, Inc. [AMD] Device 43bc (rev 02)
0000:02:00.1 SATA controller: Advanced Micro Devices, Inc. [AMD] Device 43b8 (rev 02)
0000:02:00.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Device 43b3 (rev 02)
0000:03:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:03:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:03:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:03:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port (rev 02)
0000:07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 0c)
0000:08:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] (rev c6)
0000:08:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Raven/Raven2/Fenghuang HDMI/DP Audio Controller
0000:08:00.2 Encryption controller: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 10h-1fh) Platform Security Processor
0000:08:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
0000:08:00.4 USB controller: Advanced Micro Devices, Inc. [AMD] Raven USB 3.1
0000:09:00.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 61)

$ rocm-smi


========================ROCm System Management Interface========================
================================================================================
WARNING: GPU[1]	: Unable to read /sys/class/drm/card1/device/gpu_busy_percent
GPU  Temp   AvgPwr   SCLK     MCLK    Fan     Perf  PwrCap  VRAM%  GPU%  
0    31.0c  59.154W  1411Mhz  300Mhz  16.86%  auto  175.0W    1%   0%    
1    39.0c  N/A      400Mhz   933Mhz  None%   auto  N/A      15%   N/A   
================================================================================
==============================End of ROCm SMI Log ==============================

Ropid · 2019-12-25 23:06:57

Well, that power usage looks annoying, I guess you do have to manually power it down somehow.

I tried looking around a bit, and I saw someone write that "tlp" disables the second card when he runs "tlp bat" to switch to battery mode. This should then mean that it's possible? Maybe you can experiment with that "tlp" program and see if it does something interesting on your computer, and then if it does work, then try to find out what it does behind the scenes. That "tlp" thing is this here:

https://wiki.archlinux.org/index.php/TLP

Something else I've found is that you can check with "sudo lscpi -vv" and look for the "capabilities: ... power management" section of the output:

0d:00.0 VGA compatible controller: Advanced Micro Devices, ...
...
	Capabilities: [50] Power Management version 3
		Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1+,D2+,D3hot+,D3cold+)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
...

The "status" line is interesting. Here in this example the card is in "D0" which apparently means it's fully powered on. In the previous line you can see there's a list of other states available up to "D3". There's an article here in the kernel documentation about the stuff:

https://www.kernel.org/doc/Documentation/power/pci.txt

Maybe there's a way to put the card into "D3" manually, without the driver having to do it?

The other thing you could play around with is completely removing the card. That can be done like this (this example here is how I was resetting the audio controller on my PC):

echo 1 | sudo tee '/sys/bus/pci/devices/0000:00:1b.0/remove'

Hopefully the card then goes into the D3 state by itself when its driver is gone. I don't know if you can still check in software what the card is doing after it's removed. You get the card back like this:

echo 1 | sudo tee /sys/bus/pci/rescan

Last edited by Ropid (2019-12-25 23:08:15)

Arch Linux

#1 2019-12-07 16:01:46

Using two AMD GPUs simultaneously

#2 2019-12-08 11:55:43

Re: Using two AMD GPUs simultaneously

#3 2019-12-09 20:40:21

Re: Using two AMD GPUs simultaneously

#4 2019-12-09 21:43:36

Re: Using two AMD GPUs simultaneously

#5 2019-12-10 13:00:43

Re: Using two AMD GPUs simultaneously

#6 2019-12-14 23:26:46

Re: Using two AMD GPUs simultaneously

#7 2019-12-15 04:13:57

Re: Using two AMD GPUs simultaneously

#8 2019-12-15 18:30:53

Re: Using two AMD GPUs simultaneously

#9 2019-12-16 00:56:40

Re: Using two AMD GPUs simultaneously

#10 2019-12-16 11:06:26

Re: Using two AMD GPUs simultaneously

#11 2019-12-18 20:14:10

Re: Using two AMD GPUs simultaneously

#12 2019-12-21 12:52:13

Re: Using two AMD GPUs simultaneously

#13 2019-12-23 16:54:35

Re: Using two AMD GPUs simultaneously

#14 2019-12-24 11:57:41

Re: Using two AMD GPUs simultaneously

#15 2019-12-25 11:32:37

Re: Using two AMD GPUs simultaneously

#16 2019-12-25 12:31:53

Re: Using two AMD GPUs simultaneously

#17 2019-12-25 21:12:28

Re: Using two AMD GPUs simultaneously

#18 2019-12-25 21:31:28

Re: Using two AMD GPUs simultaneously

#19 2019-12-25 22:16:05

Re: Using two AMD GPUs simultaneously

#20 2019-12-25 23:06:57

Re: Using two AMD GPUs simultaneously

Board footer