Arch Linux

NeroBurner

Member

Registered: 2013-09-05

Posts: 30

[SOLVED] PCI Passthrough AMD RX470 fails

I'm trying to set up PCI passthrough of my AMD RX 470. I Followed the steps from https://wiki.archlinux.org/index.php/PC … h_via_OVMF and isolated the GPU, rebooted, install new Windows/ubuntu/arch guest, pass the GPU. And with the GPU passed the virtual machine is not able to start

I found the following big error messages in dmesg, but I don't know where to look next

[  134.536552] [drm] amdgpu: finishing device.
[  134.536596] WARNING: CPU: 2 PID: 557 at drivers/gpu/drm/drm_crtc.c:108 drm_crtc_force_disable+0x5d/0x70 [drm]
[  134.536597] Modules linked in: fuse devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache nvidia_drm(PO) amdkfd nvidia_modeset(PO) amd_iommu_v2 nvidia(PO) amdgpu nls_iso8859_1 nls_cp437 vfat fat edac_mce_amd kvm_amd wmi_bmof mxm_wmi snd_usb_audio snd_hda_codec_realtek chash kvm snd_hda_codec_generic ttm snd_hda_codec_hdmi snd_usbmidi_lib drm_kms_helper snd_rawmidi input_leds snd_seq_device crct10dif_pclmul crc32_pclmul snd_hda_intel ghash_clmulni_intel joydev pcbc mousedev led_class snd_hda_codec igb drm snd_hda_core ccp ptp snd_hwdep pps_core snd_pcm i2c_algo_bit alx dca rng_core mdio ipmi_devintf ipmi_msghandler agpgart snd_timer aesni_intel syscopyarea sysfillrect sysimgblt snd fb_sys_fops soundcore aes_x86_64 sp5100_tco crypto_simd glue_helper cryptd i2c_piix4
[  134.536632]  pcspkr shpchp k10temp rtc_cmos gpio_amdpt evdev wmi pinctrl_amd mac_hid acpi_cpufreq sg crypto_user ip_tables x_tables ext4 crc16 mbcache jbd2 fscrypto sd_mod hid_generic usbhid hid ahci xhci_pci libahci crc32c_intel xhci_hcd libata usbcore scsi_mod usb_common vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
[  134.536652] CPU: 2 PID: 557 Comm: libvirtd Tainted: P           O     4.15.9-1-ARCH #1
[  134.536653] Hardware name: Gigabyte Technology Co., Ltd. AX370-Gaming 5/AX370-Gaming 5, BIOS F10 12/07/2017
[  134.536663] RIP: 0010:drm_crtc_force_disable+0x5d/0x70 [drm]
[  134.536664] RSP: 0018:ffffbc488b6c7cc0 EFLAGS: 00010286
[  134.536666] RAX: ffffffffc1c27540 RBX: ffff9bdfca8ae800 RCX: 0000000000000000
[  134.536667] RDX: ffffbc488b6c7cc0 RSI: ffff9bdfca8ae800 RDI: ffffbc488b6c7cf0
[  134.536667] RBP: ffff9bdfca837368 R08: ffffffffc07b8220 R09: ffff9bdfb02fc200
[  134.536668] R10: fffff5a2d00d1cc0 R11: 0000000000000000 R12: ffff9bdfca837000
[  134.536669] R13: ffffffffc1ce8670 R14: 0000000000000060 R15: ffff9bdfacf48fe0
[  134.536670] FS:  00007fd5468c8700(0000) GS:ffff9bdfeec80000(0000) knlGS:0000000000000000
[  134.536671] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  134.536671] CR2: 00007fd5468c44a8 CR3: 0000000401cf8000 CR4: 00000000003406e0
[  134.536672] Call Trace:
[  134.536687]  drm_crtc_force_disable_all+0x4e/0x70 [drm]
[  134.536728]  amdgpu_device_fini+0x1c5/0x1e0 [amdgpu]
[  134.536760]  amdgpu_driver_unload_kms+0x4a/0x90 [amdgpu]
[  134.536770]  drm_dev_unregister+0x3f/0xe0 [drm]
[  134.536801]  amdgpu_pci_remove+0x19/0x40 [amdgpu]
[  134.536805]  pci_device_remove+0x3b/0xb0
[  134.536810]  device_release_driver_internal+0x15a/0x220
[  134.536812]  unbind_store+0x82/0x150
[  134.536815]  kernfs_fop_write+0x10f/0x190
[  134.536819]  __vfs_write+0x36/0x180
[  134.536821]  ? __check_object_size+0x9b/0x1a0
[  134.536823]  vfs_write+0xb1/0x1a0
[  134.536826]  SyS_write+0x52/0xc0
[  134.536829]  do_syscall_64+0x74/0x190
[  134.536832]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[  134.536834] RIP: 0033:0x7fd5504808eb
[  134.536835] RSP: 002b:00007fd5468c7670 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
[  134.536836] RAX: ffffffffffffffda RBX: 00007fd528019e04 RCX: 00007fd5504808eb
[  134.536837] RDX: 000000000000000c RSI: 00007fd528019e04 RDI: 0000000000000014
[  134.536838] RBP: 000000000000000c R08: 0000000000000000 R09: 7665642f6963702f
[  134.536838] R10: 0000000000000000 R11: 0000000000000293 R12: 00007fd528019e04
[  134.536839] R13: 0000000000000000 R14: 0000000000000014 R15: 00007fd5533ac35b
[  134.536840] Code: 48 8b 80 98 03 00 00 48 83 78 20 00 75 1d 48 89 d7 e8 88 ff ff ff 48 8b 4c 24 30 65 48 33 0c 25 28 00 00 00 75 09 48 83 c4 38 c3 <0f> 0b eb df e8 0a e5 8c f0 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 
[  134.536864] ---[ end trace b0cc8ceb3ad2cd0c ]---
[  134.536893] WARNING: CPU: 2 PID: 557 at drivers/gpu/drm/drm_crtc.c:499 drm_mode_set_config_internal+0x21/0x30 [drm]
[  134.536894] Modules linked in: fuse devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache nvidia_drm(PO) amdkfd nvidia_modeset(PO) amd_iommu_v2 nvidia(PO) amdgpu nls_iso8859_1 nls_cp437 vfat fat edac_mce_amd kvm_amd wmi_bmof mxm_wmi snd_usb_audio snd_hda_codec_realtek chash kvm snd_hda_codec_generic ttm snd_hda_codec_hdmi snd_usbmidi_lib drm_kms_helper snd_rawmidi input_leds snd_seq_device crct10dif_pclmul crc32_pclmul snd_hda_intel ghash_clmulni_intel joydev pcbc mousedev led_class snd_hda_codec igb drm snd_hda_core ccp ptp snd_hwdep pps_core snd_pcm i2c_algo_bit alx dca rng_core mdio ipmi_devintf ipmi_msghandler agpgart snd_timer aesni_intel syscopyarea sysfillrect sysimgblt snd fb_sys_fops soundcore aes_x86_64 sp5100_tco crypto_simd glue_helper cryptd i2c_piix4
[  134.536914]  pcspkr shpchp k10temp rtc_cmos gpio_amdpt evdev wmi pinctrl_amd mac_hid acpi_cpufreq sg crypto_user ip_tables x_tables ext4 crc16 mbcache jbd2 fscrypto sd_mod hid_generic usbhid hid ahci xhci_pci libahci crc32c_intel xhci_hcd libata usbcore scsi_mod usb_common vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
[  134.536923] CPU: 2 PID: 557 Comm: libvirtd Tainted: P        W  O     4.15.9-1-ARCH #1
[  134.536924] Hardware name: Gigabyte Technology Co., Ltd. AX370-Gaming 5/AX370-Gaming 5, BIOS F10 12/07/2017
[  134.536931] RIP: 0010:drm_mode_set_config_internal+0x21/0x30 [drm]
[  134.536931] RSP: 0018:ffffbc488b6c7cb8 EFLAGS: 00010286
[  134.536932] RAX: ffffffffc1c27540 RBX: ffff9bdfca8ae800 RCX: 0000000000000000
[  134.536933] RDX: ffffbc488b6c7cc0 RSI: ffff9bdfca8ae800 RDI: ffffbc488b6c7cc0
[  134.536933] RBP: ffff9bdfca837368 R08: ffffffffc07b8220 R09: ffff9bdfb02fc200
[  134.536934] R10: fffff5a2d00d1cc0 R11: 0000000000000000 R12: ffff9bdfca837000
[  134.536934] R13: ffffffffc1ce8670 R14: 0000000000000060 R15: ffff9bdfacf48fe0
[  134.536935] FS:  00007fd5468c8700(0000) GS:ffff9bdfeec80000(0000) knlGS:0000000000000000
[  134.536936] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  134.536937] CR2: 00007fd5468c44a8 CR3: 0000000401cf8000 CR4: 00000000003406e0
[  134.536937] Call Trace:
[  134.536945]  drm_crtc_force_disable+0x48/0x70 [drm]
[  134.536952]  drm_crtc_force_disable_all+0x4e/0x70 [drm]
[  134.536973]  amdgpu_device_fini+0x1c5/0x1e0 [amdgpu]
[  134.536994]  amdgpu_driver_unload_kms+0x4a/0x90 [amdgpu]
[  134.537001]  drm_dev_unregister+0x3f/0xe0 [drm]
[  134.537021]  amdgpu_pci_remove+0x19/0x40 [amdgpu]
[  134.537023]  pci_device_remove+0x3b/0xb0
[  134.537025]  device_release_driver_internal+0x15a/0x220
[  134.537029]  unbind_store+0x82/0x150
[  134.537030]  kernfs_fop_write+0x10f/0x190
[  134.537032]  __vfs_write+0x36/0x180
[  134.537034]  ? __check_object_size+0x9b/0x1a0
[  134.537036]  vfs_write+0xb1/0x1a0
[  134.537038]  SyS_write+0x52/0xc0
[  134.537040]  do_syscall_64+0x74/0x190
[  134.537041]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[  134.537043] RIP: 0033:0x7fd5504808eb
[  134.537043] RSP: 002b:00007fd5468c7670 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
[  134.537045] RAX: ffffffffffffffda RBX: 00007fd528019e04 RCX: 00007fd5504808eb
[  134.537045] RDX: 000000000000000c RSI: 00007fd528019e04 RDI: 0000000000000014
[  134.537046] RBP: 000000000000000c R08: 0000000000000000 R09: 7665642f6963702f
[  134.537047] R10: 0000000000000000 R11: 0000000000000293 R12: 00007fd528019e04
[  134.537048] R13: 0000000000000000 R14: 0000000000000014 R15: 00007fd5533ac35b
[  134.537049] Code: 89 e8 5b 5d 41 5c c3 0f 1f 00 0f 1f 44 00 00 48 8b 47 08 48 8b 00 48 8b 80 98 03 00 00 48 83 78 20 00 75 07 31 f6 e9 cf fe ff ff <0f> 0b 31 f6 e9 c6 fe ff ff 66 0f 1f 44 00 00 0f 1f 44 00 00 48 
[  134.537073] ---[ end trace b0cc8ceb3ad2cd0d ]---
[  134.537091] WARNING: CPU: 2 PID: 557 at drivers/gpu/drm/drm_atomic.c:272 drm_atomic_get_crtc_state+0xe6/0x100 [drm]
[  134.537092] Modules linked in: fuse devlink rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache nvidia_drm(PO) amdkfd nvidia_modeset(PO) amd_iommu_v2 nvidia(PO) amdgpu nls_iso8859_1 nls_cp437 vfat fat edac_mce_amd kvm_amd wmi_bmof mxm_wmi snd_usb_audio snd_hda_codec_realtek chash kvm snd_hda_codec_generic ttm snd_hda_codec_hdmi snd_usbmidi_lib drm_kms_helper snd_rawmidi input_leds snd_seq_device crct10dif_pclmul crc32_pclmul snd_hda_intel ghash_clmulni_intel joydev pcbc mousedev led_class snd_hda_codec igb drm snd_hda_core ccp ptp snd_hwdep pps_core snd_pcm i2c_algo_bit alx dca rng_core mdio ipmi_devintf ipmi_msghandler agpgart snd_timer aesni_intel syscopyarea sysfillrect sysimgblt snd fb_sys_fops soundcore aes_x86_64 sp5100_tco crypto_simd glue_helper cryptd i2c_piix4
[  134.537111]  pcspkr shpchp k10temp rtc_cmos gpio_amdpt evdev wmi pinctrl_amd mac_hid acpi_cpufreq sg crypto_user ip_tables x_tables ext4 crc16 mbcache jbd2 fscrypto sd_mod hid_generic usbhid hid ahci xhci_pci libahci crc32c_intel xhci_hcd libata usbcore scsi_mod usb_common vfio_pci irqbypass vfio_virqfd vfio_iommu_type1 vfio
[  134.537120] CPU: 2 PID: 557 Comm: libvirtd Tainted: P        W  O     4.15.9-1-ARCH #1
[  134.537120] Hardware name: Gigabyte Technology Co., Ltd. AX370-Gaming 5/AX370-Gaming 5, BIOS F10 12/07/2017
[  134.537128] RIP: 0010:drm_atomic_get_crtc_state+0xe6/0x100 [drm]
[  134.537129] RSP: 0018:ffffbc488b6c7c08 EFLAGS: 00010246
[  134.537130] RAX: 0000000000000000 RBX: ffffbc488b6c7cc0 RCX: 0000000000000000
[  134.537130] RDX: ffffffffc07e195d RSI: ffff9bdfca8ae800 RDI: ffff9bdfba985a00
[  134.537131] RBP: ffffbc488b6c7cc0 R08: ffff9bdfacf4f600 R09: ffff9bdfacf4f600
[  134.537132] R10: 0000000000000400 R11: 0000000000000000 R12: 0000000000000000
[  134.537133] R13: ffffffffc1ce8670 R14: ffff9bdfca8ae800 R15: ffff9bdfacf48fe0
[  134.537134] FS:  00007fd5468c8700(0000) GS:ffff9bdfeec80000(0000) knlGS:0000000000000000
[  134.537134] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  134.537135] CR2: 00007fd5468c44a8 CR3: 0000000401cf8000 CR4: 00000000003406e0
[  134.537136] Call Trace:
[  134.537142]  __drm_atomic_helper_set_config+0x38/0x2f0 [drm_kms_helper]
[  134.537148]  drm_atomic_helper_set_config+0x36/0x90 [drm_kms_helper]
[  134.537155]  __drm_mode_set_config_internal+0x67/0x110 [drm]
[  134.537163]  drm_crtc_force_disable+0x48/0x70 [drm]
[  134.537170]  drm_crtc_force_disable_all+0x4e/0x70 [drm]
[  134.537191]  amdgpu_device_fini+0x1c5/0x1e0 [amdgpu]
[  134.537212]  amdgpu_driver_unload_kms+0x4a/0x90 [amdgpu]
[  134.537219]  drm_dev_unregister+0x3f/0xe0 [drm]
[  134.537240]  amdgpu_pci_remove+0x19/0x40 [amdgpu]
[  134.537241]  pci_device_remove+0x3b/0xb0
[  134.537244]  device_release_driver_internal+0x15a/0x220
[  134.537245]  unbind_store+0x82/0x150
[  134.537246]  kernfs_fop_write+0x10f/0x190
[  134.537248]  __vfs_write+0x36/0x180
[  134.537249]  ? __check_object_size+0x9b/0x1a0
[  134.537251]  vfs_write+0xb1/0x1a0
[  134.537253]  SyS_write+0x52/0xc0
[  134.537254]  do_syscall_64+0x74/0x190
[  134.537256]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
[  134.537257] RIP: 0033:0x7fd5504808eb
[  134.537257] RSP: 002b:00007fd5468c7670 EFLAGS: 00000293 ORIG_RAX: 0000000000000001
[  134.537259] RAX: ffffffffffffffda RBX: 00007fd528019e04 RCX: 00007fd5504808eb
[  134.537259] RDX: 000000000000000c RSI: 00007fd528019e04 RDI: 0000000000000014
[  134.537260] RBP: 000000000000000c R08: 0000000000000000 R09: 7665642f6963702f
[  134.537260] R10: 0000000000000000 R11: 0000000000000293 R12: 00007fd528019e04
[  134.537261] R13: 0000000000000000 R14: 0000000000000014 R15: 00007fd5533ac35b
[  134.537262] Code: 10 02 00 00 48 c7 c2 98 37 7e c0 8b 4d 60 53 4c 8b 45 20 48 89 44 24 08 e8 18 17 ff ff 58 48 8b 04 24 48 83 c4 08 5b 5d 41 5c c3 <0f> 0b e9 34 ff ff ff 48 c7 c0 f4 ff ff ff e9 40 ff ff ff 48 98 
[  134.537290] ---[ end trace b0cc8ceb3ad2cd0e ]---

Specs:
- Motherboard: GIGABYTE Aorus GA-AX370-Gaming 5
  - most up to date BIOS F10 (same error with F6)
  - IOMMU enabled
  - virtualization enabled
- CPU: AMD Ryzen 5 1600 Six-Core Processor
- Host-GPU: Nvidia GTX 780Ti
- Guest-GPU: AMD RX470
- Virtualization: virt-manager + KVM

Last edited by NeroBurner (2018-03-20 21:30:57)

Lone_Wolf

Forum Moderator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 11,920

Re: [SOLVED] PCI Passthrough AMD RX470 fails

Do you get that error on host of guest ?

please post lspci -k and  a full dmesg / journal log from booting on the host.

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

(A works at time B)  && (time C > time B ) ≠  (A works at time C)

NeroBurner

Member

Registered: 2013-09-05

Posts: 30

Re: [SOLVED] PCI Passthrough AMD RX470 fails

I did forget to reissue mkinitcpio after changing the vfio-pci stub from my old guest card (GTX 560Ti) to my borrowed AMD RX470. I thought I didn't have to do it, because I changed no hooks. But lspci -k showed me, that the amdgpu driver was loaded instead of vfio. After mkinitcpio everything works fine. Thanks!