PCI Passthrough host freeze

StillerHarpo · 2016-12-11 14:46:16

I had a working pci passthrough before with seabios and pci-stub. After i reinstalled arch on my ssd. and configured my system for pci passthrough according to the wiki page (this time i used ovmf and vfio-pci), my entire system freeze most of the time, when i start the vm. I get windows installed. Because of the freezes i wanted to reinstall it. So i writed /dev/zero to the disk with windows. But the freeszes occured again, before i could install windows. Sometimes i dont get them, but then i dont get a signal from my gpu(screen dont get signal).

$ dmesg|grep -e DMAR -e IOMM
-------------------------------------------------------------------------------
[    1.752554] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40

#!/bin/bash
shopt -s nullglob
for d in /sys/kernel/iommu_groups/*/devices/*; do 
    n=${d#*/iommu_groups/*}; n=${n%%/*}
    printf 'IOMMU Group %s ' "$n"
    lspci -nns "${d##*/}"
done;

output:

IOMMU Group 0 00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD/ATI] RD9x0/RX980 Host Bridge [1002:5a14] (rev 02)
IOMMU Group 10 00:13.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 10 00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
IOMMU Group 11 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller [1002:4385] (rev 42)
IOMMU Group 12 00:14.2 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) [1002:4383] (rev 40)
IOMMU Group 13 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d] (rev 40)
IOMMU Group 14 00:14.4 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge [1002:4384] (rev 40)
IOMMU Group 15 00:14.5 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller [1002:4399]
IOMMU Group 16 00:15.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) [1002:43a0]
IOMMU Group 16 00:15.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1) [1002:43a1]
IOMMU Group 16 00:15.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 2) [1002:43a2]
IOMMU Group 16 00:15.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 3) [1002:43a3]
IOMMU Group 16 0a:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 09)
IOMMU Group 16 0b:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 16 0c:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 17 00:16.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 17 00:16.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
IOMMU Group 18 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GT218 [GeForce 210] [10de:0a65] (rev a2)
IOMMU Group 18 01:00.1 Audio device [0403]: NVIDIA Corporation High Definition Audio Controller [10de:0be3] (rev a1)
IOMMU Group 19 02:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU Group 1 00:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GFX port 0) [1002:5a16]
IOMMU Group 20 03:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
IOMMU Group 21 04:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller [1b21:1142]
IOMMU Group 22 06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X] [1002:6798]
IOMMU Group 22 06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti HDMI Audio [Radeon HD 7870 XT / 7950/7970] [1002:aaa0]
IOMMU Group 2 00:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 0) [1002:5a18]
IOMMU Group 3 00:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 1) [1002:5a19]
IOMMU Group 4 00:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 4) [1002:5a1c]
IOMMU Group 5 00:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP Port 5) [1002:5a1d]
IOMMU Group 6 00:0b.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD990 PCI to PCI bridge (PCI Express GFX2 port 0) [1002:5a1f]
IOMMU Group 7 00:0d.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] RD890/RD9x0/RX980 PCI to PCI bridge (PCI Express GPP2 Port 0) [1002:5a1e]
IOMMU Group 8 00:11.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] [1002:4391] (rev 40)
IOMMU Group 9 00:12.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
IOMMU Group 9 00:12.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]

$ modinfo vfio-pci
-------------------------------------------------------------------------------------------------------------
filename:       /lib/modules/4.8.12-3-ARCH/kernel/drivers/vfio/pci/vfio-pci.ko.gz
description:    VFIO PCI - User Level meta-driver
author:         Alex Williamson <alex.williamson@redhat.com>
license:        GPL v2
version:        0.2
srcversion:     69F76C851580CBBE7E2531C
depends:        vfio,irqbypass,vfio_virqfd
intree:         Y
vermagic:       4.8.12-3-ARCH SMP preempt mod_unload modversions 
parm:           ids:Initial PCI IDs to add to the vfio driver, format is "vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]" and multiple comma separated entries can be specified (string)
parm:           nointxmask:Disable support for PCI 2.3 style INTx masking.  If this resolves problems for specific devices, report lspci -vvvxxx to linux-pci@vger.kernel.org so the device can be fixed automatically via the broken_intx_masking flag. (bool)
parm:           disable_vga:Disable VGA resource access through vfio-pci (bool)
parm:           disable_idle_d3:Disable using the PCI D3 low power state for idle, unused devices (bool)

$ dmesg | grep -i vfio
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------       
[    1.798256] VFIO - User Level meta-driver version: 0.3
[    1.815881] vfio_pci: add [1002:6798[ffff:ffff]] class 0x000000/00000000
[    1.832548] vfio_pci: add [1002:aaa0[ffff:ffff]] class 0x000000/00000000

$ lspci -nnk -d 1002:6798
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
06:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X] [1002:6798]
	Subsystem: XFX Pine Group Inc. Double D HD 7970 [1682:3214]
	Kernel driver in use: vfio-pci
	Kernel modules: radeon

$ lspci -nnk -d 1002:aaa0
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
06:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti HDMI Audio [Radeon HD 7870 XT / 7950/7970] [1002:aaa0]
	Subsystem: XFX Pine Group Inc. Device [1682:aaa0]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

The script to start my vm

/usr/bin/synergys --daemon --config /etc/synergy.conf
xrandr --output VGA-0 --off
QEMU_PA_SAMPLES=128 QEMU_AUDIO_DRV=pa
qemu-system-x86_64 \
  -serial none \
  -parallel none \
  -drive if=pflash,format=raw,file=/home/florian/kvm/ovmf_x64.bin \
  -enable-kvm \
  -m 8192\
  -cpu host,kvm=off,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time  \
  -smp sockets=1,cores=4,threads=1 \
  -nographic \
  -boot order=c \
  -device virtio-scsi-pci,id=scsi \
  -soundhw hda \
  -drive file=/home/florian/kvm/virtio-win-0.1.126.iso,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd \
  -drive file=/home/florian/kvm/en_windows_10_education_version_1511_x64_dvd_7223856.iso,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
  -drive file=/dev/sdb,id=ssd,format=raw,if=none -device scsi-hd,drive=ssd  \
  -drive file=/dev/sdc,id=disk,format=raw,if=none -device scsi-hd,drive=disk \
  -device vfio-pci,host=06:00.0,multifunction=on  \
  -device vfio-pci,host=06:00.1   \
  -usb -usbdevice host:0461:4d0f \
  -usb -usbdevice host:1a34:0806 
setxkbmap de
xrandr --auto
xrandr --output VGA-0 --left-of DVI-I-1
killall synergys

$ pacman -Qs qemu
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
local/ovmf 18419-1
    Tianocore UEFI firmware for qemu.
local/qemu 2.7.0-2
    A generic and open source machine emulator and virtualizer
local/vde2 2.3.2-8
    Virtual Distributed Ethernet for emulators like qemu

$ uname -r
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
4.8.12-3-ARCH

Edit:

With seabios and pci-stup the same thing happens (sometimes windows starts, but most of the time the entire pc freeze)

I have installed windows without passing through the gpu (if i dont pass it throug everything works fine).

Last edited by StillerHarpo (2016-12-18 16:08:45)

Arch Linux

#1 2016-12-11 14:46:16

PCI Passthrough host freeze

Board footer