PCI Passthrough with 2 identical GPUs?

qKUqm3wtY4 · 2016-02-23 18:03:15

I'm trying to set up a Windows 10 VM with PCI passthrough. There are several extensive guides that are helping me through the process, but I've scoured the internet to solve this particular problem and I'm coming up short - I have two identical GPUs (see below).

# lspci
01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)

#lspci -n -s 1:
01:00.0 0300: 10de:13c2 (rev a1)
01:00.1 0403: 10de:0fbb (rev a1)

#lspci -n -s 2:
02:00.0 0300: 10de:13c2 (rev a1)
02:00.1 0403: 10de:0fbb (rev a1)

Due to this fact, I've gathered that I need to assign both GPUs to pci-stub (easy enough) and then bind each gpu to the appropriate driver (one to nvidia and the other to vfio-pci). I'm only an intermediate linux user, so please forgive me of any gaps in knowledge. I have written a script (see below) to do what I think needs to be done, but when I run this in initramfs via mkinitcpio, the nvidia driver fails to load at boot.

echo 0000:01:00.0 > /sys/bus/pci/drivers/pci-stub/unbind
echo 0000:01:00.1 > /sys/bus/pci/drivers/pci-stub/unbind
echo 0000:02:00.0 > /sys/bus/pci/drivers/pci-stub/unbind
echo 0000:02:00.1 > /sys/bus/pci/drivers/pci-stub/unbind
echo 0000:01:00.0 > /sys/bus/pci/drivers/nvidia/bind
echo 0000:01:00.1 > /sys/bus/pci/drivers/nvidia/bind
echo 0000:02:00.0 > /sys/bus/pci/drivers/vfio-pci/bind
echo 0000:02:00.1 > /sys/bus/pci/drivers/vfio-pci/bind

I'm using systemd as my bootloader, so I have pci-stub catching both GPUs there. To my understanding, initramfs should then unbind both GPUs from pci-stub, then assign nvidia to GPU1 and vfio-pci to GPU2. That isn't how it's working out, though. Clearly I don't understand something. Someone please help me understand exactly what I'm doing wrong and what needs to be done to fix it. Any help you can provide is greatly appreciated. Once I have this figured out, I intend to heavily update the Arch Wiki about PCI passthrough. Hope to hear from you soon!

loqs · 2016-02-23 18:54:46

https://bbs.archlinux.org/viewtopic.php … 8#p1568678 see link to a slightly different approach that avoids the unbinding no personal experience if it works.

qKUqm3wtY4 · 2016-02-23 19:04:25

loqs wrote:

https://bbs.archlinux.org/viewtopic.php … 8#p1568678 see link to a slightly different approach that avoids the unbinding no personal experience if it works.

Thanks for the assist. I've actually got that vfio.blogspot.co.uk post in one of my tabs. I tried this script originally and it didn't work. The script is too complex for me to understand and the guide doesn't exactly hold your hand. I may have done some things wrong on the first try, so I think I'll go back to this solution and give it another shot.

loqs · 2016-02-23 19:22:30

did you try?

[#!/bin/sh

DEVS="0000:01:00.0 0000:01:00.1 0000:02:00.0 0000:02:00.1"

for DEV in $DEVS; do
    echo "vfio-pci" > /sys/bus/pci/devices/$DEV/driver_override
done

modprobe -i vfio-pci

or

#!/bin/sh

for i in $(find /sys/devices/pci* -name boot_vga); do
        if [ $(cat $i) -eq 0 ]; then
                GPU=$(dirname $i)
                AUDIO=$(echo $GPU | sed -e "s/0$/1/")
                echo "vfio-pci" > $GPU/driver_override
                if [ -d $AUDIO ]; then
                        echo "vfio-pci" > $AUDIO/driver_override
                fi
        fi
done

modprobe -i vfio-pci

Arch Linux

#1 2016-02-23 18:03:15

PCI Passthrough with 2 identical GPUs?

#2 2016-02-23 18:54:46

Re: PCI Passthrough with 2 identical GPUs?

#3 2016-02-23 19:04:25

Re: PCI Passthrough with 2 identical GPUs?

#4 2016-02-23 19:22:30

Re: PCI Passthrough with 2 identical GPUs?

Board footer