You are not logged in.

Pages: 1 … 18 19 20 21 22 … 234

Topic closed

#476 2013-09-17 17:23:43

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I need help.

I use this script to start VM, what I need to change to add another one graphics card to VM?

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

Last edited by chUpa (2013-09-17 17:28:53)

Offline

#477 2013-09-17 23:01:05

SpacePirate
Member
Registered: 2013-09-16
Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

While reading the Thread I`ve got another question:

Is it possible to pass the GPU in the first PCI Slot to the Guest System? If not, is this a performance issue since many Mainboards only have PCIe x4 in the second slot...

Offline

#478 2013-09-18 05:30:11

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

SpacePirate wrote:

While reading the Thread I`ve got another question:

Is it possible to pass the GPU in the first PCI Slot to the Guest System? If not, is this a performance issue since many Mainboards only have PCIe x4 in the second slot...

I do not see any problem in this. In my motherboard, I have 6 pcie ports. My host using card that which is in the lower slot. Its address 0000:05:00.0. Passthrough cards have addresses 0000:01:00.0/0000:02:00.0.

Offline

#479 2013-09-18 12:18:29

SpacePirate
Member
Registered: 2013-09-16
Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:
SpacePirate wrote:

While reading the Thread I`ve got another question:

Is it possible to pass the GPU in the first PCI Slot to the Guest System? If not, is this a performance issue since many Mainboards only have PCIe x4 in the second slot...

I do not see any problem in this. In my motherboard, I have 6 pcie ports. My host using card that which is in the lower slot. Its address 0000:05:00.0. Passthrough cards have addresses 0000:01:00.0/0000:02:00.0.

Okay, thats nice, i think i just missunderstood some earlier post then.

Offline

#480 2013-09-18 14:24:29

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:

I need help.

I use this script to start VM, what I need to change to add another one graphics card to VM?

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

You could add a "GPU2", "GPU_SND2", and "VBIOS2" variable, then add this 

if [ -n "$GPU2" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS2" ]; then
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,rombar=0,romfile=$VBIOS2,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND2" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND2,addr=0.1,bus=root.2"
  fi
fi

Offline

#481 2013-09-18 14:25:39

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

SpacePirate wrote:
chUpa wrote:
SpacePirate wrote:

While reading the Thread I`ve got another question:

Is it possible to pass the GPU in the first PCI Slot to the Guest System? If not, is this a performance issue since many Mainboards only have PCIe x4 in the second slot...

I do not see any problem in this. In my motherboard, I have 6 pcie ports. My host using card that which is in the lower slot. Its address 0000:05:00.0. Passthrough cards have addresses 0000:01:00.0/0000:02:00.0.

Okay, thats nice, i think i just missunderstood some earlier post then.

As long as its not your primary card you could use any slot you want

Offline

#482 2013-09-18 15:21:44

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
chUpa wrote:

I need help.

I use this script to start VM, what I need to change to add another one graphics card to VM?

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

You could add a "GPU2", "GPU_SND2", and "VBIOS2" variable, then add this 

if [ -n "$GPU2" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS2" ]; then
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,rombar=0,romfile=$VBIOS2,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND2" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND2,addr=0.1,bus=root.2"
  fi
fi

I tried to do this, here is my edited script:

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
SND=true
SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES=""
GPU0="01:00.0"
GPU_SND0="01:00.1"
GPU1="02:00.0"
GPU_SND1="02:00.1"
VBIOS0="/home/cd/rom0.rom"
VBIOS1="/home/cd/rom1.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU0" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS0" ]; then
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,rombar=0,romfile=$VBIOS0,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND0" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND0,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$GPU1" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS1" ]; then
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,rombar=0,romfile=$VBIOS1,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND1" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND1,addr=0.1,bus=root.2"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

It starts without errors, but there is no output from the graphics card (from both I get - no signal detected).
When I comment the variables:

GPU1="02:00.0"
GPU_SND1="02:00.1"
VBIOS1="/home/cd/rom1.rom"

works as before

Spasibo for the help, I hope we will find a solution (:

Russia and Uruguay 27 December 2011 signed a visa-free status. I travel much. We can have a beer or coffee. ((:

Last edited by chUpa (2013-09-18 15:33:09)

Offline

#483 2013-09-18 15:42:28

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:
nbhs wrote:
chUpa wrote:

I need help.

I use this script to start VM, what I need to change to add another one graphics card to VM?

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

You could add a "GPU2", "GPU_SND2", and "VBIOS2" variable, then add this 

if [ -n "$GPU2" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS2" ]; then
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,rombar=0,romfile=$VBIOS2,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND2" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND2,addr=0.1,bus=root.2"
  fi
fi

I tried to do this, here is my edited script:

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
SND=true
SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES=""
GPU0="01:00.0"
GPU_SND0="01:00.1"
GPU1="02:00.0"
GPU_SND1="02:00.1"
VBIOS0="/home/cd/rom0.rom"
VBIOS1="/home/cd/rom1.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU0" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS0" ]; then
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,rombar=0,romfile=$VBIOS0,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND0" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND0,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$GPU1" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS1" ]; then
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,rombar=0,romfile=$VBIOS1,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND1" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND1,addr=0.1,bus=root.2"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

It starts without errors, but there is no output from the graphics card (from both I get - no signal detected).
When I comment the variables:

GPU1="02:00.0"
GPU_SND1="02:00.1"
VBIOS1="/home/cd/rom1.rom"

works as before

Spasibo for the help, I hope we will find a solution (:

Are you using radeon cards?

chUpa wrote:

Russia and Uruguay 27 December 2011 signed a visa-free status. I travel much. We can have a beer or coffee. ((:

Sure man anytime smile

Offline

#484 2013-09-18 15:55:09

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
chUpa wrote:
nbhs wrote:

You could add a "GPU2", "GPU_SND2", and "VBIOS2" variable, then add this 

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

I tried to do this, here is my edited script:

if [ -n "$GPU2" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS2" ]; then
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,rombar=0,romfile=$VBIOS2,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND2" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND2,addr=0.1,bus=root.2"
  fi
fi

It starts without errors, but there is no output from the graphics card (from both I get - no signal detected).
When I comment the variables:

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
SND=true
SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES=""
GPU0="01:00.0"
GPU_SND0="01:00.1"
GPU1="02:00.0"
GPU_SND1="02:00.1"
VBIOS0="/home/cd/rom0.rom"
VBIOS1="/home/cd/rom1.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU0" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS0" ]; then
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,rombar=0,romfile=$VBIOS0,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU0,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND0" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND0,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$GPU1" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS1" ]; then
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,rombar=0,romfile=$VBIOS1,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU1,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND1" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND1,addr=0.1,bus=root.2"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

works as before

Spasibo for the help, I hope we will find a solution (:

Are you using radeon cards?

chUpa wrote:

Russia and Uruguay 27 December 2011 signed a visa-free status. I travel much. We can have a beer or coffee. ((:

ATI Radeon (gigabyte HD 7970 gv-r7970c-3gd)

Sure man anytime smile

You have the summer there soon ... (:

Last edited by chUpa (2013-09-18 15:55:52)

Offline

#485 2013-09-18 16:00:18

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:
nbhs wrote:
chUpa wrote:

I tried to do this, here is my edited script:

#!/bin/sh
CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

CPUSET=/sys/fs/cgroup/cpuset
QEMU=/usr/bin/qemu-system-x86_64

VCPUS="6" #must be > 0 < cpu core number 
CONFIG="-M q35 -enable-kvm -nographic"
MEM="8192"
MEM_PATH=""
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
#NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
NET=""
SND=""
#SND=true
#SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
USB_DEVICES="tablet host:058f:6254 host:046d:c526 host:045e:0745"
PCI_DEVICES="06:06.0"
GPU="01:00.0"
GPU_SND="01:00.1"
VBIOS="/home/cd/rom.rom"
HDD="/home/cd/vhdd/win8x64.img"
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
  CONFIG+=" -vga none -nographic"
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
  if [ -n "$VBIOS" ]; then
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,rombar=0,romfile=$VBIOS,addr=0.0,multifunction=on,bus=root.1"
  else
    DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
  fi
  
  if [ -n "$GPU_SND" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
  fi
fi

if [ -n "$PCI_DEVICES" ]; then
  for d in $PCI_DEVICES; do
    DEV+=" -device vfio-pci,host=$d"
  done
fi 

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
  DEV+=" -device ahci,bus=pcie.0,id=ahci"
  if [ -n "$HDD" ]; then
    STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
  fi
  if [ -n "$CDROM" ]; then
    STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
  fi
fi

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$CPU" ]; then
  CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
  CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
  BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEM="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEM+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
  for u in $USB_DEVICES; do
    USB_DEV+=" -usbdevice $u"
  done
fi

if [ ! -d  $CPUSET/system ]; then
    mkdir $CPUSET/system
fi

if [ ! -d  $CPUSET/qemu ]; then
    mkdir $CPUSET/qemu
fi

echo $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1)) > $CPUSET/qemu/cpuset.cpus
echo 1 > $CPUSET/qemu/cpuset.cpu_exclusive
echo 0 > $CPUSET/qemu/cpuset.mems

echo 0-$((CPU_CORE_NR-VCPUS-1))> $CPUSET/system/cpuset.cpus
echo 1 > $CPUSET/system/cpuset.cpu_exclusive
echo 0 > $CPUSET/system/cpuset.mems
  
for t in `cat $CPUSET/tasks`; do 
  echo $t > $CPUSET/system/tasks 
done > /dev/null 2>&1

set -m

$QEMU $CONFIG $BIOS $CPU $MEM $DEV $USB_DEV $SND $NET $STORAGE $EXTRA_ARGS &

PID=$!

(
while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
    i=$((CPU_CORE_NR-VCPUS))
    for t in `ls /proc/$PID/task/`; do
      if [ $i -ne $((CPU_CORE_NR)) ]; then
	if [ $t -ne $PID ];then
	  echo $t > $CPUSET/qemu/tasks
	  taskset -pc $i $t; let i++
	fi
      fi
    done
    break
  fi
done
) &

fg %1

wait

for t in `cat $CPUSET/system/tasks`; do
  echo $t > $CPUSET/tasks 
done > /dev/null 2>&1

rmdir $CPUSET/system
rmdir $CPUSET/qemu

echo "All Done!"

It starts without errors, but there is no output from the graphics card (from both I get - no signal detected).
When I comment the variables:

if [ -n "$GPU2" ]; then
  DEV+=" -device ioh3420,bus=pcie.0,addr=1c.1,multifunction=on,port=2,chassis=2,id=root.2"
  if [ -n "$VBIOS2" ]; then
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,rombar=0,romfile=$VBIOS2,addr=0.0,multifunction=on,bus=root.2"
  else
    DEV+=" -device vfio-pci,host=$GPU2,x-vga=on,addr=0.0,multifunction=on,bus=root.2"
  fi
  
  if [ -n "$GPU_SND2" ]; then
    DEV+=" -device vfio-pci,host=$GPU_SND2,addr=0.1,bus=root.2"
  fi
fi

works as before

Spasibo for the help, I hope we will find a solution (:

Are you using radeon cards?

chUpa wrote:

Russia and Uruguay 27 December 2011 signed a visa-free status. I travel much. We can have a beer or coffee. ((:

ATI Radeon (gigabyte HD 7970 gv-r7970c-3gd)

Sure man anytime smile

You have the summer there soon ... (:

Try without "x-vga=on" on GPU1

Last edited by nbhs (2013-09-18 16:00:58)

Offline

#486 2013-09-18 16:05:02

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just uploaded qemu 1.6, kernel 3.11 and seabios 1.7.3 packages to the main guide, qemu 1.6 seems to require seabios 1.7.3

Offline

#487 2013-09-18 16:10:22

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

Try without "x-vga=on" on GPU1

still no signal

Offline

#488 2013-09-18 16:19:00

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:
nbhs wrote:

Try without "x-vga=on" on GPU1

still no signal

Im assuming its working ok with one card, but if you passthrough another card it doesnt work, right? Could you post your hardware specs? you can try to passthrough the secondary card as a regular pci device

-device vfio-pci,host=02:00.0

Offline

#489 2013-09-18 16:33:02

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, correctly. With one card Windows work fine, and i have to install Catalyst.

I have 2 radeon 01:00.0/1 and 02:00.0/1 and Nvidia 210 05:00.0/1 (open source driver), radeon and fglrx modules blacklisted.

I try add my second card in PCI_DEVICES="02:00.0" its not working, shows no signal too.

just in case, i was add my ati cards in pci-stub, and try. Nothing has changed.

lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port C)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port A)
00:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890S PCI Express bridge for GPP2 port 1
00:11.0 RAID bus controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [RAID5 mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 2)
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]
03:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
04:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller (rev 11)
05:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
05:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)
06:06.0 Multimedia audio controller: Creative Labs SB X-Fi
06:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 06)
08:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
09:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller (rev 11)
lspci -n
00:00.0 0600: 1002:5a14 (rev 02)
00:00.2 0806: 1002:5a23
00:02.0 0604: 1002:5a16
00:03.0 0604: 1002:5a17
00:09.0 0604: 1002:5a1c
00:0a.0 0604: 1002:5a1d
00:0c.0 0604: 1002:5a20
00:11.0 0104: 1002:4393 (rev 40)
00:12.0 0c03: 1002:4397
00:12.2 0c03: 1002:4396
00:13.0 0c03: 1002:4397
00:13.2 0c03: 1002:4396
00:14.0 0c05: 1002:4385 (rev 42)
00:14.2 0403: 1002:4383 (rev 40)
00:14.3 0601: 1002:439d (rev 40)
00:14.4 0604: 1002:4384 (rev 40)
00:14.5 0c03: 1002:4399
00:15.0 0604: 1002:43a0
00:15.1 0604: 1002:43a1
00:15.2 0604: 1002:43a2
00:16.0 0c03: 1002:4397
00:16.2 0c03: 1002:4396
00:18.0 0600: 1022:1600
00:18.1 0600: 1022:1601
00:18.2 0600: 1022:1602
00:18.3 0600: 1022:1603
00:18.4 0600: 1022:1604
00:18.5 0600: 1022:1605
01:00.0 0300: 1002:6798
01:00.1 0403: 1002:aaa0
02:00.0 0300: 1002:6798
02:00.1 0403: 1002:aaa0
03:00.0 0c03: 1b6f:7023 (rev 01)
04:00.0 0106: 1b4b:9172 (rev 11)
05:00.0 0300: 10de:0a65 (rev a2)
05:00.1 0403: 10de:0be3 (rev a1)
06:06.0 0401: 1102:0005
06:0e.0 0c00: 1106:3044 (rev c0)
07:00.0 0200: 10ec:8168 (rev 06)
08:00.0 0c03: 1b6f:7023 (rev 01)
09:00.0 0106: 1b4b:9172 (rev 11)

Last edited by chUpa (2013-09-18 16:38:23)

Offline

#490 2013-09-18 16:36:04

SpacePirate
Member
Registered: 2013-09-16
Posts: 55

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
SpacePirate wrote:
chUpa wrote:

I do not see any problem in this. In my motherboard, I have 6 pcie ports. My host using card that which is in the lower slot. Its address 0000:05:00.0. Passthrough cards have addresses 0000:01:00.0/0000:02:00.0.

Okay, thats nice, i think i just missunderstood some earlier post then.

As long as its not your primary card you could use any slot you want

When does a graphic card become my primary one?

Offline

#491 2013-09-18 16:39:08

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

SpacePirate wrote:
nbhs wrote:
SpacePirate wrote:

Okay, thats nice, i think i just missunderstood some earlier post then.

As long as its not your primary card you could use any slot you want

When does a graphic card become my primary one?

Its the one that you see the POST messages

Offline

#492 2013-09-18 16:41:29

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

SpacePirate wrote:
nbhs wrote:
SpacePirate wrote:

Okay, thats nice, i think i just missunderstood some earlier post then.

As long as its not your primary card you could use any slot you want

When does a graphic card become my primary one?

I think you should choose it in you BIOS. I boot from PEG3 card.

Offline

#493 2013-09-18 16:44:57

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:

Yes, correctly. With one card Windows work fine, and i have to install Catalyst.

I have 2 radeon 01:00.0/1 and 02:00.0/1 and Nvidia 210 05:00.0/1 (open source driver), radeon and fglrx modules blacklisted.

I try add my second card in PCI_DEVICES="02:00.0" its not working, shows no signal too.

just in case, i was add my ati cards in pci-stub, and try. Nothing has changed.

lspci
00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
00:03.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port C)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
00:0a.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx1 port A)
00:0c.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890S PCI Express bridge for GPP2 port 1
00:11.0 RAID bus controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [RAID5 mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:15.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0)
00:15.1 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1)
00:15.2 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB900 PCI to PCI bridge (PCIE port 2)
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT [Radeon HD 7970]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]
03:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
04:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller (rev 11)
05:00.0 VGA compatible controller: NVIDIA Corporation GT218 [GeForce 210] (rev a2)
05:00.1 Audio device: NVIDIA Corporation High Definition Audio Controller (rev a1)
06:06.0 Multimedia audio controller: Creative Labs SB X-Fi
06:0e.0 FireWire (IEEE 1394): VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller (rev c0)
07:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller (rev 06)
08:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
09:00.0 SATA controller: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller (rev 11)
lspci -n
00:00.0 0600: 1002:5a14 (rev 02)
00:00.2 0806: 1002:5a23
00:02.0 0604: 1002:5a16
00:03.0 0604: 1002:5a17
00:09.0 0604: 1002:5a1c
00:0a.0 0604: 1002:5a1d
00:0c.0 0604: 1002:5a20
00:11.0 0104: 1002:4393 (rev 40)
00:12.0 0c03: 1002:4397
00:12.2 0c03: 1002:4396
00:13.0 0c03: 1002:4397
00:13.2 0c03: 1002:4396
00:14.0 0c05: 1002:4385 (rev 42)
00:14.2 0403: 1002:4383 (rev 40)
00:14.3 0601: 1002:439d (rev 40)
00:14.4 0604: 1002:4384 (rev 40)
00:14.5 0c03: 1002:4399
00:15.0 0604: 1002:43a0
00:15.1 0604: 1002:43a1
00:15.2 0604: 1002:43a2
00:16.0 0c03: 1002:4397
00:16.2 0c03: 1002:4396
00:18.0 0600: 1022:1600
00:18.1 0600: 1022:1601
00:18.2 0600: 1022:1602
00:18.3 0600: 1022:1603
00:18.4 0600: 1022:1604
00:18.5 0600: 1022:1605
01:00.0 0300: 1002:6798
01:00.1 0403: 1002:aaa0
02:00.0 0300: 1002:6798
02:00.1 0403: 1002:aaa0
03:00.0 0c03: 1b6f:7023 (rev 01)
04:00.0 0106: 1b4b:9172 (rev 11)
05:00.0 0300: 10de:0a65 (rev a2)
05:00.1 0403: 10de:0be3 (rev a1)
06:06.0 0401: 1102:0005
06:0e.0 0c00: 1106:3044 (rev c0)
07:00.0 0200: 10ec:8168 (rev 06)
08:00.0 0c03: 1b6f:7023 (rev 01)
09:00.0 0106: 1b4b:9172 (rev 11)

Are you sure the nvidia card is your primary card? have you tried to comment out the VBIOS variable?

Offline

#494 2013-09-18 16:56:38

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

Are you sure the nvidia card is your primary card? have you tried to comment out the VBIOS variable?

I draw my conclusions from the fact that the bios and arch boot and work from my nvidia card, even if I turn off second HDMI cable from my PC.

when I did like this:

#GPU1="02:00.0"
#GPU_SND1="02:00.1"
#VBIOS0="/home/cd/rom0.rom"
#VBIOS1="/home/cd/rom1.rom"

all working fine. The only difference is fan in my ati card is worked louder until not windows boot.

Last edited by chUpa (2013-09-18 17:06:05)

Offline

#495 2013-09-18 17:48:13

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

and when i try this:

PCI_DEVICES="02:00.0 02:00.1"
GPU0="01:00.0"
GPU_SND0="01:00.1"
#GPU1="02:00.0"
#GPU_SND1="02:00.1"
VBIOS0="/home/cd/rom0.rom" (or #VBIOS0="/home/cd/rom0.rom")
#VBIOS1="/home/cd/rom1.rom"

I have no effect. No signal as global result. (:
But my credo never give up, hold on to the result! (;

Last edited by chUpa (2013-09-18 17:51:08)

Offline

#496 2013-09-18 18:08:48

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

chUpa wrote:

and when i try this:

PCI_DEVICES="02:00.0 02:00.1"
GPU0="01:00.0"
GPU_SND0="01:00.1"
#GPU1="02:00.0"
#GPU_SND1="02:00.1"
VBIOS0="/home/cd/rom0.rom" (or #VBIOS0="/home/cd/rom0.rom")
#VBIOS1="/home/cd/rom1.rom"

I have no effect. No signal as global result. (:
But my credo never give up, hold on to the result! (;

comment out the VBIOS line

Offline

#497 2013-09-18 18:18:45

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

comment out the VBIOS line


Tried both variants

Offline

#498 2013-09-19 08:17:05

chUpa
Member
From: Chelyabinsk, Russia
Registered: 2013-09-17
Posts: 13

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Today has updated the system. Now i have

Linux barby 3.11.1-1-ARCH #1 SMP PREEMPT Sat Sep 14 19:30:21 CEST 2013 x86_64 GNU/Linux

and

qemu-git 1.7.r28759.g6c2679f-1
cat /proc/cmdline BOOT_IMAGE=/vmlinuz-linux root=UUID=df9ed763-fa79-41fd-8314-ca4a108ee274 rw pci-stub.ids=1002:6798,1002:aaa0,1102:0005
ls /etc/modprobe.d/
blacklist.conf kvm.conf vfio_iommu_type1.conf
cat /etc/modprobe.d/blacklist.conf 
blacklist radeon
blacklist fglrx
cat /etc/modprobe.d/kvm.conf 
options kvm ignore_msrs=1
cat /etc/modprobe.d/vfio_iommu_type1.conf 
options vfio_iommu_type1 allow_unsafe_interrupts=1
ls /etc/modules-load.d/
kvm_amd.conf  kvm.conf vfio.conf  vfio_iommu_type1.conf vfio_pci.conf
cat /etc/modules-load.d/kvm_amd.conf
kvm_amd
cat /etc/modules-load.d/kvm.conf
kvm
 
cat /etc/modules-load.d/vfio.conf 
vfio
cat /etc/modules-load.d/vfio_iommu_type1.conf 
vfio_iommu_type1
cat /etc/modules-load.d/vfio_pci.conf 
vfio_pci

Without any patches. Nothing has changed. Only one card working. When I try a second, I have no signal.

Last edited by chUpa (2013-09-19 08:18:45)

Offline

#499 2013-09-19 19:44:32

gosheto
Member
Registered: 2013-09-04
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,

i`m now here:
FluxBB bbcode test

Second monitor, connected to passthrough VGA - no output (black screen).

Boot guest using this script:

/usr/bin/qemu-system-x86_64 \
-enable-kvm \
-M q35 \
-m 4096 \
-cpu host \
-bios /home/gosheto/Downloads/bios.bin \
-vga cirrus \
-smp 2,sockets=1,cores=2,threads=1 \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=05:00.0,bus=root.1,addr=00.0,multifunction=on,romfile=/home/gosheto/Downloads/Palit.GTX580.1536.101102.rom \
-device vfio-pci,host=05:00.1,bus=root.1,addr=00.1 \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/home/gosheto/VM/windowsq35.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk \
-drive file=/home/gosheto/VM/windows7.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-boot menu=on

If using
-vga none and x-vga=on get black QEMU windows and system freeze after 10 seconds (1 of cpu get 100% load).

In dmesg get this messages:

vfio-pci 0000:05:00.0: enabling device (0000 -> 0003)
vfio_ecap_init: 0000:05:00.1 hiding ecap 0x0@0x100
kvm: zapping shadow pages for mmio generation wraparound
assign device 0:5:0.0
assign device 0:5:0.1
etc

Any ideas?

PS: 0000:05:00.1 (NVIDIA HD Audio working properly)

Thank you in advance!

Last edited by gosheto (2013-09-19 19:46:42)

Offline

#500 2013-09-19 19:52:38

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

gosheto wrote:

Hello,

i`m now here:
http://s15.postimg.org/ad6dnw2zv/Selection_001.png

Second monitor, connected to passthrough VGA - no output (black screen).

Boot guest using this script:

/usr/bin/qemu-system-x86_64 \
-enable-kvm \
-M q35 \
-m 4096 \
-cpu host \
-bios /home/gosheto/Downloads/bios.bin \
-vga cirrus \
-smp 2,sockets=1,cores=2,threads=1 \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=05:00.0,bus=root.1,addr=00.0,multifunction=on,romfile=/home/gosheto/Downloads/Palit.GTX580.1536.101102.rom \
-device vfio-pci,host=05:00.1,bus=root.1,addr=00.1 \
-device ahci,bus=pcie.0,id=ahci \
-drive file=/home/gosheto/VM/windowsq35.img,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk \
-drive file=/home/gosheto/VM/windows7.iso,id=isocd -device ide-cd,bus=ahci.1,drive=isocd \
-boot menu=on

If using
-vga none and x-vga=on get black QEMU windows and system freeze after 10 seconds (1 of cpu get 100% load).

In dmesg get this messages:

vfio-pci 0000:05:00.0: enabling device (0000 -> 0003)
vfio_ecap_init: 0000:05:00.1 hiding ecap 0x0@0x100
kvm: zapping shadow pages for mmio generation wraparound
assign device 0:5:0.0
assign device 0:5:0.1
etc

Any ideas?

PS: 0000:05:00.1 (NVIDIA HD Audio working properly)

Thank you in advance!

Never seen that message before what kernel/qemu version are you using?

Offline

Pages: 1 … 18 19 20 21 22 … 234

Topic closed

Board footer

Powered by FluxBB