KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs · 2013-05-22 03:27:31

@ Powerhouse

Thanks a lot man!

@ Jgott

Great!

Audio: I managed to fix all the audio scraching/stutter by disabling nested pages and rebuilding my kernel ( see my second post on first page about performance numbers ), with audio passthrough using QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128, and i also had a problem with my headset but its fixed now with the new kernel config.

DISK: To tell you the truth i havent even tried virtio-blk since i just pass my second sata controller instead of using the emulated one, so it boots directly from the connected disk.

I also had some timing issues playing bioshock infinite, i solved it by following this guide: https://access.redhat.com/site/document … ement.html

EDIT: BTW im using cpuset cgroup and taskset to pin the 6 qemu threads to each vcpu, so in total there's 7 threads + 1 if audio passthrough is enabled, the main qemu thread + audio thread is running on cpus [0-1] while the vcpus threads run on cpus [2-7]

Last edited by nbhs (2013-05-22 04:46:13)

jgott · 2013-05-22 03:42:24

nbhs wrote:

I managed to fix all the audio scraching/stutter by disabling nested pages and rebuilding my kernel ( see my second post on first page about performance numbers )

Interesting that nested pages makes audio performance worse; I never would have guessed. I'll have to give this a try.

nbhs wrote:

EDIT: BTW im using cpuset cgroup and taskset to pin the 6 qemu threads to each vcpu, so in total there's 7 threads + 1 if audio passthrough is enabled, the main qemu thread + audio thread is running on cpus [0-1] while the vcpus threads run on cpus [2-7]

I wasn't setting CPU affinity before, but suddenly I can see why this matters, since qemu emulated devices (e.g. audio) are competing with the VM guest for CPU time. I'll have to try this, too.

Thanks for the tips. I should have looked at the performance tuning parts sooner.

nbhs · 2013-05-22 03:51:29

I sugest you use cgroups and move all the (movable) host threads to a couple of cores, and dedicate the rest to the vm, that way, even if your building a kernel on the host it wont interfere with the vm, you can create the cgroups manually or you can also use cset

https://code.google.com/p/cpuset/
https://rt.wiki.kernel.org/index.php/Cp … y/tutorial

You might need to patch the lastest version of cset (1.5.6)

EDIT: Patch here:

cset.py.patch

My VM script using CSET:

#!/bin/sh

CPU_CORE_NR=$(cat /proc/cpuinfo | egrep "core id|physical id" | tr -d "\n" | sed s/physical/\\nphysical/g | grep -v ^$ | sort | uniq | wc -l)

QEMU=/usr/bin/qemu-system-x86_64
CSET=/usr/bin/cset

VCPUS="6" #must be > 0 < cpu core number
CONFIG="-M q35 -enable-kvm"
MEM="8192"
MEM_PATH="/dev/hugepages"
CPU="host"
BIOS="/usr/share/qemu/bios.bin"
NET="-netdev bridge,br=br0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0"
USB_DEVICES="tablet"
PCI_DEVICES="00:11.0 04:00.0 05:00.0 06:00.0"
SND=true
SND_DRIVER_OPTS="QEMU_AUDIO_DRV=pa QEMU_PA_SAMPLES=128"
GPU="07:00.0"
GPU_SND="07:00.1"
VBIOS="/usr/local/share/qemu/vgabios-6950.bin"
HDD=""
CDROM=""
EXTRA_ARGS="-boot menu=on -monitor stdio"

if [ -n "$GPU" ]; then
	CONFIG+=" -vga none -nographic"
	DEV+=" -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1"
	if [ -n "$VBIOS" ]; then
		DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1,rombar=0,romfile=$VBIOS"
	else
		DEV+=" -device vfio-pci,host=$GPU,x-vga=on,addr=0.0,multifunction=on,bus=root.1"
	fi
  
	if [ -n "$GPU_SND" ]; then
		DEV+=" -device vfio-pci,host=$GPU_SND,addr=0.1,bus=root.1"
	fi
fi

if [ -n "$PCI_DEVICES" ]; then
	for d in $PCI_DEVICES; do
		DEV+=" -device vfio-pci,host=$d"
	done
fi 

if $SND ; then
  SND="-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 -device hda-duplex,cad=0"
  export $SND_DRIVER_OPTS
else
  SND=""
fi

if [ -n "$HDD" ] || [ -n "$CDROM" ]; then
	DEV+="  -device ahci,bus=pcie.0,id=ahci"
	if [ -n "$HDD" ]; then
		STORAGE+=" -drive file=$HDD,id=disk,format=raw -device ide-hd,bus=ahci.0,drive=disk"
	fi
	if [ -n "$CDROM" ]; then
		STORAGE+=" -drive file=$CDROM,id=isocd -device ide-cd,bus=ahci.1,drive=isocd"
	fi
fi

if [ -n "$CPU" ]; then
	CPU="-cpu $CPU -smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
else
	CPU="-smp $VCPUS,sockets=1,cores=$VCPUS,threads=1"
fi

if [ -n "$BIOS" ]; then
	BIOS=" -bios $BIOS"
fi

if [ -n "$MEM" ]; then
  MEMORY+="-m $MEM"
  if [ -n "$MEM_PATH" ]; then
    MEMORY+=" -mem-path $MEM_PATH"
  fi
else
  if [ -n "$MEM_PATH" ]; then
    MEMORY+=" -mem-path $MEM_PATH"
  fi
fi

if [ -n "$USB_DEVICES" ]; then
	for u in $USB_DEVICES; do
		USB_DEV+=" -usbdevice $u"
	done
fi

sync && echo 1 > /proc/sys/vm/drop_caches &
wait

HUGEPAGES_NR="$(($MEM/$(($(grep Hugepagesize /proc/meminfo | awk '{print $2}')/1024))))"

echo $HUGEPAGES_NR > /proc/sys/vm/nr_hugepages

if [ $HUGEPAGES_NR -eq $(cat /proc/sys/vm/nr_hugepages) ]; then

	$CSET shield -c $((CPU_CORE_NR-VCPUS))-$((CPU_CORE_NR-1))

	set -m
	
	$QEMU $CONFIG $BIOS $CPU $MEMORY $DEV $USB_DEV $NET $SND $STORAGE $EXTRA_ARGS   &

	PID=$!

	(
	while ( (kill -s 0 $PID > /dev/null 2>&1) ); do
	  if [ "$(grep Threads /proc/$PID/status | awk '{print $2}' )" -ge "$((VCPUS+1))" > /dev/null 2>&1 ]; then
		i=$((CPU_CORE_NR-VCPUS))
		for t in `ls /proc/$PID/task/`; do
		  if [ $i -ne $((CPU_CORE_NR)) ]; then
			if [ $t -ne $PID ];then
			  $CSET shield --shield --pid $t
			  taskset -pc $i $t
			  let i++
			fi
		  fi
		done
		break
	  fi
	done
	) &

	fg %1

	wait

	$CSET shield --reset
	echo 0 > /proc/sys/vm/nr_hugepages
	echo "All Done!!"
else
  echo "Cannot reserve memory!"
fi

Last edited by nbhs (2013-05-22 04:27:31)

GreatEmerald · 2013-05-22 14:53:30

I attempted to get this working as well. I have an integrated Intel HD Graphics 2500 (for the host) and a dedicated NVIDIA GeForce GTX 660 (for the guest).

I installed QEMU 1.5, kernel 3.9 (the VFIO components are modularised) and set intel_iommu=on in the boot options. Then I disabled everything that used NVIDIA (blacklisted, set libGL and libglx to point to Mesa, removed the paths from ldconfig.d, set xorg.conf.d to load the intel drivers). Then executed this:

echo 1 | sudo tee /sys/module/vfio_iommu_type1/parameters/allow_unsafe_interrupts

Then used your script to bind the two parts of the graphics card to vfio. This had an effect:

[ 5517.777364] VFIO - User Level meta-driver version: 0.3
[ 5692.736558] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO0: (unknown bits 0x00008000)
[ 5692.736567] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO0: ch 1 [DRM] subc 0 mthd 0x0000 data 0x00000000
[ 5695.738853] nouveau E[     DRM] failed to idle channel 0xcccc0000 [DRM]
[ 5695.741961] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO2: (unknown bits 0x00008000)
[ 5695.741965] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO2: ch 0 [DRM] subc 0 mthd 0x0000 data 0x00000000
[ 5698.744410] nouveau E[     DRM] failed to idle channel 0xcccc0001 [DRM]
[ 5698.748200] [TTM] Finalizing pool allocator
[ 5698.748203] [TTM] Finalizing DMA pool allocator
[ 5698.748217] [TTM] Zone  kernel: Used memory at exit: 0 kiB
[ 5698.748219] [TTM] Zone   dma32: Used memory at exit: 0 kiB

I also added the systemd unit file and enabled it, but haven't restarted just yet.

Finally, I executed this:

pkexec qemu-system-x86_64 --enable-kvm -M q35 -m 1024 -cpu host -smp 2,sockets=1,cores=2,threads=1 -vga none -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 -bios /home/dainius/seabios/out/bios.bin

The reason why it's using pkexec is that using sudo makes it fail due to missing display, kdesu eats all the stdout/stderr (even when passed the -t option), and qemu-system seemingly ignores the -D option, which leaves me with only pkexec that can actually give me a GUI and show debugging information at the same time (that required me to fiddle with policy settings, since you need to explicitly add a policy that enables org.freedesktop.policykit.exec.allow_gui for each GUI application you want to pkexec).

The output of the command is:

Warning: default mac address being used, creating potential for address conflict
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Warning, device 0000:01:00.0 does not support reset
qemu-system-x86_64: -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1: Warning, device 0000:01:00.1 does not support reset

And dmesg gives me:

[11151.655299] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[11151.717650] vfio_ecap_init: 0000:01:00.1 hiding ecap 0x0@0x100

I get the QEMU window, showing "This VM has no graphic display device". When I connect my monitor to the GTX 660's DVI ports, I see no output at all (the screen goes into sleep mode).

So, I'm out of ideas right now. Maybe I should give pci-assign a try as well, although I'm not yet sure how it works.

I'm also not entirely sure how the whole passthrough is supposed to work as well. Does SeaBIOS see my graphics card as something it needs to initialise and use, so in essence the graphics card is supposed to think that it's being initialised in a clean boot process? But doesn't it generally need something to be connected to the card during that process, so that it could get EDID data and such? I have only one monitor here, so the only way to do so would be to connect the monitor before launching the virtual machine...

alphaniner · 2013-05-22 16:32:06

Can you elaborate on exactly what you're doing with audio? In the first post you PCI-passthrough both your video card and the associated audio device. But you also mention

AUDIO PASSTHROUGH:

-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

Isn't the second method emulation rather than passthrough?

And in any case, is the second method what you're talking about wrt cpu pinning in post 51?

nbhs · 2013-05-22 20:36:48

GreatEmerald wrote:

Then used your script to bind the two parts of the graphics card to vfio. This had an effect:

[ 5517.777364] VFIO - User Level meta-driver version: 0.3
[ 5692.736558] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO0: (unknown bits 0x00008000)
[ 5692.736567] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO0: ch 1 [DRM] subc 0 mthd 0x0000 data 0x00000000
[ 5695.738853] nouveau E[     DRM] failed to idle channel 0xcccc0000 [DRM]
[ 5695.741961] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO2: (unknown bits 0x00008000)
[ 5695.741965] nouveau E[   PFIFO][0000:01:00.0] SUBFIFO2: ch 0 [DRM] subc 0 mthd 0x0000 data 0x00000000
[ 5698.744410] nouveau E[     DRM] failed to idle channel 0xcccc0001 [DRM]
[ 5698.748200] [TTM] Finalizing pool allocator
[ 5698.748203] [TTM] Finalizing DMA pool allocator
[ 5698.748217] [TTM] Zone  kernel: Used memory at exit: 0 kiB
[ 5698.748219] [TTM] Zone   dma32: Used memory at exit: 0 kiB

You need to block nouveau modules as well

Also try this:

modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

Also make sure seabios is patched as described on my guide, or use the seabios-git package i provided.

Other users reported it not working when using the IGP as primary device, you might want to try with 2 discrete cards and IGP off.

EDIT: This guy got some 660 cards working,unfortunately his page is down so i'll link webcache http://webcache.googleusercontent.com/s … e&ie=UTF-8

Last edited by nbhs (2013-05-22 21:00:42)

nbhs · 2013-05-22 20:44:20

alphaniner wrote:

Can you elaborate on exactly what you're doing with audio? In the first post you PCI-passthrough both your video card and the associated audio device. But you also mention
AUDIO PASSTHROUGH:
-device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0
Isn't the second method emulation rather than passthrough?
And in any case, is the second method what you're talking about wrt cpu pinning in post 51?

Yes this is audio emulation, i called it passthrough because it pass the audio from the vm to the host, it creates an emulated sound card inside the vm, but im also using a usb headset connected to my usb3 ports which are passed thru to my vm.

Last edited by nbhs (2013-05-22 21:40:46)

GreatEmerald · 2013-05-22 21:01:32

nbhs wrote:

You need to block nouveau modules as well
Also try this:
modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0
Also make sure seabios is patched as described on my guide, or use the seabios-git package i provided.
Other users reported it not working when using the IGP as primary device, you might want to try with 2 discrete cards and IGP off.

Well, nouveau never was loaded. It's not on lsmod. So I don't know why it's mentioned in dmesg. Just to make sure, I added it to blacklist as well, but I'll have to restart to try it again.

And yes, I did load kvm_intel like that, I just forgot to mention it.

As for SeaBIOS, the patch actually isn't needed, because it's been merged upstream already: http://code.coreboot.org/p/seabios/sour … 97fbab821/ And yes, I am using SeaBIOS built from from git.

As for two dedicated cards, unfortunately I don't have such a luxury... I could try reversing the cards, so that the host uses NVIDIA and the guest uses Intel, but that's about it.

nbhs · 2013-05-22 21:03:12

It hasnt been merged here git.qemu.org/seabios.git, which is the one we need to use, so please use my package.

GreatEmerald · 2013-05-22 21:52:52

nbhs wrote:

It hasnt been merged here git.qemu.org/seabios.git, which is the one we need to use, so please use my package.

The QEMU SeaBIOS repository is an exact clone of the upstream SeaBIOS repository, just lagging behind (you can diff the git logs to make sure). So you can literally get the same result by fetching upstream and then doing a `git reset --hard f465e1ec1b01826100c92b890487a2ab09bfe2c1`. So unless the new commits actually break QEMU (unlikely, given that it hasn't broken it in a single commit up to now, unless upstream git is prone to breaking before it gets fixed up down the road), there really is no reason not to use the upstream SeaBIOS repository.

I suppose I can reset and cherry-pick the needed commit if you are so inclined. I just highly doubt there will be any difference.

nbhs · 2013-05-22 21:59:52

GreatEmerald wrote:

nbhs wrote:
It hasnt been merged here git.qemu.org/seabios.git, which is the one we need to use, so please use my package.
The QEMU SeaBIOS repository is an exact clone of the upstream SeaBIOS repository, just lagging behind (you can diff the git logs to make sure). So you can literally get the same result by fetching upstream and then doing a `git reset --hard f465e1ec1b01826100c92b890487a2ab09bfe2c1`. So unless the new commits actually break QEMU (unlikely, given that it hasn't broken it in a single commit up to now, unless upstream git is prone to breaking before it gets fixed up down the road), there really is no reason not to use the upstream SeaBIOS repository.
I suppose I can reset and cherry-pick the needed commit if you are so inclined. I just highly doubt there will be any difference.

Yes i know, thing is upstream seabios does break my system, i cant boot windows with it, you should be able to get output on your card tho, but that might be broken too.

Last edited by nbhs (2013-05-22 22:02:34)

Kamek · 2013-05-24 00:46:06

Is there a way to share a single GPU between the host and the guest ? I only have one GTX 670 in my rig, but I'm sick and tired of rebooting every time I want to play a game.

nbhs · 2013-05-24 00:59:03

Kamek wrote:

Is there a way to share a single GPU between the host and the guest ? I only have one GTX 670 in my rig, but I'm sick and tired of rebooting every time I want to play a game.

Not using this method, if you had a radeon card you could use kvm pci-assign and pass it as secondary but on NV cards this is the only reliable method (unless you're willing to try xen, and its counteless unreliable hacks)

darkspider · 2013-05-24 18:37:10

Hey nbhs,
I followed your guide and got everything running, but I have an issue with the pci-passthrough of an USB Controller. My board has an Etron USB 3.0 controller I never use, so I wanted to pass it to my Win7 guest.
Here is the output of lspci:

03:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
05:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)

So I use "vfio-bind 0000:03:00.0 0000:05:00.0" and add "-device vfio-pci,host=03:00.0,bus=pcie.0 -device vfio-pci,host=05:00.0,bus=pcie.0" to the start command of qemu and plug my mouse and keyboard into the 2 USB 3.0 ports.
But both aren't even powered (Razer-logo doesn't fade in and out and keyboard keys aren't lit) and the guest doesn't recognize them, same happens if I use any other USB Controller that lspci spits out. The mainboard I am using is an Gigabyte 990XA-UD3, which seems to have a quite good implementation of IOMMU. So, if I pass my mouse or my keyboard using the usb-function of qemu, it works, but of course it isn't available for the host anymore and I'd like to use a KVM switch to switch between the guest and the host.
Any suggestions on how I can fix the problem?

nbhs · 2013-05-24 21:20:30

darkspider wrote:

Hey nbhs,
I followed your guide and got everything running, but I have an issue with the pci-passthrough of an USB Controller. My board has an Etron USB 3.0 controller I never use, so I wanted to pass it to my Win7 guest.
Here is the output of lspci:
03:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
05:00.0 USB controller: Etron Technology, Inc. EJ168 USB 3.0 Host Controller (rev 01)
So I use "vfio-bind 0000:03:00.0 0000:05:00.0" and add "-device vfio-pci,host=03:00.0,bus=pcie.0 -device vfio-pci,host=05:00.0,bus=pcie.0" to the start command of qemu and plug my mouse and keyboard into the 2 USB 3.0 ports.
But both aren't even powered (Razer-logo doesn't fade in and out and keyboard keys aren't lit) and the guest doesn't recognize them, same happens if I use any other USB Controller that lspci spits out. The mainboard I am using is an Gigabyte 990XA-UD3, which seems to have a quite good implementation of IOMMU. So, if I pass my mouse or my keyboard using the usb-function of qemu, it works, but of course it isn't available for the host anymore and I'd like to use a KVM switch to switch between the guest and the host.
Any suggestions on how I can fix the problem?

Great man what card did you manage to pass through?

Make sure every device on /sys/bus/pci/drivers/vfio-pci/0000\:03\:00.0/iommu_group/devices/ and /sys/bus/pci/drivers/vfio-pci/0000\:05\:00.0/iommu_group/devices/ are passed through

Or can try passing through usb2 ports, but they normally come in pairs, so you have to pass both for example:

lspci

00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller

You might have to pass it through as a single multifunction device.

Last edited by nbhs (2013-05-24 21:27:05)

darkspider · 2013-05-24 21:45:53

Thank your for your quick reply and your great tutorial!
I managed to pass through a Nvidia GeForce GTX 460 while the host is running on an old Quadro FX 1700 (nouveau) I had laying around.
I saw a YouTube Video by Ubisoft, who benchmarked a 2 VM-system running on 2 GTX 460s with the only difference that they were using XEN. So I thought I'd try this out, but XEN didn't really seem to be what I was looking for and then I found your guide and yeah, it worked (except the USB part, which I'll try later) .

So, let's come to the USB part:
I tried your second example before in which I exactly did what you described, but it would turn out the same way as my USB3 controller approach. Only thing I didn't try is to "pass it through as a single multifunction device". Can you please explain how this works?
Thank you

Edit: Just remembered this: your systemd service contains an error (or typo):

ExecStart=-/usr/bin/vfio-pci $DEVICES

should say vfio-bind, because the script in /usr/bin is named vfio-bind in your tutorial

Last edited by darkspider (2013-05-24 21:53:33)

nbhs · 2013-05-24 21:53:35

darkspider wrote:

Thank your for your quick reply and your great tutorial!
I managed to pass through a Nvidia GeForce GTX 460 while the host is running on an old Quadro FX 1700 (nouveau) I had laying around.
I saw a YouTube Video by Ubisoft, who benchmarked a 2 VM-system running on 2 GTX 460s with the only difference that they were using XEN. So I thought I'd try this out, but XEN didn't really seem to be what I was looking for and then I found your guide and yeah, it worked (except the USB part, which I'll try later) .
So, let's come to the USB part:
I tried your second example before in which I exactly did what you described, but it would turn out the same way as my USB3 controller approach. Only thing I didn't try is to "pass it through as a single multifunction device". Can you please explain how this works?
Thank you

Yeah for example for my usb 2 ports i would do something like this:

qemu-system-x86_64 -device vfio-pci,host=00:13.0,multifunction=on,addr=13.0 \
 -device vfio-pci,host=00:13.2,addr=13.2

EDIT: Thanks didnt notice the typo ill correct it.

Last edited by nbhs (2013-05-24 21:58:46)

darkspider · 2013-05-24 22:08:14

Typing this from my Win7 VM, pass through of USB2 ports works, the Etron controller doesn't want to work. "multifunction" won't do the trick for it, but it works like a charm for a pair of USB2 ports. Thanks a lot!
My last step is now getting audio to work (haven't tried it before, I was happy enough to see something on my screen )

nbhs · 2013-05-24 22:14:34

Perhaps this "etron" controller needs an option rom to work, if you manage to dump it/find it somewhere (its most likely included in your mobo bios, you could extract it from there) you can pass it to the vm with

-option-rom /path/to/rom

Audio is painful, see my second post on first page on how to optimize for my amd cpu and also my first post in this page, i had to disable nested pages even for a usb headset to work without stuttering/scratchy sound.

Last edited by nbhs (2013-05-24 22:23:29)

darkspider · 2013-05-24 22:57:42

Guess what? Audio's working perfectly with

 -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

and

QEMU_PA_SAMPLES=128 QEMU_AUDIO_DRV=pa qemu-system-x86_64...

just like how you mentioned in your first post, no stuttering, no scratching, everything is perfectly fine !

I have extracted a recent version of my BIOS, but I can't find any files that may be related to the Etron USB 3.0 Controller (there are some "GROUP ROM"s, like tgroup.bin, t1group.bin up to "t4" and ggroup, ffgroup) and I honestly never did something like this (extracting a BIOS) before, also Google doesn't deliver anything about the controller's ROM (just many forum entries about issues with this controller in normal use i.e. driver issues)

nbhs · 2013-05-24 23:04:35

darkspider wrote:

Guess what? Audio's working perfectly with
 -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0  
and
QEMU_PA_SAMPLES=128 QEMU_AUDIO_DRV=pa qemu-system-x86_64...
just like how you mentioned in your first post, no stuttering, no scratching, everything is perfectly fine !
I have extracted a recent version of my BIOS, but I can't find any files that may be related to the Etron USB 3.0 Controller (there are some "GROUP ROM"s, like tgroup.bin, t1group.bin up to "t4" and ggroup, ffgroup) and I honestly never did something like this (extracting a BIOS) before, also Google doesn't deliver anything about the controller's ROM (just many forum entries about issues with this controller in normal use i.e. driver issues)

Well if you encounter, audio problems playing games, take a look at my second post on first page, like you i had no problems with audio, until i started playing metro last light, and then all kind of problems started to arrise, like scratchy audio, stuttering in sound and fps, i solved all that doing the steps i mentioned, and rebuilding my kernel with the config file i attached

GreatEmerald · 2013-05-25 13:29:22

So I tried a few other things, like blacklisting nouveau, setting pci_stub to claim the graphics card and cherry-picking SeaBIOS. Still no go. I get this in dmesg:

[19144.183320] VFIO - User Level meta-driver version: 0.3
[19433.567127] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[19433.567278] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[19433.641853] vfio-pci 0000:01:00.1: enabling device (0000 -> 0002)
[19433.641964] vfio_ecap_init: 0000:01:00.1 hiding ecap 0x0@0x100

And this from the console:

qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Warning, device 0000:01:00.0 does not support reset
qemu-system-x86_64: -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1: Warning, device 0000:01:00.1 does not support reset

I also tried the commands suggested at the FrankD page. But it resulted in QEMU eating one core or outright dying with this message followed by a backtrace:

KVM internal error. Suberror: 1
emulation failure

The card still shows no output either way.

nbhs · 2013-05-25 16:44:10

GreatEmerald wrote:

So I tried a few other things, like blacklisting nouveau, setting pci_stub to claim the graphics card and cherry-picking SeaBIOS. Still no go. I get this in dmesg:
[19144.183320] VFIO - User Level meta-driver version: 0.3
[19433.567127] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[19433.567278] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x900
[19433.641853] vfio-pci 0000:01:00.1: enabling device (0000 -> 0002)
[19433.641964] vfio_ecap_init: 0000:01:00.1 hiding ecap 0x0@0x100
And this from the console:
qemu-system-x86_64: -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on: Warning, device 0000:01:00.0 does not support reset
qemu-system-x86_64: -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1: Warning, device 0000:01:00.1 does not support reset
I also tried the commands suggested at the FrankD page. But it resulted in QEMU eating one core or outright dying with this message followed by a backtrace:
KVM internal error. Suberror: 1
emulation failure
The card still shows no output either way.

You should fill a bug report @qemu-devel see: http://lists.gnu.org/archive/html/qemu- … 00432.html

darkspider · 2013-05-25 17:05:32

nbhs, thanks for your help, you're very kind.
Could you elaborate on your kernel config? What's the purpose of using a 1000HZ timer frequency and a voluntary Preemption Model? I've seen some posts of people, who recommend this settings for a Desktop setup, because it reduces latency, is this the reason?
Since I'm pretty much new to this kind of virtualization, I wonder if those kernel settings have an impact on the usability of the host system. So, would you recommend booting the modified kernel on a daily basis for productive work, or just if virtualization is needed? I'd say there isn't that much difference to the "vanilla" Arch kernel, so there's is nothing that would really speak against using the modified kernel even if I don't intend to use virtualization every time, right?

nbhs · 2013-05-25 17:19:18

darkspider wrote:

nbhs, thanks for your help, you're very kind.
Could you elaborate on your kernel config? What's the purpose of using a 1000HZ timer frequency and a voluntary Preemption Model? I've seen some posts of people, who recommend this settings for a Desktop setup, because it reduces latency, is this the reason?
Since I'm pretty much new to this kind of virtualization, I wonder if those kernel settings have an impact on the usability of the host system. So, would you recommend booting the modified kernel on a daily basis for productive work, or just if virtualization is needed? I'd say there isn't that much difference to the "vanilla" Arch kernel, so there's is nothing that would really speak against using the modified kernel even if I don't intend to use virtualization every time, right?

Im using 1000hz instead of dynticks on my system because running the vm on cpus != of 0 would result in my vm hanging specially gaming, and im using voluntary preempt because without it my vm feels sluggish with npt disabled.

And yeah you shouldnt notice any issue on your host using my kernel config.

Last edited by nbhs (2013-05-25 17:21:19)

Arch Linux

#51 2013-05-22 03:27:31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#52 2013-05-22 03:42:24

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#53 2013-05-22 03:51:29

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#54 2013-05-22 14:53:30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#55 2013-05-22 16:32:06

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#56 2013-05-22 20:36:48

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#57 2013-05-22 20:44:20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#58 2013-05-22 21:01:32

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#59 2013-05-22 21:03:12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#60 2013-05-22 21:52:52

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#61 2013-05-22 21:59:52

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#62 2013-05-24 00:46:06

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#63 2013-05-24 00:59:03

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#64 2013-05-24 18:37:10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#65 2013-05-24 21:20:30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#66 2013-05-24 21:45:53

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#67 2013-05-24 21:53:35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#68 2013-05-24 22:08:14

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#69 2013-05-24 22:14:34

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#70 2013-05-24 22:57:42

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#71 2013-05-24 23:04:35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#72 2013-05-25 13:29:22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#73 2013-05-25 16:44:10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#74 2013-05-25 17:05:32

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

#75 2013-05-25 17:19:18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Board footer