Arch Linux

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

If you are dual booting Windows, just use GPU-Z to dump your GPU BIOS into a file. Then you'll have VGA reset from QEMU.

you can dump it aslo from virtual machine (sometimes it doesn't work at first but it does after couple of tries/gpu-z versions...)

mbirkis

Member

Registered: 2013-12-12

Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Is there still an issue with the nouveau driver on the host, i am currently runnin the patched nvidia driver but i would like to switch for the open source one on my host.

Anyone running nouveau on host?

Chousuke

Member

Registered: 2014-07-29

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'd like to chime in here with a success report. I'm not an archlinux user, but I have my Radeon R9 270 successfully passed through to a kvm Windows 7 guest on Fedora 20.

The thing is, I have no idea what actually made it work. I tried all sorts of things, but in the end I reinstalled and created the virtual machine purely with virt-manager (from Fedora virt-preview repos) and just selected the GPU (only) as pass-through from the GUI. My version of virt-manager seems to use vfio by default. I did have to manually modprobe vfio-pci etc. though, and of course ensure that the radeon driver wasn't being used... X runs using the IGP and I see no graphics corruption or other issues even after repeated reboots of the VM. 3d acceleration also seems to work on the host.

I have an emulated Cirrus VGA chip on the VM too, but it stops working as soon as the guest boots up properly. I used it to install the Catalyst drivers (for some reason, with QXL the installer crashed)

The full virt-manager XML is here:
http://pastebin.com/ZX0Kfex5

My software versions are as follows:

- Kernel 3.15.6-200.fc20.x86_64 custom built with vga arbitration patches (81b5c7bc8de3e6f63419139c2fc91bf81dea8a7d, 6e1b4fdad5157bb9e88777d525704aba24389bee) though I am not sure if they're actually needed... I see no indication of it. Maybe because I do EFI boot?
- virt-manager 1.0.1
- qemu 2.0.93

and hardware:
Intel(R) Core(TM) i5-4670 CPU @ 3.40GHz
AMD Radeon R9 270
ASUS H87 Pro

dwe11er

Member

Registered: 2014-03-18

Posts: 73

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'd like to chime in here with a success report. I'm not an archlinux user, but I have my Radeon R9 270 successfully passed through to a kvm Windows 7 guest on Fedora 20.

The thing is, I have no idea what actually made it work. I tried all sorts of things, but in the end I reinstalled and created the virtual machine purely with virt-manager (from Fedora virt-preview repos) and just selected the GPU (only) as pass-through from the GUI. My version of virt-manager seems to use vfio by default. I did have to manually modprobe vfio-pci etc. though, and of course ensure that the radeon driver wasn't being used... X runs using the IGP and I see no graphics corruption or other issues even after repeated reboots of the VM. 3d acceleration also seems to work on the host.

I have an emulated Cirrus VGA chip on the VM too, but it stops working as soon as the guest boots up properly. I used it to install the Catalyst drivers (for some reason, with QXL the installer crashed)

The full virt-manager XML is here:
http://pastebin.com/ZX0Kfex5

My software versions are as follows:

- Kernel 3.15.6-200.fc20.x86_64 custom built with vga arbitration patches (81b5c7bc8de3e6f63419139c2fc91bf81dea8a7d, 6e1b4fdad5157bb9e88777d525704aba24389bee) though I am not sure if they're actually needed... I see no indication of it. Maybe because I do EFI boot?
- virt-manager 1.0.1
- qemu 2.0.93

and hardware:
Intel(R) Core(TM) i5-4670 CPU @ 3.40GHz
AMD Radeon R9 270
ASUS H87 Pro

It's because you're not using vfio-pci, but normal pci-assign.

Chousuke

Member

Registered: 2014-07-29

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It's because you're not using vfio-pci, but normal pci-assign.

I am quite sure I'm using vfio, since the guest refuses to start without the vfio modules loaded:

Error starting domain: internal error: process exited while connecting to monitor: pulseaudio: pa_context_connect() failed
pulseaudio: Reason: Connection refused
pulseaudio: Failed to initialize PA contextaudio: Could not init `pa' audio driver
2014-07-29T13:28:42.126919Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=0: vfio: No available IOMMU models
2014-07-29T13:28:42.126941Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=0: vfio: failed to setup container for group 1
2014-07-29T13:28:42.126951Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=0: vfio: failed to get group 1
2014-07-29T13:28:42.126963Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=0: Device initialization failed.
2014-07-29T13:28:42.126975Z qemu-system-x86_64: -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x9,rombar=0: Device 'vfio-pci' could not be initialized

EDIT: Curiously enough, this setup also works just fine with a completely unmodified 3.15.4-200.fc20.x86_64 Fedora kernel.
EDIT2: It is entirely possible that the virt-manager and qemu packages I am using have some special modifications which enable this to function. I just now discovered that the qemu binary seems to lie about its version... the package I have installed comes from the fedora-virt-preview repository and yum tells me its version is:

Version     : 2.1.0
Release     : 0.5.rc3.fc20

Last edited by Chousuke (2014-07-29 13:46:05)

chillum

Member

Registered: 2014-07-30

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

what do I do if I have exactly the same times 2 wireless keyboard/ mouse; ID bit is the same

...
Bus 002 Device 006: ID 1a81:1006 Holtek Semiconductor, Inc.
...
Bus 011 Device 004: ID 1a81:1006 Holtek Semiconductor, Inc.
...
how do I tell KVM/windows which one to passthrough?

chillum

Member

Registered: 2014-07-30

Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I know

-usbdevice host:1a81:1006

vogets

Member

Registered: 2006-11-25

Posts: 31

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Anyone running nouveau on host?

A 9500 GT with nouveau runs smoothly for me. Unpatched.

CharlieBra7o

Member

Registered: 2014-03-01

Posts: 20

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

After I booted up my freshly installed Windows 8.1 for the first time it worked for about 60 seconds or so, then the monitor lost its signal. From this moment on the monitor always loses its signal during the Windows loading screen. The VM's CPU load from this moment on seems to be completely steady using up 1 full thread.

My guess is that Windows tried to install some crappy GPU/chipset drivers or apply some wrong settings...

Does anyone have an idea what might be going on there?

Additional info:
- Radeon HD 7950
- 3.15.6 from the OP unchanged
- qemu-git 2.1.r34391.gf368c33-1 + libvirt 1.2.6-1 from GIT

Just tried installing Windows 7 and after the installation I spent about 5 minutes clicking around, then shutdown the VM to make an image and started it up again. Now it always freezes during the "Starting Windows" screen and 'virsh list' lists the VM as 'paused'.
dmesg doesn't show anything peculiar, except maybe

kvm: zapping shadow pages for mmio generation wraparound

Honestly, I have absolutely no clue what to try next...

virsh tells you the reason, why vm was paused. I had similar issue (I/O Error), because of vm's disks being raw file images on btrfs (and having really low performance).

Could you please tell me where it states the reason and how you fixed your problem?
That's my disk...

Device           Start          End   Size Type
...
/dev/sda7    167774208    330248191  77.5G Linux filesystem

It worked perfectly fine with Win 8.1 without GPU passthrough and the first time I booted Win7 with GPU passthrough

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/disk/by-partuuid/771af6c5-d5db-421d-9f67-8070092961bf'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x04' function='0x0'/>
    </disk>

EDIT:
I just stumbled upon this post:

peaquino, your errors seem to be occuring on 01:00.1 not 01:00.0, which is your audio card.

[ 2248.441807] AMD-Vi: Event logged [IO_PAGE_FAULT device=01:00.1 domain=0x0002 address=0x00000000857fc800 flags=0x0010]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series]

is there are reason why you aren't assigning it? (like so)

-device vfio-pci,host=01:00.0,bus=root.1,addr=0.0,multifunction=on,x-vga=on,romfile=/home/myuser/Tahiti.rom \
-device vfio-pci,host=01:00.1,bus=root.1,addr=0.1 \

Numerous folks have reported BSODs which were solved by moving the audio device to the root complex rather than exposing it as a sub-function.  It may work, but exposing it on pcie.0 has had more success it seems.

So what I did is just remove the Radeon HD audio device qemu command line and now Win 7 seems so be content...

Last edited by CharlieBra7o (2014-08-01 23:45:11)

devianceluka

Member

Registered: 2014-05-19

Posts: 44

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

VFIO passthrough works flawlessly on Ubuntu 14.04 with no patches just yet but I have a few questions if someone could please help.

1) How to shutdown/forceshutdown/kill the VM started with qemu over SSH?

2) What are the parameters to make VM connect to the physical network (to eth0 on host)?

3) Is it possible to see "display output" over SSH? The host is a minimal virtual machine host with only CLI and no GUI.

4) Whats the best way to autostart 3 VMs with host, ofcourse each with its own vfio/config?

Thanks!

Last edited by devianceluka (2014-08-01 21:35:23)

tritron4

Member

Registered: 2012-04-14

Posts: 153

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Did anyone managed to run osx? is kvm better than xen ?

kristopher004

Member

Registered: 2014-06-30

Posts: 33

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

osx works great, virtio network drivers crash though, better off with e1000, have to put it on high pcie bus addr though or osx won't see it.

penetal

Member

Registered: 2014-06-15

Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I can not seem to get audio working right. No matter what I do I end up with crackling sound.
If I set QEMU_PA_SAMPLES to 128 its way way worse.
This affects all sound, not just the VM. If I have something playing on the host once I start the VM the sound gets all fucky.

I have no idea if it could possibly help, but here is the pulseauido output for when I start the VM, and then kill it almost right away (made sure the sound playing on the host had gone bad before I stopped it.

I: [pulseaudio] client.c: Created 3 "Native client (UNIX socket client)"
I: [pulseaudio] protocol-native.c: Client authenticated anonymously.
I: [pulseaudio] module-stream-restore.c: Restoring volume for sink input sink-input-by-application-name:/run/user/1000/pulse/native.
I: [pulseaudio] module-stream-restore.c: Restoring mute state for sink input sink-input-by-application-name:/run/user/1000/pulse/native.
I: [pulseaudio] sink-input.c: Created input 3 "qemu" on alsa_output.pci-0000_00_1b.0.analog-stereo with sample spec s16le 2ch 44100Hz and channel map front-left,front-right
I: [pulseaudio] sink-input.c:     media.name = "qemu"
I: [pulseaudio] sink-input.c:     application.name = "/run/user/1000/pulse/native"
I: [pulseaudio] sink-input.c:     native-protocol.peer = "UNIX socket client"
I: [pulseaudio] sink-input.c:     native-protocol.version = "29"
I: [pulseaudio] sink-input.c:     application.process.id = "7409"
I: [pulseaudio] sink-input.c:     application.process.user = "root"
I: [pulseaudio] sink-input.c:     application.process.host = "ArcherBase"
I: [pulseaudio] sink-input.c:     application.process.binary = "qemu-system-x86_64"
I: [pulseaudio] sink-input.c:     application.language = "C"
I: [pulseaudio] sink-input.c:     window.x11.display = ":0"
I: [pulseaudio] sink-input.c:     application.process.machine_id = "81157a0d6cb84ce9b5fd22006cf55017"
I: [pulseaudio] sink-input.c:     module-stream-restore.id = "sink-input-by-application-name:/run/user/1000/pulse/native"
I: [pulseaudio] protocol-native.c: Requested tlength=10.00 ms, minreq=4.99 ms
I: [pulseaudio] protocol-native.c: Final latency 10.98 ms = 0.50 ms + 2*4.99 ms + 0.50 ms
I: [alsa-source-ALC1150 Analog] alsa-source.c: Trying resume...
I: [alsa-source-ALC1150 Analog] alsa-util.c: Trying to disable ALSA period wakeups, using timers only
I: [alsa-source-ALC1150 Analog] alsa-util.c: ALSA period wakeups disabled
I: [alsa-source-ALC1150 Analog] alsa-source.c: Time scheduling watermark is 20.00ms
I: [alsa-source-ALC1150 Analog] alsa-source.c: Resumed successfully...
I: [alsa-source-ALC1150 Analog] alsa-source.c: Starting capture.
I: [pulseaudio] source-output.c: Created output 1 "qemu" on alsa_input.pci-0000_00_1b.0.analog-stereo with sample spec s16le 2ch 44100Hz and channel map front-left,front-right
I: [pulseaudio] source-output.c:     media.name = "qemu"
I: [pulseaudio] source-output.c:     application.name = "/run/user/1000/pulse/native"
I: [pulseaudio] source-output.c:     native-protocol.peer = "UNIX socket client"
I: [pulseaudio] source-output.c:     native-protocol.version = "29"
I: [pulseaudio] source-output.c:     application.process.id = "7409"
I: [pulseaudio] source-output.c:     application.process.user = "root"
I: [pulseaudio] source-output.c:     application.process.host = "ArcherBase"
I: [pulseaudio] source-output.c:     application.process.binary = "qemu-system-x86_64"
I: [pulseaudio] source-output.c:     application.language = "C"
I: [pulseaudio] source-output.c:     window.x11.display = ":0"
I: [pulseaudio] source-output.c:     application.process.machine_id = "81157a0d6cb84ce9b5fd22006cf55017"
I: [pulseaudio] source-output.c:     module-stream-restore.id = "source-output-by-application-name:/run/user/1000/pulse/native"
I: [pulseaudio] protocol-native.c: Final latency 2000.00 ms = 1000.00 ms + 1000.00 ms
I: [alsa-sink-ALC1150 Analog] alsa-sink.c: Underrun!
I: [alsa-sink-ALC1150 Analog] alsa-sink.c: Increasing minimal latency to 1.00 ms
I: [pulseaudio] source-output.c: Freeing output 1 "qemu"
I: [pulseaudio] sink-input.c: Freeing input 3 "qemu"
I: [pulseaudio] client.c: Freed 3 "/run/user/1000/pulse/native"
I: [pulseaudio] protocol-native.c: Connection died.

When I kill the VM sound goes back to normal and the audio playing on the host is fine.

Script to start the VM

  1 #!/bin/sh
  2 
  3 vfio-bind 0000:01:00.0 0000:01:00.1
  4 
  5 export QEMU_PA_SAMPLES=2048
  6 export QEMU_PA_SERVER=/run/user/1000/pulse/native
  7 
  8 qemu-system-x86_64 -enable-kvm -M q35 -m 10240 -cpu host \
  9         -smp 6,sockets=1,cores=3,threads=2 \
 10         -bios /usr/share/qemu/bios.bin \
 11         -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 12         -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
 13         -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
 14         -drive id=system,file=/home/thomas/vm/GamingWindows.img,format=raw \
 15         -device ide-hd,bus=ide.0,drive=system \
 16         -vga none \
 17         -monitor stdio \
 18         -drive id=storage,file=/home/thomas/vm/GamingWindowsStorage.qcow2,format=qcow2 \
 19         -device ide-hd,bus=ide.1,drive=storage \
 20         -usb -device usb-host,hostbus=1,hostaddr=7 \
 21         -usb -device usb-host,hostbus=1,hostaddr=14 \
 22         -usb -usbdevice host:046d:c52b \
 23         -device ich9-intel-hda,bus=pcie.0,addr=1b.0,id=sound0 \
 24         -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

valkaiser

Member

Registered: 2014-08-02

Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all, great thread.
I have successfully passed through an nVidia 7900GS to a windows XP SP3 guest.
Host hardware: ASUS P9X79 WS, i7 4930K, nVidia 7900GS x2, ATI X300 x2.
Host is running Arch with stock kernel, qemu, and seabios.
Guest is using emulated IDE for HDD as I had trouble getting XP to run, but I'm still fiddling with it.
QEMU invocation script:

#!/bin/bash
modprobe vfio-pci
vfio-bind 0000:04:00.0
qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host \
 -smp 6,sockets=1,cores=6,threads=1 \
 -bios /usr/share/qemu/bios.bin -vga none \
 -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
 -device vfio-pci,host=04:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
 -device piix4-ide,bus=pcie.0,id=piix4-ide \
 -drive file=/media/Data3/OS/windows.img,id=disk,format=raw -device ide-hd,bus=piix4-ide.0,drive=disk \
 -drive file=virtio-win-0.1-81.iso,id=isocd2 -device ide-cd,bus=piix4-ide.1,drive=isocd2 \
 -usb -usbdevice host:05ac:0201 -usbdevice host:046d:c03d \
 -boot once=d \
 -no-acpi \
 -nographic

Anyways, what has me stumped currently is the nVidia drivers on the guest. I installed the latest available for my 7900, 307.83, and the install goes smoothly. However, upon rebooting, XP does not use the driver.
The device status screen gives me this:
Your computer's system firmware does not include enough information to properly configure and use this device. To use this device, contact your computer manufacturer to obtain a firmware or BIOS update. [CODE 35]
Are there other drivers for the guest that I needed to install first?

Any ideas?

Oh, and thanks for pooling all this information here. It has been enormously helpful.

UPDATE: I get exactly the same results when passing through an ATI X300.
UPDATE: Now using linux-mainline+patches from OP. No change.

Last edited by valkaiser (2014-08-03 05:13:13)

rg80

Member

Registered: 2014-07-23

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi again.

Do you have any idea how to add -cpu kvm=off option into a virt-manager xml?
I already tried with qemu arg:

    <qemu:arg value='-cpu'/>
    <qemu:arg value='kvm=off'/>

but it does not work.

Should i put it somewhere into CPU options?

  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>

Thanks.

siddharta

Member

Registered: 2014-05-03

Posts: 30

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm in the process of assembling a new rig primarily for gaming and would run Win 8.1 as VM, I've had good experiences with this on another setup as HTPC. I understand that GPU performance should be close to native, how does the CPU side of the equation look in terms of overhead? Is it worth getting the i7 4790K over the i5 4690K, the latter being one third cheaper (close to 100 euro difference here).

I would think that in case of using Steam in-home streaming the i7 would be better due to not having nVidia or Intel QuickSync available for hardware assisted h.264 encoding (using AMD and impossible to KVM passthrough the iGPU). If I disregard that scenario it looks less worth it. Is that correct? Thanks.

slis

Member

Registered: 2014-06-02

Posts: 127

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm in the process of assembling a new rig primarily for gaming and would run Win 8.1 as VM, I've had good experiences with this on another setup as HTPC. I understand that GPU performance should be close to native, how does the CPU side of the equation look in terms of overhead? Is it worth getting the i7 4790K over the i5 4690K, the latter being one third cheaper (close to 100 euro difference here).

I would think that in case of using Steam in-home streaming the i7 would be better due to not having nVidia or Intel QuickSync available for hardware assisted h.264 encoding (using AMD and impossible to KVM passthrough the iGPU). If I disregard that scenario it looks less worth it. Is that correct? Thanks.

i7 is much better... 2x more vcpu-s..u won't have good streaming performance in software mode... but valve said they are working on AMD VCE and no ETA.

Last edited by slis (2014-08-03 07:56:31)

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Having some trouble with sound. I am using the command in the op, but the sound in windows is filled with alot of static. When I switch to usb audio, it works, but I prefer to keep it analog.

Any fix?

kameloc

Member

Registered: 2014-06-28

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Having some trouble with sound. I am using the command in the op, but the sound in windows is filled with alot of static. When I switch to usb audio, it works, but I prefer to keep it analog.

Any fix?

I had this issue on Windows 8 and tried using different cards to emulate with no fix. Switched to Windows 7 and using ac97 (-sound ac97) works really well.

shrubuntu

Member

Registered: 2012-12-26

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Having some trouble with sound. I am using the command in the op, but the sound in windows is filled with alot of static. When I switch to usb audio, it works, but I prefer to keep it analog.

Any fix?

I had this issue on Windows 8 and tried using different cards to emulate with no fix. Switched to Windows 7 and using ac97 (-sound ac97) works really well.

-device ac97,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

like this?

kameloc

Member

Registered: 2014-06-28

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Having some trouble with sound. I am using the command in the op, but the sound in windows is filled with alot of static. When I switch to usb audio, it works, but I prefer to keep it analog.

Any fix?

I had this issue on Windows 8 and tried using different cards to emulate with no fix. Switched to Windows 7 and using ac97 (-sound ac97) works really well.

-device ac97,bus=pcie.0,addr=1b.0,id=sound0 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0

like this?

Oh sorry I don't passthrough my onboard audio, just share it. If you add

-soundhw ac97

It emulates Realtek AC'97 for the guest. It allowed me to use my speakers from the VM or host, and at the same time

zerotri

Member

Registered: 2014-08-03

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Been having trouble getting this all setup on my system. Tried with both Arch and Fedora 20 with similar results.
Motherboard is an Asus P9X79 Pro, cpu is an i7-3820, both support VT-d.

When I start my host it seems to stay in low resolution efifb mode. It also never reaches a login screen, just stays at service startup output. I have to switch to a login using Ctrl + Alt + F2. I've tried going through quite a few of the pages in this thread and am not seeing anyone getting similar results.

dmesg | grep -e DMAR -e IOMMU:

[root@Heimdall ~]# dmesg | grep -e DMAR -e IOMMU
[    0.000000] ACPI: DMAR 0x00000000BE4C32B0 0000B4 (v01 A M I  OEMDMAR  00000001 INTL 00000001)
[    0.000000] Intel-IOMMU: enabled
[    0.166673] dmar: IOMMU 0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0462 ecap f020fe
[    0.166765] IOAPIC id 0 under DRHD base  0xfbffc000 IOMMU 0
[    0.166766] IOAPIC id 2 under DRHD base  0xfbffc000 IOMMU 0
[    0.771339] IOMMU 0 0xfbffc000: using Queued invalidation
[    0.771341] IOMMU: Setting RMRR:
[    0.771352] IOMMU: Setting identity map for device 0000:00:1a.0 [0xbea9a000 - 0xbeaa8fff]
[    0.771375] IOMMU: Setting identity map for device 0000:00:1d.0 [0xbea9a000 - 0xbeaa8fff]
[    0.771388] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.771395] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    1.231728] dmar: DMAR:[DMA Read] Request device [09:00.1] fault addr fffe0000 
DMAR:[fault reason 02] Present bit in context entry is clear

Command line:

BOOT_IMAGE=/vmlinuz-3.15.7-200.fc20.x86_64 root=/dev/mapper/fedora-root ro vconsole.font=latarcyrheb-sun16 rd.lvm.lv=fedora/root intel_iommu=on pci-stub.ids=1002:68ba,1002:aa58 pcie_acs_override=downstream rhgb quiet LANG=en_US.UTF-8

lspci -v (cut out just the GPUs):

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 6770] (prog-if 00 [VGA controller])
	Subsystem: XFX Pine Group Inc. Device 3156
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at d0000000 (64-bit, prefetchable) [size=256M]
	Memory at fbe20000 (64-bit, non-prefetchable) [size=128K]
	I/O ports at e000 [size=256]
	Expansion ROM at fbe00000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: vfio-pci
	Kernel modules: radeon

01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]
	Subsystem: XFX Pine Group Inc. Device aa58
	Flags: bus master, fast devsel, latency 0, IRQ 10
	Memory at fbe40000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Juniper XT [Radeon HD 6770] (prog-if 00 [VGA controller])
	Subsystem: XFX Pine Group Inc. Device 3156
	Flags: fast devsel, IRQ 11
	Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Memory at fbd20000 (64-bit, non-prefetchable) [disabled] [size=128K]
	I/O ports at d000 [disabled] [size=256]
	Expansion ROM at fbd00000 [disabled] [size=128K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: pci-stub
	Kernel modules: radeon

02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Juniper HDMI Audio [Radeon HD 5700 Series]
	Subsystem: XFX Pine Group Inc. Device aa58
	Flags: fast devsel, IRQ 10
	Memory at fbd40000 (64-bit, non-prefetchable) [disabled] [size=16K]
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: pci-stub
	Kernel modules: snd_hda_intel

ls -al /dev/vfio/:

[root@Heimdall vfio]# ls -al /dev/vfio
total 0
drwxr-xr-x.  2 root root       80 Aug  3 15:45 .
drwxr-xr-x. 21 root root     3420 Aug  3 15:45 ..
crw-------.  1 root root 246,   0 Aug  3 15:45 15
crw-rw-rw-.  1 root root  10, 196 Aug  3 15:45 vfio

I'm out of ideas as to what it could be. I've tried with various command line options, tried swapping around which graphics card is bound as well as tried binding both.
Is there anything that is glaringly obvious to someone who has this working that I'm just completely missing here? Could it be due to me using two of the same graphics card?

And on a much less important matter for aw, I've been reading up on the dma-alias patch because I have a Marvell 88SE9128 sata controller that causes boot to take several minutes with intel_iommu=on. It is my understanding that this patch has been rejected from mainline so I'm likely out of luck. Would you happen to have any suggestions for how to deal with this?

Thank you for any and all help you can provide,
- Wynter Woods

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

And on a much less important matter for aw, I've been reading up on the dma-alias patch because I have a Marvell 88SE9128 sata controller that causes boot to take several minutes with intel_iommu=on. It is my understanding that this patch has been rejected from mainline so I'm likely out of luck. Would you happen to have any suggestions for how to deal with this?

The latest attempt will come together in v3.17.  I assume that's what's causing your DMAR fault.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

atmouse

Member

Registered: 2013-06-21

Posts: 6

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I have two video cards, HIS HD7870 x2. The hardware ID,vender just the same.

How could I use vga-passthrough one of them to qemu-kvm?  is it that setting pci-stub.ids to the same id block both of the gpu? working or not？

CPU is AMD-FX8350, no gpu integrated.

Last edited by atmouse (2014-08-04 13:56:52)

rg80

Member

Registered: 2014-07-23

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi again.

Do you have any idea how to add -cpu kvm=off option into a virt-manager xml?
I already tried with qemu arg:

    <qemu:arg value='-cpu'/>
    <qemu:arg value='kvm=off'/>

but it does not work.

Should i put it somewhere into CPU options?

  <cpu mode='host-passthrough'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>

Thanks.

Bumping in a different format.
I want to disable the KVM Signture to update my nvidia windows drivers to any newer than v335.

Then got latest GIT version of qemu, applied this patch, then config, make, make install, reboot.

Applied:

    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,kvm=off'/>

but qemu reports .kvm is not an available option for qemu-system-x86_64.

On the contrary:

    <qemu:arg value='-cpu'/>
    <qemu:arg value='host,hv-time'/>

works.

Am i doing something wrong?

Note:
For some reason, the original patch posted on page 1 gives me a format error on:

diff --git a/target-i386/cpu.c b/target-i386/cpu.c

it's not a blank, i can't understand where the problem is...

Thanks!