Arch Linux

Iksf

Member

Registered: 2015-04-13

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Is there any way to have two identical GPU's, one as pass-through one not.

lspci | grep NVIDIA

01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)

lspci -nn

01:00.0 0300: 10de:13c2 (rev a1)
01:00.1 0403: 10de:0fbb (rev a1)
02:00.0 0300: 10de:13c2 (rev a1)
02:00.1 0403: 10de:0fbb (rev a1)

They have the same id's so I can't seem to separate them.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Is there any way to have two identical GPU's, one as pass-through one not.

lspci | grep NVIDIA

01:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation GM204 [GeForce GTX 970] (rev a1)
02:00.1 Audio device: NVIDIA Corporation GM204 High Definition Audio Controller (rev a1)

lspci -nn

01:00.0 0300: 10de:13c2 (rev a1)
01:00.1 0403: 10de:0fbb (rev a1)
02:00.0 0300: 10de:13c2 (rev a1)
02:00.1 0403: 10de:0fbb (rev a1)

They have the same id's so I can't seem to separate them.

Search the thread for xen-pciback.hide, which for our purposes is similar to pci-stub, but allows devices to be specified by address rather than vendor and device ID.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

zopilote

Member

Registered: 2014-11-02

Posts: 2

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Since my GT610 does support UEFI, it's very strange that your GTX680 doesn't.

Hi, could you provide me detailed info about your GT 610? I'm very interested about getting it cause my evga gt 630 doesn't support uefi, thank you so much in advance!

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Since my GT610 does support UEFI, it's very strange that your GTX680 doesn't.

Hi, could you provide me detailed info about your GT 610? I'm very interested about getting it cause my evga gt 630 doesn't support uefi, thank you so much in advance!

Have you contacted EVGA support?  My EVGA GTX660 didn't come with UEFI support either, but they sent me a new BIOS when I asked.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Since my GT610 does support UEFI, it's very strange that your GTX680 doesn't.

Hi, could you provide me detailed info about your GT 610? I'm very interested about getting it cause my evga gt 630 doesn't support uefi, thank you so much in advance!

Looks like i've got this piece of cra hardware. I've got it(somewhere a year ago) very cheap, for something like 20$ or so. It was used, but on a warranty. So far so good, it works despite being plugged into 16x-1x riser card. There's pci-e x1 version of that card available, but i couldn't get it.
But the performance is, well, worth the price - it's possible to play quake 3 or play full hd video on this, so i'm satisfied.
If i could - i'd swap my CPU with APU, that'd give me one GPU less without losing host output.

Last edited by Duelist (2015-04-14 15:00:33)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

paperinick

Member

Registered: 2015-04-14

Posts: 8

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nevermind

Last edited by paperinick (2015-04-14 18:12:43)

logix

Member

Registered: 2013-11-04

Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I think I tracked down the source of my poor Windows guest performance. I noticed that whenever I move my mouse the CPU usage, as noted by task manager in the guest, shoots up to 30-40% usage (idle sits around 2-4%).

I'm passing my USB controller though to the guest, which should give me the best performance. Possibly bad interrupt handling?

Bronek

Member

From: London

Registered: 2014-02-14

Posts: 123

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

A quick question - I need to downgrade my kernel from 3.18 to 3.14 (former does not play well with ZFS, due to changes in vfs, and the latter is second-best LTS kernel). Is there any patch I should apply on 3.14.38 (current 3.14) for full vfio functionality, as required for full GPU passthrough functionality and qemu 2.2.1 ? In particular I'm thinking https://lkml.org/lkml/2014/5/25/94 , is there anything else?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Possibly bad interrupt handling?

USB HID devices do not generate interrupts, instead, they're polled by the operating system every so often.
Try changing your display and pointing device in QEMU. I'd try VNC as a 100% working solution, but you could just add -device qxl to your QEMU command-line startup script and have SDL graphics window which somehow passes your mouse input into VM.

Aren't you the guy with 8 virtual cores on 4 real cores?.. Yes, you're that guy. Did you sort out your strange CPU configuration? Because, well, overallocating vCPUs is a major performance killer.

Last edited by Duelist (2015-04-15 19:55:12)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

fseven

Member

Registered: 2015-04-15

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,
I've been trying to get this working, but been running into some problems.
I have 2 differend Radeon HD cards, one plugged into the main PCIex16 slot, another in the PCIex4 slot.
The card I want to passthrough is the first one.
PCI-Stub claims the primary card and no radeon driver gets loaded for it, but video is still only output on this one.
How can i switch to the other card or boot with it to begin with? (To clarify: I want the arch host to use the graphics card in the second slot, which is currently not doing anything even though lspci -v shows a loaded radeon driver.)

(My MB doesn't allow me to choose the primary card.)

Is what I'm trying to do possible, or do I have to passthrough the second card? (Not ideal in this case, as I think the x4 slot would hinder the cards performance).

Last edited by fseven (2015-04-16 00:12:25)

Bronek

Member

From: London

Registered: 2014-02-14

Posts: 123

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,
Is what I'm trying to do possible, or do I have to passthrough the second card? (Not ideal in this case, as I think the x4 slot would hinder the cards performance).

Preferably you should switch BIOS (or UEFI) motherboard configuration option to select PCIe slot for primary VGA card, or to disable ROM of the card in 16x slot (which will have the same effect). In case you cannot do that, it is perfectly possible to passthrough primary GPU card, that's what I do (my host is running headless, both primary and secondary GPUs are for passed to two VMs).

Last edited by Bronek (2015-04-16 07:42:49)

fseven

Member

Registered: 2015-04-15

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Preferably you should switch BIOS (or UEFI) motherboard configuration option to select PCIe slot for primary VGA card, or to disable ROM of the card in 16x slot (which will have the same effect).

As I said, I fear i don't have the option to switch the primary slot. (MB is a Gigabyte  970A DS3P).

Could you elaborate on the other thing? How would I disable the ROM?

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Preferably you should switch BIOS (or UEFI) motherboard configuration option to select PCIe slot for primary VGA card, or to disable ROM of the card in 16x slot (which will have the same effect).

As I said, I fear i don't have the option to switch the primary slot. (MB is a Gigabyte  970A DS3P).

Could you elaborate on the other thing? How would I disable the ROM?

Usually gigabyte has that option. Whatever.
You can edit your xorg.conf to choose the device manually by busid, like

Section "Device"
    Identifier     "Device0"
    Driver         "YOUR DRIVER"
    VendorName     "LIKE AMD OR SOMETHING"
    BoardName      "MAY BE YOUR MODEL"
    BusID          "PCI:02:00:00" <<THAT'S YOUR PCI BDF ADDRESS, AS FOUND IN LSPCI
EndSectio

You can use fglrx and hope it's fixed now or radeon. Just try and see if radeon ignores pci-stub like fglrx does. I think it shouldn't do that, and you don't have to explicitly blacklist it.

So, edit the xorg.conf, load a driver for your GPU and it should be fine. Or maybe you don't even need to add that BusID line at all.

Last edited by Duelist (2015-04-16 09:20:09)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

ughman

Member

Registered: 2015-04-12

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I will work on this then. Will this require me to get the guest OS UEFI-bootable? At the moment it is installed on an MBR-partitioned virtual drive.

Yes, the OS will need to be UEFI-bootable. Using OVMF whitepaper you can get Windows 7 to boot.

I believe I have the guest booting now with OVMF. This is my current qemu launch script:

#!/bin/sh

export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_DAC_FIXED_FREQ=48000
export QEMU_ALSA_DAC_BUFFER_SIZE=11054
#export QEMU_ALSA_DAC_PERIOD_SIZE=170
qemu-system-x86_64 \
        -enable-kvm \
        -cpu host,kvm=off \
        -m 4096 \
        -smp cores=2,threads=1,sockets=1 \
        -drive file=OVMF.fd,format=raw,if=pflash \
        -drive file=/dev/mapper/mass-win--uefi,format=raw,if=virtio \
        -usb \
        -usbdevice host:13ba:0018 \
        -usbdevice host:0810:0001 \
        -device vfio-pci,host=01:00.0 \
        -device vfio-pci,host=01:00.1 \
        -soundhw hda \
        -vga none \
        $*

Unfortunately, this hasn't fixed the problem either. I have once gotten what is I assume to be a bsod, but nothing came up on the screen after it went black and turned off, as well as freezes on the later attempts.

Bronek

Member

From: London

Registered: 2014-02-14

Posts: 123

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Has anyone tried passing Radeon 295 X2 ? Asking because it apparently has builtin PLX chip, thus I'm guessing that PCIe topology can be a little "funky".

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I will work on this then. Will this require me to get the guest OS UEFI-bootable? At the moment it is installed on an MBR-partitioned virtual drive.

Yes, the OS will need to be UEFI-bootable. Using OVMF whitepaper you can get Windows 7 to boot.

I believe I have the guest booting now with OVMF. This is my current qemu launch script:

#!/bin/sh

export QEMU_AUDIO_DRV=alsa
export QEMU_ALSA_DAC_FIXED_FREQ=48000
export QEMU_ALSA_DAC_BUFFER_SIZE=11054
#export QEMU_ALSA_DAC_PERIOD_SIZE=170
qemu-system-x86_64 \
        -enable-kvm \
        -cpu host,kvm=off \
        -m 4096 \
        -smp cores=2,threads=1,sockets=1 \
        -drive file=OVMF.fd,format=raw,if=pflash \
        -drive file=/dev/mapper/mass-win--uefi,format=raw,if=virtio \
        -usb \
        -usbdevice host:13ba:0018 \
        -usbdevice host:0810:0001 \
        -device vfio-pci,host=01:00.0 \
        -device vfio-pci,host=01:00.1 \
        -soundhw hda \
        -vga none \
        $*

Unfortunately, this hasn't fixed the problem either. I have once gotten what is I assume to be a bsod, but nothing came up on the screen after it went black and turned off, as well as freezes on the later attempts.

What? Does the OS boots? Or you see a black screen?
What windows do you have? Have you added -device qxl-vga for win7/srv2008r2 as stated in the whitepaper?

You should see tianocore logo with memcheck in upper right corner, then a black screen, after some time "starting windowns" logo animation will show up.
Of course, OS should be installed from UEFI, because it overwrites ESP and UEFIVARS.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

ughman

Member

Registered: 2015-04-12

Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

What? Does the OS boots? Or you see a black screen?

Sorry, it does boot fine. Nothing appears on the screen at first but windows eventually appears with a login prompt.

What windows do you have?

Windows 7 Ultimate x64

Have you added -device qxl-vga for win7/srv2008r2 as stated in the whitepaper?

I was not using that when I posted, but after reading your post I have now tried it and eventually got a freeze regardless. This time it happened while I was scrolling through some text, which appeared to be corrupted once it froze. The display on the passed-through card went black after a few seconds, and then the qxl output went black a few seconds after that.

You should see tianocore logo with memcheck in upper right corner, then a black screen, after some time "starting windowns" logo animation will show up.

That I do (when using -device qxl-vga). Afterwards, the qxl output stays at the logo animation, and the login screen appears on the display attached to the card.

Of course, OS should be installed from UEFI, because it overwrites ESP and UEFIVARS.

Wait, I'm supposed to reinstall the OS after switching to UEFI? That's rather annoying, I was hoping I could keep the same install, as this setup was previously running on bare metal for several months before I moved it into the VM.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Windows 7 Ultimate x64

Have you added -device qxl-vga for win7/srv2008r2 as stated in the whitepaper?

I was not using that when I posted, but after reading your post I have now tried it and eventually got a freeze regardless. This time it happened while I was scrolling through some text, which appeared to be corrupted once it froze. The display on the passed-through card went black after a few seconds, and then the qxl output went black a few seconds after that.

You should see tianocore logo with memcheck in upper right corner, then a black screen, after some time "starting windowns" logo animation will show up.

That I do (when using -device qxl-vga). Afterwards, the qxl output stays at the logo animation, and the login screen appears on the display attached to the card.

Of course, OS should be installed from UEFI, because it overwrites ESP and UEFIVARS.

Wait, I'm supposed to reinstall the OS after switching to UEFI? That's rather annoying, I was hoping I could keep the same install, as this setup was previously running on bare metal for several months before I moved it into the VM.

Seeing a logo stuck on qxl-vga is 100% fine, you can drop the qxl-vga device once you've installed the drivers.
Also, if it boots and works - you don't need to reinstall it. Maybe reinstall some device drivers, but nothing much. There may be a hidden windows "hardware migration" problem somewhere, but if it boots - i think it should work.

I don't know what causes the freeze, but here's my trinity-based system for general reference:

options kvm-amd npt=1 nested=0
options kvm ignore_msrs=0 allow_unsafe_assigned_interrupts=0
options vfio_iommu_type1 disable_hugepages=1 allow_unsafe_interrupts=0

GRUB_CMDLINE_LINUX="vconsole.font=latarcyrheb-sun16 $([ -x /usr/sbin/rhcrashkernel-param ] && /usr/sbin/rhcrashkernel-param || :) nofb pci-stub.ids=1022:780d,1002:683f,1002:aab0 enable_mtrr_cleanup iommu=pt acpi_enforce_resources=lax"

<domain type='kvm' id='2'>
  <name>vm1</name>
  <uuid>f6958833-174f-44e6-817b-2a6c73c6f041</uuid>
  <memory unit='KiB'>4194304</memory>
  <currentMemory unit='KiB'>4194304</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.1'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/vm1_VARS.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <topology sockets='1' cores='4' threads='1'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='directsync' io='native'/>
      <source dev='/dev/sda'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0c' function='0x0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <alias name='usb0'/>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <alias name='usb0'/>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <alias name='usb0'/>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:f0:d5:39'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <model type='rtl8139'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/2'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/2'>
      <source path='/dev/pts/2'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='usb'>
      <alias name='input0'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' listen='127.0.0.1'>
      <listen type='address' address='127.0.0.1'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom bar='on' file='/mnt/hdd/qemu/hybridmagic.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <rom bar='on' file='/mnt/hdd/qemu/hybridmagic.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x02' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev3'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='kvm'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x14' function='0x2'/>
      </source>
      <alias name='hostdev4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir0'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <alias name='redir1'/>
    </redirdev>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='dynamic' model='selinux' relabel='yes'>
    <label>system_u:system_r:svirt_t:s0:c127,c267</label>
    <imagelabel>system_u:object_r:svirt_image_t:s0:c127,c267</imagelabel>
  </seclabel>
</domain>

^^^^
that libvirt xml generates this
v v v v

/usr/bin/qemu-system-x86_64 -machine accel=kvm -name vm1 -S -machine pc-i440fx-2.1,accel=kvm,usb=off -cpu Opteron_G5,+bmi1,+perfctr_nb,+perfctr_core,+topoext,+nodeid_msr,+tce,+lwp,+wdt,+skinit,+ibs,+osvw,+cr8legacy,+extapic,+cmp_legacy,+fxsr_opt,+mmxext,+osxsave,+monitor,+ht,+vme -drive file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/libvirt/qemu/nvram/vm1_VARS.fd,if=pflash,format=raw,unit=1 -m 4096 -realtime mlock=off -smp 4,sockets=1,cores=4,threads=1 -uuid f6958833-174f-44e6-817b-2a6c73c6f041 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm1.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/dev/sda,if=none,id=drive-virtio-disk0,format=raw,cache=directsync,aio=native -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0xc,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:f0:d5:39,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev spicevmc,id=charchannel0,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -device usb-mouse,id=input0 -vnc 127.0.0.1:0 -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,bus=pci.0,addr=0x4 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0x8,rombar=1,romfile=/mnt/hdd/qemu/hybridmagic.rom -device vfio-pci,host=01:00.1,id=hostdev1,bus=pci.0,addr=0x9 -device vfio-pci,host=02:00.0,id=hostdev2,bus=pci.0,addr=0xa,rombar=1,romfile=/mnt/hdd/qemu/hybridmagic.rom -device vfio-pci,host=02:00.1,id=hostdev3,bus=pci.0,addr=0xb -device pci-assign,configfd=24,host=00:14.2,id=hostdev4,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on

(qemu-2.1.3-5.fc21)
Name        : edk2.git-ovmf-x64
Release     : 20150416.b962.ged6fde4
3.19.3-200.fc21.x86_64

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    2
Socket(s):             1
NUMA node(s):          1
Vendor ID:             AuthenticAMD
CPU family:            21
Model:                 16
Model name:            AMD Athlon(tm) X4 750K Quad Core Processor
Stepping:              1
CPU MHz:               1900.000
CPU max MHz:           3400.0000
CPU min MHz:           1400.0000
BogoMIPS:              6834.29
Virtualization:        AMD-V
L1d cache:             16K
L1i cache:             64K
L2 cache:              2048K
NUMA node0 CPU(s):     0-3

-[0000:00]-+-00.0  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Root Complex [1022:1410]
           +-00.2  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) I/O Memory Management Unit [1022:1419]
           +-02.0-[01]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750 / R7 250E] [1002:683f]
           |            \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
           +-04.0-[02]--+-00.0  Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750 / R7 250E] [1002:683f]
           |            \-00.1  Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
           +-11.0  Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7801]
           +-12.0  Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller [1022:7807]
           +-12.2  Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller [1022:7808]
           +-13.0  Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller [1022:7807]
           +-13.2  Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller [1022:7808]
           +-14.0  Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:780b]
           +-14.2  Advanced Micro Devices, Inc. [AMD] FCH Azalia Controller [1022:780d]
           +-14.3  Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:780e]
           +-14.4-[03]----05.0  Ensoniq 5880B [AudioPCI] [1274:5880]
           +-15.0-[04]--+-00.0  NVIDIA Corporation GF119 [GeForce GT 610] [10de:104a]
           |            \-00.1  NVIDIA Corporation GF119 HDMI Audio Controller [10de:0e08]
           +-15.1-[05]----00.0  ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
           +-15.2-[06]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168]
           +-16.0  Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller [1022:7807]
           +-16.2  Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller [1022:7808]
           +-18.0  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 0 [1022:1400]
           +-18.1  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 1 [1022:1401]
           +-18.2  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 2 [1022:1402]
           +-18.3  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 3 [1022:1403]
           +-18.4  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 4 [1022:1404]
           \-18.5  Advanced Micro Devices, Inc. [AMD] Family 15h (Models 10h-1fh) Processor Function 5 [1022:1405]

Try running dmesg -w in some other window while trying to "freeze" the VM.
You can also plug in ISA debug console to check logs, like this:

-debugcon file:debug.log -global isa-debugcon.iobase=0x402

Last edited by Duelist (2015-04-16 18:53:53)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

erganzi

Member

Registered: 2014-07-09

Posts: 19

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, aw
the AMD Firepro R5000 can passthrough successfully, but when I install vga driver, it restart with BSOD like atikmpag.sys error

my kernel:
linux-3.18.6
qemu:
qemu-2.1.3

commandline:
/usr/libexec/qemu-kvm-passthrough -cpu host -smp 1,sockets=1,cores=1,threads=1 \
-m 8192 -M q35 -enable-kvm \
-rtc base=localtime,clock=host \
-acpitable file=/usr/share/qemu/win-pre-install-with-slic.bin \
-drive file=/var/lib/libvirt/images/fwq_clean_win7_64bit-patched.img,id=virtio-scsi-disk0 \
-device virtio-blk-pci,drive=virtio-scsi-disk0 \
-net nic,model=virtio,macaddr=06:d8:e8:00:00:1b,addr=0x3 \
-net tap,ifname=tap0,script=/root/qemu-ifup \
-vga none -display none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=07:00.1,bus=root.1,addr=00.1 \
-device vfio-pci,host=06:00.0,bus=root.1,addr=00.2 \
-device vfio-pci,host=06:00.1,bus=root.1,addr=00.3 \
-device vfio-pci,host=06:00.2,bus=root.1,addr=00.4 \
-monitor stdio

apex8

Member

Registered: 2014-03-29

Posts: 60

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,
when I want to use a newer version of the ovmf-svn package in my VM, is it possible to keep the part where the settings are stored in from the old build?

Thanks
apex

fseven

Member

Registered: 2015-04-15

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm using a different board now and got it working fine, but I have another question:
Is it possible to passthrough a single SATA port? Or how would passing through the disk like in the first place not impact performance? (Because then it wouldn't be in AHCI mode.)
I can't passthrough an entire controller, since I only have one.

apex8

Member

Registered: 2014-03-29

Posts: 60

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm using a different board now and got it working fine, but I have another question:
Is it possible to passthrough a single SATA port? Or how would passing through the disk like in the first place not impact performance? (Because then it wouldn't be in AHCI mode.)
I can't passthrough an entire controller, since I only have one.

I don´t believe passing through a single port is possible. The best would be installing a separate controller, if that is an option for you.

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,
when I want to use a newer version of the ovmf-svn package in my VM, is it possible to keep the part where the settings are stored in from the old build?

Thanks
apex

When using EDK2 GIT builds, you have the following files:

/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF-with-csm.fd
/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF_CODE-with-csm.fd
/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF_VARS-with-csm.fd

If you will add both OVMF_CODE and OVMF_VARS, you can update only OVMF_CODE(just copy the file, or libvirt does it's stuff) without changing the VARS. lersek may provide more info on this.

@fseven: you can achieve a similar effect when doing -device virtio-blk-pci with drive /dev/sda or even /dev/sda4(but that will bork the partition table, as the VM will think it's a whole disk). If you poke windows and it's EFI-loader the right way, you may even boot baremetal from that partitions.

I have /dev/sda1 as 500~GB junkyard FS mounted to linux, and windows knows about it(since it has full /dev/sda), but neither OS can mount other's FS. Linux prevents you, windows doesn't do ext4.
There are some caching and scheduling issues present, but they don't affect the performance much.

I haven't heard any success-stories about dedicating a single port to some VM.

Last edited by Duelist (2015-04-17 10:03:15)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

fseven

Member

Registered: 2015-04-15

Posts: 16

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I'm using a different board now and got it working fine, but I have another question:
Is it possible to passthrough a single SATA port? Or how would passing through the disk like in the first place not impact performance? (Because then it wouldn't be in AHCI mode.)
I can't passthrough an entire controller, since I only have one.

I don´t believe passing through a single port is possible. The best would be installing a separate controller, if that is an option for you.

I'm looking into that, but they do seem to cost a fair bit, not sure if it's worth it.
I get about 90MB/s write on Guest and 380MB/s on host, which is quite a big difference. At this speed I might as well be using a normal HD.

Update: Just tried passing it through using the -hda option instead of using a virtio-scsi device like in the first post, performance seems to be within 10-15% which is completely acceptable.

apex8

Member

Registered: 2014-03-29

Posts: 60

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi,
when I want to use a newer version of the ovmf-svn package in my VM, is it possible to keep the part where the settings are stored in from the old build?

Thanks
apex

When using EDK2 GIT builds, you have the following files:

/usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF-with-csm.fd
/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF_CODE-with-csm.fd
/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd
/usr/share/edk2.git/ovmf-x64/OVMF_VARS-with-csm.fd

@fseven: you can achieve a similar effect when doing -device virtio-blk-pci with drive /dev/sda or even /dev/sda4(but that will bork the partition table, as the VM will think it's a whole disk). If you poke windows and it's EFI-loader the right way, you may even boot baremetal from that partitions.

Can you give some more details about "poking windows and it´s EFI-loader the right way"? Furthermore I would expect Windows to complain about changed chipset/driver mismatch and crashing, or is this an issue which doesn't exist in newer Windows versions?

Last edited by apex8 (2015-04-17 12:07:19)