Arch Linux

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The acpi bios thing its because of seabios, use my package (1.7.2 patched), the other message is likely because you're using the wrong kernel without pcie bus reset support

Yes, You are right - I have no BSOD with 1.7.2 but I could not apply kernel-vfio-vga-reset.patch to Linux kernel from vga-current branch: I get

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

You dont need to patch that kernel, i extracted the commits from that branch and use it against vanilla kernel, and it probably doesnt work anymore after you reboot the vm because you're using the wrong kernel (it cant reset the card), i posted a qemu and kernel builds for you on page 13 you can use that or build it yourself

Last edited by nbhs (2013-07-28 22:20:23)

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I'm getting the same problem as a guy a few pages back... "This device can not find enough free resources that it can use. (Code 12)"... apparently something to do with using the intel integrated graphics. I read the post that was linked but it seems like a dead end?

- I can't get the graphics card to show the BIOS.
- If I boot the ArchLinux ISO in the VM then it shows it's loading screen on the graphics card.
- Windows shows the device in task manager and AMD installs the catalyst drivers.

[root@localhost ~]# dmesg | grep IOMMU
[    0.000000] Intel-IOMMU: enabled
[    0.021443] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.021447] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.021516] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.297980] IOMMU 0 0xfed90000: using Queued invalidation
[    0.297981] IOMMU 1 0xfed91000: using Queued invalidation
[    0.436381] IOMMU: software identity mapping for device 0000:00:00.0
[    0.436382] IOMMU: software identity mapping for device 0000:00:01.0
[    0.436383] IOMMU: software identity mapping for device 0000:00:14.0
[    0.436384] IOMMU: software identity mapping for device 0000:00:16.0
[    0.436385] IOMMU: software identity mapping for device 0000:00:1a.0
[    0.436386] IOMMU: software identity mapping for device 0000:00:1b.0
[    0.436387] IOMMU: software identity mapping for device 0000:00:1c.0
[    0.436388] IOMMU: software identity mapping for device 0000:00:1c.2
[    0.436389] IOMMU: software identity mapping for device 0000:00:1d.0
[    0.436390] IOMMU: software identity mapping for device 0000:00:1f.0
[    0.436391] IOMMU: software identity mapping for device 0000:00:1f.2
[    0.436392] IOMMU: software identity mapping for device 0000:00:1f.3
[    0.436396] IOMMU: software identity mapping for device 0000:01:00.0
[    0.436397] IOMMU: software identity mapping for device 0000:01:00.1
[    0.436401] IOMMU: software identity mapping for device 0000:02:00.0
[    0.436402] IOMMU: Setting RMRR:
[    0.436410] IOMMU: Setting identity map for device 0000:00:02.0 [0xcb800000 - 0xcf9fffff]
[    0.436696] IOMMU: Setting identity map for device 0000:00:1d.0 [0xc9d82000 - 0xc9d8efff]
[    0.436701] IOMMU: Setting identity map for device 0000:00:1a.0 [0xc9d82000 - 0xc9d8efff]
[    0.436702] IOMMU: Setting identity map for device 0000:00:14.0 [0xc9d82000 - 0xc9d8efff]
[    0.436704] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.436705] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@localhost ~]# 

[root@localhost ~]# dmesg | grep pci-stub
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=/dev/md1 ro quiet pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on iommu=pt
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=/dev/md1 ro quiet pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on iommu=pt
[    0.705428] pci-stub: add 1002:679A sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.705438] pci-stub 0000:01:00.0: claimed by stub
[    0.705442] pci-stub: add 1002:AAA0 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.705448] pci-stub 0000:01:00.1: claimed by stub
[root@localhost ~]# 

[   60.412232] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[   60.438127] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x270
[   60.438134] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1b@0x2d0

qemu-system-x86_64 -enable-kvm -M q35 -m 4096 -cpu host \
-smp 2 \
-bios /usr/share/qemu/bios.bin \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-device virtio-scsi-pci,id=scsi \
-drive if=none,format=raw,file=/root/win_os.raw,id=disk \
-device scsi-hd,drive=disk \
-net nic,model=virtio \
-net tap,ifname=vm-win,script=no,downscript=no

Are you using the lastest builds i posted?  iommu=pt is an amd thing only (i think for intel is intel_iommu=pt) and i have no idea what igfx_on does, it seems myweb has partially got it to work on his intel board (except the bsod which im sure its a seabios thing), perhaps he knows more since i dont have an intel board to test

I just managed to get it working (using secondary graphics)

qemu-system-x86_64 -enable-kvm -m 4096 -cpu host \
-smp 2 \
-bios /usr/share/qemu/bios.bin \
-device pci-assign,host=01:00.0 \
-device pci-assign,host=01:00.1

For anyone interested in my hardware;
Mobo: B75-D3V, BIOS F8 04/17/2013 (The integrated LAN on this board is utter crap... maybe I have a dodgy board but i'm using an Intel e1000e PCI LAN card to avoid a mountain of problems)
CPU: i5-3570
GPU: Radeon HD 7950 Gigabyte MSI Twin Fozr III 3GB

It's a shame I can't get the BIOS level working though. If switching to an AMD board would work I would consider it well worthwhile. I just want to make sure I buy one which is confirmed to be working as "upgrading" once will cost enough xD

I compiled the sources using your scripts today so I think i'm using the latest versions.

[root@localhost ~]# qemu-system-x86_64 --version
QEMU emulator version 1.5.1, Copyright (c) 2003-2008 Fabrice Bellard

[root@localhost ~]# uname -a
Linux localhost 3.10.1-1-mainline #1 SMP PREEMPT Sun Jul 28 17:04:33 IST 2013 x86_64 GNU/Linux

Don't know how to check the seabios version, but I did use your script to build it.

I've tried using a lot of kernel param combinations but I see no difference. Now i'm using "pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on,pass-through,pt" but it doesn't look right to me! XD

Last edited by BulliteShot (2013-07-28 22:16:21)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, I'm getting the same problem as a guy a few pages back... "This device can not find enough free resources that it can use. (Code 12)"... apparently something to do with using the intel integrated graphics. I read the post that was linked but it seems like a dead end?

- I can't get the graphics card to show the BIOS.
- If I boot the ArchLinux ISO in the VM then it shows it's loading screen on the graphics card.
- Windows shows the device in task manager and AMD installs the catalyst drivers.

[root@localhost ~]# dmesg | grep IOMMU
[    0.000000] Intel-IOMMU: enabled
[    0.021443] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.021447] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.021516] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.297980] IOMMU 0 0xfed90000: using Queued invalidation
[    0.297981] IOMMU 1 0xfed91000: using Queued invalidation
[    0.436381] IOMMU: software identity mapping for device 0000:00:00.0
[    0.436382] IOMMU: software identity mapping for device 0000:00:01.0
[    0.436383] IOMMU: software identity mapping for device 0000:00:14.0
[    0.436384] IOMMU: software identity mapping for device 0000:00:16.0
[    0.436385] IOMMU: software identity mapping for device 0000:00:1a.0
[    0.436386] IOMMU: software identity mapping for device 0000:00:1b.0
[    0.436387] IOMMU: software identity mapping for device 0000:00:1c.0
[    0.436388] IOMMU: software identity mapping for device 0000:00:1c.2
[    0.436389] IOMMU: software identity mapping for device 0000:00:1d.0
[    0.436390] IOMMU: software identity mapping for device 0000:00:1f.0
[    0.436391] IOMMU: software identity mapping for device 0000:00:1f.2
[    0.436392] IOMMU: software identity mapping for device 0000:00:1f.3
[    0.436396] IOMMU: software identity mapping for device 0000:01:00.0
[    0.436397] IOMMU: software identity mapping for device 0000:01:00.1
[    0.436401] IOMMU: software identity mapping for device 0000:02:00.0
[    0.436402] IOMMU: Setting RMRR:
[    0.436410] IOMMU: Setting identity map for device 0000:00:02.0 [0xcb800000 - 0xcf9fffff]
[    0.436696] IOMMU: Setting identity map for device 0000:00:1d.0 [0xc9d82000 - 0xc9d8efff]
[    0.436701] IOMMU: Setting identity map for device 0000:00:1a.0 [0xc9d82000 - 0xc9d8efff]
[    0.436702] IOMMU: Setting identity map for device 0000:00:14.0 [0xc9d82000 - 0xc9d8efff]
[    0.436704] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.436705] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[root@localhost ~]# 

[root@localhost ~]# dmesg | grep pci-stub
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=/dev/md1 ro quiet pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on iommu=pt
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-linux-mainline root=/dev/md1 ro quiet pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on iommu=pt
[    0.705428] pci-stub: add 1002:679A sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.705438] pci-stub 0000:01:00.0: claimed by stub
[    0.705442] pci-stub: add 1002:AAA0 sub=FFFFFFFF:FFFFFFFF cls=00000000/00000000
[    0.705448] pci-stub 0000:01:00.1: claimed by stub
[root@localhost ~]# 

[   60.412232] vfio-pci 0000:01:00.0: enabling device (0000 -> 0003)
[   60.438127] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x19@0x270
[   60.438134] vfio_ecap_init: 0000:01:00.0 hiding ecap 0x1b@0x2d0

qemu-system-x86_64 -enable-kvm -M q35 -m 4096 -cpu host \
-smp 2 \
-bios /usr/share/qemu/bios.bin \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1 \
-device virtio-scsi-pci,id=scsi \
-drive if=none,format=raw,file=/root/win_os.raw,id=disk \
-device scsi-hd,drive=disk \
-net nic,model=virtio \
-net tap,ifname=vm-win,script=no,downscript=no

Are you using the lastest builds i posted?  iommu=pt is an amd thing only (i think for intel is intel_iommu=pt) and i have no idea what igfx_on does, it seems myweb has partially got it to work on his intel board (except the bsod which im sure its a seabios thing), perhaps he knows more since i dont have an intel board to test

I just managed to get it working (using secondary graphics)

qemu-system-x86_64 -enable-kvm -m 4096 -cpu host \
-smp 2 \
-bios /usr/share/qemu/bios.bin \
-device pci-assign,host=01:00.0 \
-device pci-assign,host=01:00.1

For anyone interested in my hardware;
Mobo: B75-D3V, BIOS F8 04/17/2013 (The integrated LAN on this board is utter crap... maybe I have a dodgy board but i'm using an Intel e1000e PCI LAN card to avoid a mountain of problems)
CPU: i5-3570
GPU: Radeon HD 7950 Gigabyte MSI Twin Fozr III 3GB

It's a shame I can't get the BIOS level working though. If switching to an AMD board would work I would consider it well worthwhile. I just want to make sure I buy one which is confirmed to be working as "upgrading" once will cost enough xD

I compiled the sources using your scripts today so I think i'm using the latest versions.

[root@localhost ~]# qemu-system-x86_64 --version
QEMU emulator version 1.5.1, Copyright (c) 2003-2008 Fabrice Bellard

[root@localhost ~]# uname -a
Linux localhost 3.10.1-1-mainline #1 SMP PREEMPT Sun Jul 28 17:04:33 IST 2013 x86_64 GNU/Linux

Don't know how to check the seabios version, but I did use your script to build it.

I've tried using a lot of kernel param combinations but I see no difference. Now i'm using "pci-stub.ids=1002:679a,1002:aaa0 intel_iommu=on,igfx_on,pass-through,pt" but it doesn't look right to me! XD

Yes, passing through a radeon card as secondary should work (wont work with nvidia though) but'll probably experience performance degradation/host crashes, if you do, you can solve it using the "eject" method i posted in the guide

Last edited by nbhs (2013-07-28 22:22:27)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@BulliteShot
I also just realized you forgot to add "-vga none" using vfio, it wont work without it, also just use intel_iommu=on

Last edited by nbhs (2013-07-28 22:26:10)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The acpi bios thing its because of seabios, use my package (1.7.2 patched), the other message is likely because you're using the wrong kernel without pcie bus reset support

Yes, You are right - I have no BSOD with 1.7.2 but I could not apply kernel-vfio-vga-reset.patch to Linux kernel from vga-current branch: I get

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

You dont need to patch that kernel, i extracted the commits from that branch and use it against vanilla kernel, and it probably doesnt work anymore after you reboot the vm because you're using the wrong kernel (it cant reset the card), i posted a qemu and kernel builds for you on page 13 you can use that or build it yourself

I use kernel and qemu compiled from your builds posted on page 13. Also I get the same behaviour (black screen after windows reboot) with kernel and qemu compiled from builds posted on the first page: I did not get any warnings about reset, but saw black screen when windows reboot.

Last edited by myweb (2013-07-28 22:40:20)

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@nbhs
Yes, the host does freeze when the VM is rebooted. I tried the eject script you provided and it works. Thank you.

I just tried rebooted to try only "intel_iommu=on" and adding "-vga none". Unfortunately I just see the Qemu console and no input from the graphics card.

Going to try using the GPU ROM now. I tried it earlier but I can't remember if I used -vga none"

Last edited by BulliteShot (2013-07-28 22:38:54)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yes, You are right - I have no BSOD with 1.7.2 but I could not apply kernel-vfio-vga-reset.patch to Linux kernel from vga-current branch: I get

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

You dont need to patch that kernel, i extracted the commits from that branch and use it against vanilla kernel, and it probably doesnt work anymore after you reboot the vm because you're using the wrong kernel (it cant reset the card), i posted a qemu and kernel builds for you on page 13 you can use that or build it yourself

I use kernel and qemu compiled from your builds posted on page 13. Also I get the same behaviour (black screen after windows reboot) with kernel and qemu builder from builds posted on the first page: I did not get any warnings about reset, but saw black screen when windows reboot.

have you tried:

modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@nbhs
Yes, the host does freeze when the VM is rebooted. I tried the eject script you provided and it works. Thank you.

I just tried rebooted to try only "intel_iommu=on" and adding "-vga none". Unfortunately I just see the Qemu console and no input from the graphics card.

Going to try using the GPU ROM now. I tried it earlier but I can't remember if I used -vga none"

well there's not much benefits from passing it through as primary or secondary other than seeing the seabios boot messages and having to use the eject script thingy

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@Myweb
You could also try loading the bios rom from a dump too

Last edited by nbhs (2013-07-28 22:47:32)

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You dont need to patch that kernel, i extracted the commits from that branch and use it against vanilla kernel, and it probably doesnt work anymore after you reboot the vm because you're using the wrong kernel (it cant reset the card), i posted a qemu and kernel builds for you on page 13 you can use that or build it yourself

I use kernel and qemu compiled from your builds posted on page 13. Also I get the same behaviour (black screen after windows reboot) with kernel and qemu builder from builds posted on the first page: I did not get any warnings about reset, but saw black screen when windows reboot.

have you tried:

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

I have tried, but it does not help.

modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

@Myweb
You could also try loading the bios rom from a dump too

Do you mean VGA bios (romfile)?

Is it correct, that at first I need to fix: Attempt to reset PCI bus for VGA support failed (Device or resource busy).  VGA may not work.?

Last edited by myweb (2013-07-28 22:56:14)

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@nbhs
Yes, here is my script running on root.

vfio-bind 0000:01:00.0 0000:01:00.1
modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

The reason I am after primary pass-through is because I'm waiting for Windows to BSOD in the background, reset and then crash the entire system >_< I suppose I can disable automatic rebooting when a BSOD happens but I don't know how reliable that is.

If anyone else has a 7950 running on primary pass-through please let me know what CPU & Motherboard you're using.

By the way - anyone who is using a KVM or the cirrus VGA to control the mouse and keyboard take a look at Synergy. It works using the network and you can define a hotkey to switch between hosts and copy the clipboard.

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It looks like I was wrong with description of issue:
Windows installation starts, copy files, reboot VM, Windows installation continue, I see message Getting device ready 0...100% and then screen switched to black.
Using VGA bios (romfile) does not help.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I use kernel and qemu compiled from your builds posted on page 13. Also I get the same behaviour (black screen after windows reboot) with kernel and qemu builder from builds posted on the first page: I did not get any warnings about reset, but saw black screen when windows reboot.

have you tried:

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

I have tried, but it does not help.

modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

@Myweb
You could also try loading the bios rom from a dump too

Do you mean VGA bios (romfile)?

Is it correct, that at first I need to fix: Attempt to reset PCI bus for VGA support failed (Device or resource busy).  VGA may not work.?

Yes your vga romfile, could you post your uname -a? using the packages posted on the first page does not produce this warning message correct?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It looks like I was wrong with description of issue:
Windows installation starts, copy files, reboot VM, Windows installation continue, I see message Getting device ready 0...100% and then screen switched to black.
Using VGA bios (romfile) does not help.

Oh, have you tried to install it first without vga passthrough?

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

have you tried:

Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp.h.rej
patching file drivers/pci/hotplug/pciehp_core.c
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file drivers/pci/hotplug/pciehp_core.c.rej
patching file drivers/pci/hotplug/pciehp_hpc.c
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file drivers/pci/hotplug/pciehp_hpc.c.rej
patching file drivers/pci/pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
5 out of 5 hunks ignored -- saving rejects to file drivers/pci/pci.c.rej
patching file drivers/vfio/pci/vfio_pci.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/pci/vfio_pci.c.rej
patching file drivers/vfio/vfio.c
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file drivers/vfio/vfio.c.rej
patching file include/linux/pci.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/pci.h.rej
patching file include/linux/pci_hotplug.h
Reversed (or previously applied) patch detected!  Skipping patch.
3 out of 3 hunks ignored -- saving rejects to file include/linux/pci_hotplug.h.rej
patching file include/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/linux/vfio.h.rej
patching file include/uapi/linux/vfio.h
Reversed (or previously applied) patch detected!  Skipping patch.
2 out of 2 hunks ignored -- saving rejects to file include/uapi/linux/vfio.h.rej
==> ERROR: A failure occurred in prepare().
    Aborting...

I have tried, but it does not help.

modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

@Myweb
You could also try loading the bios rom from a dump too

Do you mean VGA bios (romfile)?

Is it correct, that at first I need to fix: Attempt to reset PCI bus for VGA support failed (Device or resource busy).  VGA may not work.?

Yes your vga romfile, could you post your uname -a? using the packages posted on the first page does not produce this warning message correct?

uname: 3.11.0-1-mainline #1 SMP PREEMPT Sun Jul 28 16:25:59 UTC 2013 x86_64 GNU/Linux
using the packages posted on the first page does not produce this warning message - this is correct

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@nbhs
Yes, here is my script running on root.

vfio-bind 0000:01:00.0 0000:01:00.1
modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

The reason I am after primary pass-through is because I'm waiting for Windows to BSOD in the background, reset and then crash the entire system >_< I suppose I can disable automatic rebooting when a BSOD happens but I don't know how reliable that is.

If anyone else has a 7950 running on primary pass-through please let me know what CPU & Motherboard you're using.

By the way - anyone who is using a KVM or the cirrus VGA to control the mouse and keyboard take a look at Synergy. It works using the network and you can define a hotkey to switch between hosts and copy the clipboard.

Wait i dont understand you're trying to get windows to BSOD?

myweb

Member

Registered: 2013-07-13

Posts: 69

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It looks like I was wrong with description of issue:
Windows installation starts, copy files, reboot VM, Windows installation continue, I see message Getting device ready 0...100% and then screen switched to black.
Using VGA bios (romfile) does not help.

Oh, have you tried to install it first without vga passthrough?

Yes, but when I run VM with vga passthrough specified Windows can't boot and asks to repair. Repair is failed due to it cant find a partition (I gues due to virtio-blk using)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

It looks like I was wrong with description of issue:
Windows installation starts, copy files, reboot VM, Windows installation continue, I see message Getting device ready 0...100% and then screen switched to black.
Using VGA bios (romfile) does not help.

Oh, have you tried to install it first without vga passthrough?

Yes, but when I run VM with vga passthrough specified Windows can't boot and asks to repair. Repair is failed due to it cant find a partition (I gues due to virtio-blk using)

Have you tried the ide or ahci controller?

mukiex

Member

Registered: 2013-07-27

Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Could we have a separate thread to focus on building vfio-reset? This opening post is AMAZINGLY comprehensive for getting things up and running, and it almost feels like the building process would be better served a separate discussion.

nbhs : Found your 3.9 building package: http://www.filesend.net/download.php?f= … 468af53600   Thanks!

Last edited by mukiex (2013-07-29 10:08:45)

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@nbhs
Yes, here is my script running on root.

vfio-bind 0000:01:00.0 0000:01:00.1
modprobe -r kvm_intel
modprobe kvm_intel emulate_invalid_guest_state=0

The reason I am after primary pass-through is because I'm waiting for Windows to BSOD in the background, reset and then crash the entire system >_< I suppose I can disable automatic rebooting when a BSOD happens but I don't know how reliable that is.

If anyone else has a 7950 running on primary pass-through please let me know what CPU & Motherboard you're using.

By the way - anyone who is using a KVM or the cirrus VGA to control the mouse and keyboard take a look at Synergy. It works using the network and you can define a hotkey to switch between hosts and copy the clipboard.

Wait i dont understand you're trying to get windows to BSOD?

No, what I mean is that the eject script is only going to run when the VM is cleanly shutdown.

If windows crashes or resets due to a BSOD then it's going to freeze the host when it boots up again.

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

linux-mainline.tar.gz (3.10.1)
qemu.tar.gz (1.5.1)
Or
Qemu 1.5.1 patch: qemu-vfio-vga-reset.patch.tar.gz
Kernel 3.10.1 patch: kernel-vfio-vga-reset.patch.tar.gz

I'm using the top 2 files. What's the difference between the top 2 and the bottom 2 files?

mukiex

Member

Registered: 2013-07-27

Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

linux-mainline.tar.gz (3.10.1)
qemu.tar.gz (1.5.1)
Or
Qemu 1.5.1 patch: qemu-vfio-vga-reset.patch.tar.gz
Kernel 3.10.1 patch: kernel-vfio-vga-reset.patch.tar.gz

I'm using the top 2 files. What's the difference between the top 2 and the bottom 2 files?

Ah ha! I had the same thing!

One of them ONLY contains the patch. Advanced users only, I guess.

The other one is patch/PKGbuild setup. That's probably the one you want. You essentially hop in, run "makepkg".

If any dependencies are missing, you pacman -S them. Eventually you'll a .zx file with the package name. You'll want to pacman -U that file.
I'm really excited here. I really wanna write an uber-noob guide to get this going

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Ok so I think I installed the right package because I did do the makepkg route. Are these Alex Williamson's branches?

Last edited by BulliteShot (2013-07-29 13:02:17)

mukiex

Member

Registered: 2013-07-27

Posts: 18

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I can't win. I got qemu, Seabios, and the kernel built with reset support. Still the same issue. Radeon never initializes unless it's the primary. Is this a motherboard issue? Would there be a way to fake it by having Linux send an image to it beforehand?

BulliteShot

Member

Registered: 2013-07-28

Posts: 27

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Sounds like you're having the opposite problem i'm having. I can only get my radon card to work as a secondary adapter using pci-attach (but first you have to enable cirrus VGA and then install the catalyst drivers... after rebooting, the cirrus VGA will stop responding at the windows boot logo and then you'll the graphics card will take over)

Last edited by BulliteShot (2013-07-29 17:08:03)