Arch Linux

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thank you for your help, it unfortunately however gave me the exact same error again. so i tried replacing my dtc with dtc-git, and again got the exact same error again. I uninstalled dtc and it compiles just fine.(thanks to you i knew dtc was the problem)

I initially tried to install qemu-git from aur like you suggested in your guide, which then lead me to the conflict with seabios (which i assume started with v 1.6 of qemu). one of the dependencies for qemu-git was dtc, that's how i got it. (i didn't install dtc-git, but just dtc. I in fact don't know the difference between git and not git, do you have a link to somewhere that can explain that to me? i try to avoid git builds because i just don't know what they are nor if to handle them differently than any other packages)

I like this fileswap thing you're using, so much better to download from than that filesend thing, since you don't have to wait for a download button

Anyway, i'll try to use that last qemu build you sent me (without dtc installed) and the seabios files you sent as-well, tomorrow i'll try to learn how to use qemu to begin with since i never have (only virtualbox, vmware and microsofts thing) and after that we'll see how my attempt to passthrough will go! i'll get back to you when i get that far. Before i try though, i have a quick question or 2, do i need two monitors to do this? will it run windows 8? (i want to use it for it's performance over 7, otherwise i hate it's very existance, i've heard reports of it not working with qemu)

Last edited by rabcor (2013-06-22 05:49:26)

---

Making Logitech G510 Work on Linux!

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thank you for your help, it unfortunately however gave me the exact same error again. so i tried replacing my dtc with dtc-git, and again got the exact same error again. I uninstalled dtc and it compiles just fine.(thanks to you i knew dtc was the problem)

I initially tried to install qemu-git from aur like you suggested in your guide, which then lead me to the conflict with seabios (which i assume started with v 1.6 of qemu). one of the dependencies for qemu-git was dtc, that's how i got it. (i didn't install dtc-git, but just dtc. I in fact don't know the difference between git and not git, do you have a link to somewhere that can explain that to me? i try to avoid git builds because i just don't know what they are nor if to handle them differently than any other packages)

I like this fileswap thing you're using, so much better to download from than that filesend thing, since you don't have to wait for a download button

Anyway, i'll try to use that last qemu build you sent me (without dtc installed) and the seabios files you sent as-well, tomorrow i'll try to learn how to use qemu to begin with since i never have (only virtualbox, vmware and microsofts thing) and after that we'll see how my attempt to passthrough will go! i'll get back to you when i get that far. Before i try though, i have a quick question or 2, do i need two monitors to do this? will it run windows 8? (i want to use it for it's performance over 7, otherwise i hate it's very existance, i've heard reports of it not working with qemu)

yes it will run windows 8, im using it atm

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't know how to be able to tell if my board supports Interrupt Remapping

but when i try to run vfio-bind it says

# vfio-bind 0000:01:00.0
ls: cannot access /sys/bus/pci/devices/0000:01:00.0/iommu_group/devices: No such file or directory

if i run

# ls /sys/bus/pci/devices/0000\:01\:00.0/

(yes it requests "\"s like as if there are spaces between, i don't know why)
it doesn't give me any iommu directory so i assume this is because i didn't know how to add

vfio_iommu_type1.allow_unsafe_interrupts=1

to my grub.cfg, i did some googling but didn't find out how, i tried to add it myself in a place that looked about right but it did nothing

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-linux root=UUID=876f6b61-7387-489e-b980-d42deb194c34 ro quiet

where am i supposed to add the line?

---

Making Logitech G510 Work on Linux!

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't know how to be able to tell if my board supports Interrupt Remapping

but when i try to run vfio-bind it says

# vfio-bind 0000:01:00.0
ls: cannot access /sys/bus/pci/devices/0000:01:00.0/iommu_group/devices: No such file or directory

if i run

# ls /sys/bus/pci/devices/0000\:01\:00.0/

(yes it requests "\"s like as if there are spaces between, i don't know why)
it doesn't give me any iommu directory so i assume this is because i didn't know how to add

vfio_iommu_type1.allow_unsafe_interrupts=1

to my grub.cfg, i did some googling but didn't find out how, i tried to add it myself in a place that looked about right but it did nothing

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-linux root=UUID=876f6b61-7387-489e-b980-d42deb194c34 ro quiet

where am i supposed to add the line?

echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/vfio_iommu_type1.conf

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I had already done that, and the kvm thing too. think the problem might be that my proprietary nvidia drivers are running? (should i uninstall them?)

---

Making Logitech G510 Work on Linux!

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I had already done that, and the kvm thing too. think the problem might be that my proprietary nvidia drivers are running? (should i uninstall them?)

There's a patch for them, but using nouveau works too.

---

i'm sorry for my poor english wirting skills…

dogblue

Member

Registered: 2013-06-23

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

EDIT: This issue was fixed by blacklisting the nouveau module.

Hi all,

I have spent most of the last couple of days trying to get this to work, with no luck so far.
As soon as I launch the virtual machine my entire computer will freeze and require a cold reboot.

Hardware is ASRock 970 Extreme3 motherboard, AMD FX CPU, NVidia GT 520 GPU (Primary, nouveau), AMD HD 7750 GPU (Secondary, passthrough).

I am using the AUR/ABS packages that nbhs posted a page or two ago.

I get an error on boot regarding the ivrs_ioapic lines

/bin/init: eval: line 1: ivrs_ioapic[9]=00:14.0: not found
/bin/init: eval: line 1: ivrs_ioapic[10]=00:00.1: not found

I've attached as much additional information to this post as I can in the hope that someone will be able to offer a suggestion as to how I should proceed, as I am completely out of ideas.

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=8ff5d0cb-77d4-403f-9387-e04405a329e7 ro iommu=1 ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1 acpi_sleep=s3_bios,s3_mode pci-stub.ids=1002:683f,1002:aab0 vfio_iommu_type1.allow_unsafe_interrupts=1

lspci -v (abridged output)

01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 520] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd Device 352f
	Flags: bus master, fast devsel, latency 0, IRQ 24
	Memory at fd000000 (32-bit, non-prefetchable) [size=16M]
	Memory at d0000000 (64-bit, prefetchable) [size=128M]
	Memory at d8000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at fe000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100] Virtual Channel
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: nouveau
	Kernel modules: nouveau

02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750] (prog-if 00 [VGA controller])
	Subsystem: Hightech Information System Ltd. Device 2318
	Flags: fast devsel, IRQ 10
	Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Memory at fe200000 (64-bit, non-prefetchable) [disabled] [size=256K]
	I/O ports at d000 [disabled] [size=256]
	Expansion ROM at fe240000 [disabled] [size=128K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Capabilities: [270] #19
	Kernel driver in use: pci-stub
	Kernel modules: radeon

02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
	Subsystem: Hightech Information System Ltd. Device aab0
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at fe260000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: pci-stub
	Kernel modules: snd_hda_intel

I am attempting to start the VM with the following script. The machine displays a black qemu window and then halts (the whole computer, not just the VM) as soon as I run it.

#!/bin/sh
#DEVICES="0000:00:11.0 0000:04:00.0 0000:05:00.0 0000:06:00.0 0000:07:00.0 000:07:00.1"
DEVICES="0000:02:00.0 0000:02:00.1"
./vfio-bind $DEVICES
qemu-system-x86_64 --enable-kvm -M q35 -m 512 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=02:00.1,bus=root.1,addr=00.1

I would really appreciate any help that you all can offer. I actually purchased this machine a few days ago specifically to follow this guide, so I'm really looking forward to getting it all working!

Thanks in advance.

Last edited by dogblue (2013-06-24 21:35:23)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi all,

I have spent most of the last couple of days trying to get this to work, with no luck so far.
As soon as I launch the virtual machine my entire computer will freeze and require a cold reboot.

Hardware is ASRock 970 Extreme3 motherboard, AMD FX CPU, NVidia GT 520 GPU (Primary, nouveau), AMD HD 7750 GPU (Secondary, passthrough).

I am using the AUR/ABS packages that nbhs posted a page or two ago.

I get an error on boot regarding the ivrs_ioapic lines

/bin/init: eval: line 1: ivrs_ioapic[9]=00:14.0: not found
/bin/init: eval: line 1: ivrs_ioapic[10]=00:00.1: not found

I've attached as much additional information to this post as I can in the hope that someone will be able to offer a suggestion as to how I should proceed, as I am completely out of ideas.

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-linux-mainline root=UUID=8ff5d0cb-77d4-403f-9387-e04405a329e7 ro iommu=1 ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1 acpi_sleep=s3_bios,s3_mode pci-stub.ids=1002:683f,1002:aab0 vfio_iommu_type1.allow_unsafe_interrupts=1

lspci -v (abridged output)

01:00.0 VGA compatible controller: NVIDIA Corporation GF119 [GeForce GT 520] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd Device 352f
	Flags: bus master, fast devsel, latency 0, IRQ 24
	Memory at fd000000 (32-bit, non-prefetchable) [size=16M]
	Memory at d0000000 (64-bit, prefetchable) [size=128M]
	Memory at d8000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at fe000000 [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [b4] Vendor Specific Information: Len=14 <?>
	Capabilities: [100] Virtual Channel
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Kernel driver in use: nouveau
	Kernel modules: nouveau

02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde PRO [Radeon HD 7750] (prog-if 00 [VGA controller])
	Subsystem: Hightech Information System Ltd. Device 2318
	Flags: fast devsel, IRQ 10
	Memory at c0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Memory at fe200000 (64-bit, non-prefetchable) [disabled] [size=256K]
	I/O ports at d000 [disabled] [size=256]
	Expansion ROM at fe240000 [disabled] [size=128K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Capabilities: [270] #19
	Kernel driver in use: pci-stub
	Kernel modules: radeon

02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series]
	Subsystem: Hightech Information System Ltd. Device aab0
	Flags: bus master, fast devsel, latency 0, IRQ 11
	Memory at fe260000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [48] Vendor Specific Information: Len=08 <?>
	Capabilities: [50] Power Management version 3
	Capabilities: [58] Express Legacy Endpoint, MSI 00
	Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
	Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
	Capabilities: [150] Advanced Error Reporting
	Kernel driver in use: pci-stub
	Kernel modules: snd_hda_intel

I am attempting to start the VM with the following script. The machine displays a black qemu window and then halts (the whole computer, not just the VM) as soon as I run it.

#!/bin/sh
#DEVICES="0000:00:11.0 0000:04:00.0 0000:05:00.0 0000:06:00.0 0000:07:00.0 000:07:00.1"
DEVICES="0000:02:00.0 0000:02:00.1"
./vfio-bind $DEVICES
qemu-system-x86_64 --enable-kvm -M q35 -m 512 -cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-bios /usr/share/qemu/bios.bin -vga none \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=02:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=02:00.1,bus=root.1,addr=00.1

I would really appreciate any help that you all can offer. I actually purchased this machine a few days ago specifically to follow this guide, so I'm really looking forward to getting it all working!

Thanks in advance.

Those eval errors are normal dont worry about it. If you're overriding the ivrs table to enable intremap you dont need this line

vfio_iommu_type1.allow_unsafe_interrupts=1

Could you post your dmesg output?

Have you blacklisted the radeon or fglrx driver first? Also if you're using nvidia proprietary drivers you'll need to patch it (see andy123 post above). I would suggest uninstalling/blacklisting all the drivers first, and try to bind the devices with the script manually, then launch qemu.

Last edited by nbhs (2013-06-24 03:18:09)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I had already done that, and the kvm thing too. think the problem might be that my proprietary nvidia drivers are running? (should i uninstall them?)

Just to make sure can you build/rebuild the linux-mainline package i posted with:

CONFIG_VFIO_IOMMU_TYPE1=y
CONFIG_VFIO=y
CONFIG_VFIO_PCI=y
CONFIG_VFIO_PCI_VGA=y

Could you be a little more specific about your hardware?, If your primary display is nvidia, and you're using nvidia proprietary drivers see andy123 post above.

dogblue

Member

Registered: 2013-06-23

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi nbhs, thanks for the suggestions (and the guide!)

Those eval errors are normal dont worry about it. If you're overriding the ivrs table to enable intremap you dont need this line

vfio_iommu_type1.allow_unsafe_interrupts=1

Understood, I've removed that kernel option and tried again.

Could you post your dmesg output?

Absolutely. http://pastebin.com/iqmLCsqq

Have you blacklisted the radeon or fglrx driver first? Also if you're using nvidia proprietary drivers you'll need to patch it (see andy123 post above).

I have blacklisted the radeon driver as well as added the id's to pci-stub. I am also using the nouveau driver for the nvidia card, so that shouldn't be an issue at least.

I would suggest uninstalling/blacklisting all the drivers first, and try to bind the devices with the script manually, then launch qemu.

As far as I can tell the radeon module is blacklisted, and fglrx and the binary nvidia drivers have never been installed.

Here is what the drivers look like just prior to running the qemu command (after binding the devices with the vfio-bind script)

# ls /sys/bus/pci/drivers/vfio-pci/
0000:02:00.0  0000:02:00.1  bind  module  new_id  remove_id  uevent  unbind
# ls /sys/bus/pci/drivers/nouveau/
0000:01:00.0  bind  module  new_id  remove_id  uevent  unbind
# cat /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
# cat /etc/modprobe.d/blacklist.conf
blacklist radeon

I was also able to retrieve the following output by running 'cat /proc/kmsg' during the crash

<6>[  996.181525] vfio-pci 0000:02:00.0: enabling device (0000 -> 0003)
<6>[  996.207159] vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
<6>[  998.065072] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem
<6>[  998.065076] vgaarb: transferring owner from PCI:0000:01:00.0 to PCI:0000:02:00.0
<3>[  998.095398] ata3: illegal qc_active transition (00000001->97b282a8)
<3>[  998.095540] ata3.00: exception Emask 0x2 SAct 0x0 SErr 0xd0000000 action 0x6 frozen
<3>[  998.095567] ata3: SError: { }
<3>[  998.095580] ata3.00: failed command: FLUSH CACHE EXT
<3>[  998.095607] ata3.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
<3>[  998.095607]          res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x2 (HSM violation)
<3>[  998.095637] ata3.00: status: { DRDY }
<6>[  998.095653] ata3: hard resetting link
<4>[  998.103575] Clocksource tsc unstable (delta = -147227892866 ns)

Thanks again for your help!

Last edited by dogblue (2013-06-24 12:18:50)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi nbhs, thanks for the suggestions (and the guide!)

Those eval errors are normal dont worry about it. If you're overriding the ivrs table to enable intremap you dont need this line

vfio_iommu_type1.allow_unsafe_interrupts=1

Understood, I've removed that kernel option and tried again.

Could you post your dmesg output?

Absolutely. http://pastebin.com/iqmLCsqq

Have you blacklisted the radeon or fglrx driver first? Also if you're using nvidia proprietary drivers you'll need to patch it (see andy123 post above).

I have blacklisted the radeon driver as well as added the id's to pci-stub. I am also using the nouveau driver for the nvidia card, so that shouldn't be an issue at least.

I would suggest uninstalling/blacklisting all the drivers first, and try to bind the devices with the script manually, then launch qemu.

As far as I can tell the radeon module is blacklisted, and fglrx and the binary nvidia drivers have never been installed.

Here is what the drivers look like just prior to running the qemu command (after binding the devices with the vfio-bind script)

# ls /sys/bus/pci/drivers/vfio-pci/
0000:02:00.0  0000:02:00.1  bind  module  new_id  remove_id  uevent  unbind
# ls /sys/bus/pci/drivers/nouveau/
0000:01:00.0  bind  module  new_id  remove_id  uevent  unbind
# cat /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
# cat /etc/modprobe.d/blacklist.conf
blacklist radeon

I was also able to retrieve the following output by running 'cat /proc/kmsg' during the crash

<6>[  996.181525] vfio-pci 0000:02:00.0: enabling device (0000 -> 0003)
<6>[  996.207159] vfio_ecap_init: 0000:02:00.0 hiding ecap 0x19@0x270
<6>[  998.065072] vgaarb: device changed decodes: PCI:0000:01:00.0,olddecodes=io+mem,decodes=none:owns=io+mem
<6>[  998.065076] vgaarb: transferring owner from PCI:0000:01:00.0 to PCI:0000:02:00.0
<3>[  998.095398] ata3: illegal qc_active transition (00000001->97b282a8)
<3>[  998.095540] ata3.00: exception Emask 0x2 SAct 0x0 SErr 0xd0000000 action 0x6 frozen
<3>[  998.095567] ata3: SError: { }
<3>[  998.095580] ata3.00: failed command: FLUSH CACHE EXT
<3>[  998.095607] ata3.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
<3>[  998.095607]          res 40/00:00:00:00:00/00:00:00:00:00/a0 Emask 0x2 (HSM violation)
<3>[  998.095637] ata3.00: status: { DRDY }
<6>[  998.095653] ata3: hard resetting link
<4>[  998.103575] Clocksource tsc unstable (delta = -147227892866 ns)

Thanks again for your help!

Have you tried to blacklist the nouveau module and start without X? take a look at this post, it seems to describe your problem: http://www.mail-archive.com/qemu-devel@ … 73628.html,

2) If the 'nouveau.ko' driver is loaded it is even more strange. As soon
as I start qemu all my SATA links get a hard reset and kernel freezes.
No SysRQs are working anymore and only reboot helps. If needed I can
look if I can get some dumps from this freeze because it writes nothing
more to the disks.

It seems he solved the problem using patched nvidia proprietary drivers as described here: http://www.mail-archive.com/qemu-devel@ … 74066.html

dogblue

Member

Registered: 2013-06-23

Posts: 5

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Have you tried to blacklist the nouveau module and start without X? take a look at this post, it seems to describe your problem: http://www.mail-archive.com/qemu-devel@ … 73628.html,

That was it!
Blacklisting nouveau fixed it. No more crashes on starting the VM.

Thank you so much for all your help!!

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Have you tried to blacklist the nouveau module and start without X? take a look at this post, it seems to describe your problem: http://www.mail-archive.com/qemu-devel@ … 73628.html,

That was it!
Blacklisting nouveau fixed it. No more crashes on starting the VM.

Thank you so much for all your help!!

NIce! have you tried the patched nvidia drivers yet?

blackout23

Member

Registered: 2011-11-16

Posts: 781

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I just wanted to thank you for this great thread and all the effort you put into it!

I have looked at VGA-Passthrough with Xen in the past but this Wiki article here has diminished my hopes a bit. What makes me wonder is why it says "400 Series and 600 Series should work" after some hacking and only on Windows XP. Since I have a GTX 580 OC I wonder why the 500 series was excluded in this wiki. Maybe this means I can't use it with your KVM tutorial, too?

My rig currently consists of a Intel i7 2600K @ 4.5 Ghz, 8 GB RAM, ASUS P8Z86-V Pro and GTX 580 @ 900 Mhz. If no one has ever tried a 500 series card I might have to do that and report back.

Last edited by blackout23 (2013-06-25 11:30:39)

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to make sure can you build/rebuild the linux-mainline package i posted with:...

Could you be a little more specific about your hardware?, If your primary display is nvidia, and you're using nvidia proprietary drivers see andy123 post above.

Yosh! i got it right!

My hardware setup is intel igpu for linux and nvidia gtx 670 for passthrough, my motherboard supports this and it's turned on, and i assume my i7 does as-well.

I don't know if it was setting these settings to Y or if it was setting

CONFIG_INTEL_IOMMU_DEFAULT_ON=y

that did it. but it worked and vfio-bind worked on 0000:01:00.0 and 1

#lspci
...
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 670] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
...

now my next problem is

# qemu-system-x86_64 --enable-kvm -M q35 -m 1024 -cpu host \ -smp 6,sockets=1,cores=6,threads=1 \ -bios /usr/share/qemu/bios.bin -vga none \ -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \ -device vfio-pci, host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

Gives me

qemu-system-x86_64: -cpu host: could not open disk image  -smp: No such file or directory

Last edited by rabcor (2013-06-26 05:45:54)

---

Making Logitech G510 Work on Linux!

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Just to make sure can you build/rebuild the linux-mainline package i posted with:...

Could you be a little more specific about your hardware?, If your primary display is nvidia, and you're using nvidia proprietary drivers see andy123 post above.

Yosh! i got it right!

My hardware setup is intel igpu for linux and nvidia gtx 670 for passthrough, my motherboard supports this and it's turned on, and i assume my i7 does as-well.

I don't know if it was setting these settings to Y or if it was setting

CONFIG_INTEL_IOMMU_DEFAULT_ON=y

that did it. but it worked and vfio-bind worked on 0000:01:00.0 and 1

#lspci
...
01:00.0 VGA compatible controller: NVIDIA Corporation GK104 [GeForce GTX 670] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GK104 HDMI Audio Controller (rev a1)
...

now my next problem is

# qemu-system-x86_64 --enable-kvm -M q35 -m 1024 -cpu host \ 
-smp 6, sockets=1, cores=6, threads=1 \ 
-bios /usr/share/qemu/bios.bin -vga none \ 
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \ 
-device vfio-pci, host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ 
-device vfio-pci,host=01:00.1,bus=root.1,addr=00.1

Gives me

qemu-system-x86_64: -cpu host: could not open disk image  -smp: No such file or directory

i think it is this line

-smp 6, sockets=1, cores=6, threads=1

to:

-smp 6,sockets=1,cores=6,threads=1  

You should know that, from previous users, using the igp as primary might not work

Last edited by nbhs (2013-06-25 15:33:59)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I just wanted to thank you for this great thread and all the effort you put into it!

I have looked at VGA-Passthrough with Xen in the past but this Wiki article here has diminished my hopes a bit. What makes me wonder is why it says "400 Series and 600 Series should work" after some hacking and only on Windows XP. Since I have a GTX 580 OC I wonder why the 500 series was excluded in this wiki. Maybe this means I can't use it with your KVM tutorial, too?

My rig currently consists of a Intel i7 2600K @ 4.5 Ghz, 8 GB RAM, ASUS P8Z86-V Pro and GTX 580 @ 900 Mhz. If no one has ever tried a 500 series card I might have to do that and report back.

I think it means no one has tried it yet since this is experimental.

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

i think it is this line

-smp 6, sockets=1, cores=6, threads=1

to:

-smp 6,sockets=1,cores=6,threads=1  

You should know that, from previous users, using the igp as primary might not work

I tried that, it made no difference, i got the same error.

Even if it may not work with igp as primary, i'm sure it should be possible to make it past an error like that (if we can figure out the cause). I have no idea how to troubleshoot this, i don't even know what the error means to tell the truth. But if it's caused by the igp i will probably just buy another gpu later for this.

Last edited by rabcor (2013-06-25 18:05:21)

---

Making Logitech G510 Work on Linux!

alpha_one_x86

Member

Registered: 2013-06-26

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,
My kernel line used:
/kernel-genkernel-x86_64-3.9.6-gentoo root=/dev/sda2 ro radeon.audio=1 debug kvm-intel.nested=1 iommu=1 pci-stub.ids=1002:68f9,1002:aa68 ivrs_ioapic[9]=00:14.0
(You don't say how/why you use ivrs_ioapic, isn't it?).
I have an intel card (for my host), and an ATI 5450 for windows. I have uninstall all the radeon kernel module, but I have:
-device vfio-pci,host=01:00.0,addr=00.0,multifunction=on: PCI: slot 0 function 0 not available for vfio-pci, in use by i440FX
If you can help me, cheer,

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

i think it is this line

-smp 6, sockets=1, cores=6, threads=1

to:

-smp 6,sockets=1,cores=6,threads=1  

You should know that, from previous users, using the igp as primary might not work

I tried that, it made no difference, i got the same error.

Even if it may not work with igp as primary, i'm sure it should be possible to make it past an error like that (if we can figure out the cause). I have no idea how to troubleshoot this, i don't even know what the error means to tell the truth. But if it's caused by the igp i will probably just buy another gpu later for this.

I think its this typo i overlooked before

--enable-kvm

change to:

-enable-kvm

Right now AMD mobo/cpu users have like 90% chance of getting this to work, even mixing nvidia/amd cards and with broken ivrs tables, remember this is work in progress, so i wouldn't waste money yet only to find it doesn't work, but im prety sure it will in the near future.

Last edited by nbhs (2013-06-26 03:24:47)

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hello,
My kernel line used:
/kernel-genkernel-x86_64-3.9.6-gentoo root=/dev/sda2 ro radeon.audio=1 debug kvm-intel.nested=1 iommu=1 pci-stub.ids=1002:68f9,1002:aa68 ivrs_ioapic[9]=00:14.0
(You don't say how/why you use ivrs_ioapic, isn't it?).

If you're using an intel board you shouldnt be using ivrs_ioapic override, this is a workarround for amd boards

I have an intel card (for my host), and an ATI 5450 for windows. I have uninstall all the radeon kernel module, but I have:
-device vfio-pci,host=01:00.0,addr=00.0,multifunction=on: PCI: slot 0 function 0 not available for vfio-pci, in use by i440FX
If you can help me, cheer,

Yeah change this:

-device vfio-pci,host=01:00.0,addr=00.0,multifunction=on

to this:

-device vfio-pci,host=01:00.0,multifunction=on

rabcor

Banned

Registered: 2013-02-09

Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I think its this typo i overlooked before

--enable-kvm

change to:

-enable-kvm

Right now AMD mobo/cpu users have like 90% chance of getting this to work, even mixing nvidia/amd cards and with broken ivrs tables, remember this is work in progress, so i wouldn't waste money yet only to find it doesn't work, but im prety sure it will in the near future.

still nothing, i figured it seems to think that  "-cpu host" should be a disk image (which it cannot find) and that "-smp" should be a directory, which it also cannot find

if i remove "-cpu host \ -smp ...." it will give me the same error

# qemu-system-x86_64 -enable-kvm -M q35 -m 1024 \ -bios /usr/share/qemu/bios.bin -vga none \ -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \ -device vfio-pci, host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ -device vfio-pci,host=01:00.1,bus=root.1,addr=00.1
qemu-system-x86_64: -m 1024: could not open disk image  -bios: No such file or directory

---

Making Logitech G510 Work on Linux!

alpha_one_x86

Member

Registered: 2013-06-26

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Now I have:
qemu-system-x86_64: -device vfio-pci,host=01:00.0,multifunction=on: vfio_dma_map(0x7fc3e881f800, 0xfffe0000, 0x20000, 0x7fc3e0a00000) = -12 (Cannot allocate memory)
qemu-system-x86_64: vfio_dma_map(0x7fc3e881f800, 0xc8000, 0x18000, 0x7fc3e0808000) = -12 (Cannot allocate memory)
I have checked all into the web (right problem, iommu, ...)
I'm with intel haswell + linux 3.9.
What I should test?

andy123

Member

Registered: 2011-11-04

Posts: 169

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

qemu-system-x86_64: -device vfio-pci,host=01:00.0,multifunction=on: vfio_dma_map(0x7fc3e881f800, 0xfffe0000, 0x20000, 0x7fc3e0a00000) = -12 (Cannot allocate memory)
qemu-system-x86_64: vfio_dma_map(0x7fc3e881f800, 0xc8000, 0x18000, 0x7fc3e0808000) = -12 (Cannot allocate memory)
[…]
What I should test?

Sounds to me like mem-path/hugepages issues. You should probably read some things about hugepages, remove the mem-path option or lower the memory (-m parameter).

---

i'm sorry for my poor english wirting skills…

alpha_one_x86

Member

Registered: 2013-06-26

Posts: 11

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

remove the mem-path option

How? I have not this options.
Already checked for the rest, tested with 512, I can't do less.