You are not logged in.

Pages: 1 … 220 221 222 223 224 … 234

Topic closed

#5526 2015-07-09 15:43:47

Denso
Member
Registered: 2014-08-30
Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi everyone .

Sorry if this is slightly off-topic but is there any real performance or stability gains when using Libvirt over regular command line script to define and launch the VM ?

Offline

#5527 2015-07-09 16:09:53

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@Denso

libvirt is a VM management interface.  It facilitates many things about managing VMs, but it's not some magic wrapper that turbo boosts the VM.  It still emits a QEMU command line in the end.  If you want to keep using vfio-bind and manual scripts, then please also keep all the bugs that come along with them.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5528 2015-07-09 16:34:13

Denso
Member
Registered: 2014-08-30
Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

then please also keep all the bugs that come along with them.

Thank Alex . I saw that libvirt emits a longer QEMU command line too . I think using "options vfio-pci ids=" with a QEMU command script is the best solution for me . (I find it easier to edit a simple shell script that editing a libvirt XML) .

Thank you !

Offline

#5529 2015-07-10 01:03:54

morat
Member
Registered: 2015-07-07
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

After following @aw's guide, libvirt returns:

Error starting domain: Requested operation is not valid: PCI device 0000:01:00.0 is not assignable

Looking through my journal since today's boot, I have a loooot of errors. At the time I try to run the VM, I get messages similar to this for every IOMMU device (where 0000:01:00.1 is my GPU:

libvirtd[5956]: internal error: Failed to find parent device for 0000:01:00.0
libvirtd[5956]: Failed to open config space file '/sys/bus/pci/devices/0000:00:00.0/config': Permission denied

Dmesg here
Something something wit's end?

Offline

#5530 2015-07-10 01:13:43

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

morat wrote:

After following @aw's guide, libvirt returns:

Error starting domain: Requested operation is not valid: PCI device 0000:01:00.0 is not assignable

Looking through my journal since today's boot, I have a loooot of errors. At the time I try to run the VM, I get messages similar to this for every IOMMU device (where 0000:01:00.1 is my GPU:

libvirtd[5956]: internal error: Failed to find parent device for 0000:01:00.0
libvirtd[5956]: Failed to open config space file '/sys/bus/pci/devices/0000:00:00.0/config': Permission denied

Dmesg here
Something something wit's end?

session mode libvirt does not support device assignment, you need to use system mode libvirt (ie. run it as root).  Also, your attempt to load vfio-pci early:

rd.driver.pre=vfio-pci

Isn't working:

[    8.591790] nouveau 0000:01:00.0: enabling device (0004 -> 0007)
[    8.591932] nouveau  [  DEVICE][0000:01:00.0] BOOT0  : 0x124000a1
[    8.591934] nouveau  [  DEVICE][0000:01:00.0] Chipset: GM204 (NV124)
[    8.591934] nouveau  [  DEVICE][0000:01:00.0] Family : NV110
[    8.646855] nouveau  [   VBIOS][0000:01:00.0] using image from PROM
[    8.646990] nouveau  [   VBIOS][0000:01:00.0] BIT signature found
[    8.646992] nouveau  [   VBIOS][0000:01:00.0] version 84.04.31.00.c9
[    8.647160] nouveau  [ DEVINIT][0000:01:00.0] adaptor not initialised
[    8.658974] nouveau  [     PMC][0000:01:00.0] MSI interrupts enabled
[    8.658994] nouveau  [     PFB][0000:01:00.0] RAM type: GDDR5
[    8.658995] nouveau  [     PFB][0000:01:00.0] RAM size: 4096 MiB
[    8.658995] nouveau  [     PFB][0000:01:00.0]    ZCOMP: 0 tags

You'll need to figure out the equivalent trick for Arch, rd.driver.pre works with dracut-based initramfs images, like Fedora.  You may want to start with a simple pci-stub.ids option and move on to something more complicated later.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5531 2015-07-10 02:16:28

morat
Member
Registered: 2015-07-07
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Ok, some progress has been made...
When running the VM, libvirtd causes a kernel panic. After reading it, it seems like it might be caused by a bug in libvirt, but I'm no expert, so I'll defer to you. Dmesg here

Offline

#5532 2015-07-10 02:24:40

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You fat fingered your pci-stub.ids entry and didn't verify that it worked correctly.  You have:

pci-stub.ids=10de:13x0,10de:0fbb#i915.enable_hd_vgaarb=1

You want:

pci-stub.ids=10de:13c0,10de:0fbb i915.enable_hd_vgaarb=1

And then double check it worked with:

lspci -nnks 1:

Which should show the in-use driver is pci-stub.  Not sure why your libvirt install is so unhappy.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5533 2015-07-10 02:29:57

morat
Member
Registered: 2015-07-07
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

You fat fingered your pci-stub.ids entry and didn't verify that it worked correctly.  You have:

pci-stub.ids=10de:13x0,10de:0fbb#i915.enable_hd_vgaarb=1

You want:

pci-stub.ids=10de:13c0,10de:0fbb i915.enable_hd_vgaarb=1

And then double check it worked with:

lspci -nnks 1:

Which should show the in-use driver is pci-stub.  Not sure why your libvirt install is so unhappy.

Commenting out the vga fix was intentional, I have yet to patch the kernel, but that x.... Time to move on from the 2008 apple-supplied rubber dome keyboard, I think, causes more than a few mistakes.

Update:
I dont get a kernel panic any more, which is nice; but now libvirt spits this out:

Error starting domain: internal error: early end of file from monitor: possible problem:
2015-07-10T12:55:29.897699Z qemu-system-x86_64: unable to map backing store for hugepages: Cannot allocate memory


Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 89, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 125, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 83, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1433, in startup
    self._backend.create()
  File "/usr/lib/python2.7/site-packages/libvirt.py", line 1029, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error: early end of file from monitor: possible problem:
2015-07-10T12:55:29.897699Z qemu-system-x86_64: unable to map backing store for hugepages: Cannot allocate memory

Hugepages have been set up as per your guide, @aw. Why break the trend?
It's so close, I can almost taste it...

Last edited by morat (2015-07-10 03:01:22)

Offline

#5534 2015-07-10 03:04:56

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

dmesg says you have 2048 2M pages, for 4G.  What's the memory size of the guest?  You could also start simple and add hugepages once you get things working...

EDIT:  Also, you have a GTX980, which I'd fully expect to support UEFI.  Why are you considering the i915/VGA route?

Last edited by aw (2015-07-10 03:06:48)

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5535 2015-07-10 03:35:16

morat
Member
Registered: 2015-07-07
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

dmesg says you have 2048 2M pages, for 4G.  What's the memory size of the guest?  You could also start simple and add hugepages once you get things working...

EDIT:  Also, you have a GTX980, which I'd fully expect to support UEFI.  Why are you considering the i915/VGA route?

I need to use the IGP for Arch, as I don't have another discrete card. EDIT: I just realized how inane that is, I'm using the IGP right now without hassle; looks like I don't need i915. That being said, my onboard DVI is AWOL, and I have no idea why, in Arch, BIOS/UEFI, and the physical Windows.

It all seems to work now, or at least... start. I can't get networking going, libvirt lists my two physical interfaces, and an empty bridge. None of the three give the guest connectivity. I also can't mount a driver cd to the guest, virsh says hotplug isn't supported, and virt-manager claims it works, but the guest doesn't see the drive.

Last edited by morat (2015-07-10 03:37:07)

Offline

#5536 2015-07-10 03:36:54

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

morat wrote:
aw wrote:

dmesg says you have 2048 2M pages, for 4G.  What's the memory size of the guest?  You could also start simple and add hugepages once you get things working...

EDIT:  Also, you have a GTX980, which I'd fully expect to support UEFI.  Why are you considering the i915/VGA route?

I need to use the IGP for Arch, as I don't have another discrete card.

Umm... you can still use IGD with the UEFI solution

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5537 2015-07-10 03:40:24

Zeorymer
Member
Registered: 2015-07-03
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:
Zeorymer wrote:
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:02 JK-Desktop kernel: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=07:00.0 address=0x00000002451e49e0]
Jul 06 18:30:02 JK-Desktop kernel: BUG: unable to handle kernel paging request at ffffffffffffc039
Jul 06 18:30:02 JK-Desktop kernel: IP: [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: PGD 180e067 PUD 1810067 PMD 0 
Jul 06 18:30:02 JK-Desktop kernel: Oops: 0000 [#1] PREEMPT SMP 
Jul 06 18:30:02 JK-Desktop kernel: Modules linked in: nls_utf8 isofs vfio_pci vfio_iommu_type1 vfio it87 hwmon_vid eeepc_wmi asus_wmi sparse_keymap led_class rfkill video kvm_amd kvm fuse crct10dif_pclmu
Jul 06 18:30:02 JK-Desktop kernel:  hid_logitech ff_memless hid_generic hid_microsoft usbhid hid uas usb_storage atkbd libps2 ahci libahci libata ohci_pci xhci_pci ohci_hcd ehci_pci xhci_hcd ehci_hcd scs
Jul 06 18:30:02 JK-Desktop kernel: CPU: 4 PID: 720 Comm: cinnamon Not tainted 4.0.7-2-vfio #1
Jul 06 18:30:02 JK-Desktop kernel: Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO, BIOS 1604 10/16/2012
Jul 06 18:30:02 JK-Desktop kernel: task: ffff88023db2b2a0 ti: ffff8800a4144000 task.ti: ffff8800a4144000
Jul 06 18:30:02 JK-Desktop kernel: RIP: 0010:[<ffffffff81576b77>]  [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: RSP: 0018:ffff8800a4147a08  EFLAGS: 00010292
Jul 06 18:30:02 JK-Desktop kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00000000c0000100
Jul 06 18:30:02 JK-Desktop kernel: RDX: ffff8800a4147fd8 RSI: ffff88023db2b2a0 RDI: ffff88024ed13e40
Jul 06 18:30:02 JK-Desktop kernel: RBP: ffff8800a4147a18 R08: ffff8800a4144000 R09: ffff88024ed13e70
Jul 06 18:30:02 JK-Desktop kernel: R10: 0000000000000020 R11: 0000000000000004 R12: 000000000006224f
Jul 06 18:30:02 JK-Desktop kernel: R13: ffff8800a4147b80 R14: ffff8800a4147bec R15: 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: FS:  00007f532c8c39c0(0000) GS:ffff88024ed00000(0000) knlGS:0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 06 18:30:02 JK-Desktop kernel: CR2: ffffffffffffc039 CR3: 00000000a4ca2000 CR4: 00000000000407e0
Jul 06 18:30:02 JK-Desktop kernel: Stack:
Jul 06 18:30:02 JK-Desktop kernel:  ffff8800a4147bec 0000000000000000 ffff8800a4147ab8 ffffffff815798fc
Jul 06 18:30:02 JK-Desktop kernel:  ffff8800a4147a28 0000000000000000 0000000000000000 000000a9a6b041b6
Jul 06 18:30:02 JK-Desktop kernel:  000000a9a6aa1f67 ffffffff810e0bd0 ffff88024ed0e408 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: Call Trace:
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff815798fc>] schedule_hrtimeout_range_clock.part.6+0xfc/0x120
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff810e0bd0>] ? __run_hrtimer+0x250/0x250
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579891>] ? schedule_hrtimeout_range_clock.part.6+0x91/0x120
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579939>] schedule_hrtimeout_range_clock+0x19/0x50
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579983>] schedule_hrtimeout_range+0x13/0x20
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ec8f4>] poll_schedule_timeout+0x44/0x70
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ee111>] do_sys_poll+0x531/0x600
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ee2e1>] SyS_poll+0x71/0x130
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff8157a889>] system_call_fastpath+0x12/0x17
Jul 06 18:30:02 JK-Desktop kernel: Code: b8 00 00 48 89 e5 53 48 83 ec 08 48 8b 10 48 85 d2 74 0a 48 83 b8 a8 07 00 00 00 74 27 65 48 8b 1c 25 60 b8 00 00 e8 99 f5 ff ff <48> 8b 83 38 c0 ff ff a8 08 75 f
Jul 06 18:30:02 JK-Desktop kernel: RIP  [<ffffffff81576b77>] schedule+0x37/0x90

Actually, this looks very bad. I think only aw or someone his level can give you a proper advice. Strange thing those ioapic remapping commands didn't work.

Since aw is in the thread, may I have your opinion on my troubles?  That is, if you have any idea.

Offline

#5538 2015-07-10 04:31:24

morat
Member
Registered: 2015-07-07
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Great, everything works; even the PCI wifi card. Oh, except the Code 43.

I'm using a GTX 980, is it too new for the workarounds? Was all this for nothing?

Offline

#5539 2015-07-10 12:53:37

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Zeorymer wrote:
Duelist wrote:
Zeorymer wrote:
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:02 JK-Desktop kernel: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=07:00.0 address=0x00000002451e49e0]
Jul 06 18:30:02 JK-Desktop kernel: BUG: unable to handle kernel paging request at ffffffffffffc039
Jul 06 18:30:02 JK-Desktop kernel: IP: [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: PGD 180e067 PUD 1810067 PMD 0 
Jul 06 18:30:02 JK-Desktop kernel: Oops: 0000 [#1] PREEMPT SMP 
Jul 06 18:30:02 JK-Desktop kernel: Modules linked in: nls_utf8 isofs vfio_pci vfio_iommu_type1 vfio it87 hwmon_vid eeepc_wmi asus_wmi sparse_keymap led_class rfkill video kvm_amd kvm fuse crct10dif_pclmu
Jul 06 18:30:02 JK-Desktop kernel:  hid_logitech ff_memless hid_generic hid_microsoft usbhid hid uas usb_storage atkbd libps2 ahci libahci libata ohci_pci xhci_pci ohci_hcd ehci_pci xhci_hcd ehci_hcd scs
Jul 06 18:30:02 JK-Desktop kernel: CPU: 4 PID: 720 Comm: cinnamon Not tainted 4.0.7-2-vfio #1
Jul 06 18:30:02 JK-Desktop kernel: Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO, BIOS 1604 10/16/2012
Jul 06 18:30:02 JK-Desktop kernel: task: ffff88023db2b2a0 ti: ffff8800a4144000 task.ti: ffff8800a4144000
Jul 06 18:30:02 JK-Desktop kernel: RIP: 0010:[<ffffffff81576b77>]  [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: RSP: 0018:ffff8800a4147a08  EFLAGS: 00010292
Jul 06 18:30:02 JK-Desktop kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00000000c0000100
Jul 06 18:30:02 JK-Desktop kernel: RDX: ffff8800a4147fd8 RSI: ffff88023db2b2a0 RDI: ffff88024ed13e40
Jul 06 18:30:02 JK-Desktop kernel: RBP: ffff8800a4147a18 R08: ffff8800a4144000 R09: ffff88024ed13e70
Jul 06 18:30:02 JK-Desktop kernel: R10: 0000000000000020 R11: 0000000000000004 R12: 000000000006224f
Jul 06 18:30:02 JK-Desktop kernel: R13: ffff8800a4147b80 R14: ffff8800a4147bec R15: 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: FS:  00007f532c8c39c0(0000) GS:ffff88024ed00000(0000) knlGS:0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 06 18:30:02 JK-Desktop kernel: CR2: ffffffffffffc039 CR3: 00000000a4ca2000 CR4: 00000000000407e0
Jul 06 18:30:02 JK-Desktop kernel: Stack:
Jul 06 18:30:02 JK-Desktop kernel:  ffff8800a4147bec 0000000000000000 ffff8800a4147ab8 ffffffff815798fc
Jul 06 18:30:02 JK-Desktop kernel:  ffff8800a4147a28 0000000000000000 0000000000000000 000000a9a6b041b6
Jul 06 18:30:02 JK-Desktop kernel:  000000a9a6aa1f67 ffffffff810e0bd0 ffff88024ed0e408 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: Call Trace:
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff815798fc>] schedule_hrtimeout_range_clock.part.6+0xfc/0x120
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff810e0bd0>] ? __run_hrtimer+0x250/0x250
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579891>] ? schedule_hrtimeout_range_clock.part.6+0x91/0x120
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579939>] schedule_hrtimeout_range_clock+0x19/0x50
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff81579983>] schedule_hrtimeout_range+0x13/0x20
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ec8f4>] poll_schedule_timeout+0x44/0x70
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ee111>] do_sys_poll+0x531/0x600
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff811ee2e1>] SyS_poll+0x71/0x130
Jul 06 18:30:02 JK-Desktop kernel:  [<ffffffff8157a889>] system_call_fastpath+0x12/0x17
Jul 06 18:30:02 JK-Desktop kernel: Code: b8 00 00 48 89 e5 53 48 83 ec 08 48 8b 10 48 85 d2 74 0a 48 83 b8 a8 07 00 00 00 74 27 65 48 8b 1c 25 60 b8 00 00 e8 99 f5 ff ff <48> 8b 83 38 c0 ff ff a8 08 75 f
Jul 06 18:30:02 JK-Desktop kernel: RIP  [<ffffffff81576b77>] schedule+0x37/0x90

Actually, this looks very bad. I think only aw or someone his level can give you a proper advice. Strange thing those ioapic remapping commands didn't work.

Since aw is in the thread, may I have your opinion on my troubles?  That is, if you have any idea.

Have you tried disabling hugepages for the IOMMU yet? (modprobe.d: options vfio_iommu_type1 disable_hugepages=1)  What's the processor?  This seems like a common problem on APUs but seems to work fine on non-APU processors.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5540 2015-07-10 12:55:20

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

morat wrote:

Great, everything works; even the PCI wifi card. Oh, except the Code 43.

I'm using a GTX 980, is it too new for the workarounds? Was all this for nothing?

Sorry, no data, no help.  I'm not going to guess whether you followed the instructions to edit the domain xml to remove the hyper-v extensions.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5541 2015-07-10 16:12:25

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

Have you tried disabling hugepages for the IOMMU yet? (modprobe.d: options vfio_iommu_type1 disable_hugepages=1)  What's the processor?  This seems like a common problem on APUs but seems to work fine on non-APU processors.

He has a family 15h CPU with

00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]

alike device, so i think it's an FX platform(not APU, APUs don't work without their specific southbridges) and the initial problem was 

[Firmware Bug]: AMD-Vi: IOAPIC[9] not in IVRS table
[Firmware Bug]: AMD-Vi: IOAPIC[10] not in IVRS table
[Firmware Bug]: AMD-Vi: No southbridge IOAPIC found
AMD-Vi: Disabling interrupt remapping

I've suggested mapping ioapic directly via commandline, like

ivrs_ioapic[7]=00:14.0 ivrs_ioapic[8]=00:00.1

but that(after address adjustments) moved him to IOTLB_INV_TIMEOUT error.

The iommu dump he posted is:

AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
AMD-Vi:        mmio-addr: 00000000feb20000
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:00.2
AMD-Vi:   DEV_SELECT                         devid: 00:02.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 01:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 01:00.1
AMD-Vi:   DEV_SELECT                         devid: 00:04.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 02:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:05.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 03:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:06.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 04:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:07.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 05:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:11.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:12.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:12.2
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:13.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:13.2
AMD-Vi:   DEV_SELECT                         devid: 00:14.0 flags: d7
AMD-Vi:   DEV_SELECT                         devid: 00:14.3 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:14.4 flags: 00
AMD-Vi:   DEV_ALIAS_RANGE                 devid: 06:00.0 flags: 00 devid_to: 00:14.4
AMD-Vi:   DEV_RANGE_END                 devid: 06:1f.7
AMD-Vi:   DEV_SELECT                         devid: 00:14.5 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:15.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 07:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 07:00.1
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:16.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:16.2
AMD-Vi:   DEV_SPECIAL(IOAPIC[0])                devid: 00:14.0
AMD-Vi:   DEV_SPECIAL(HPET[0])                devid: 00:14.0
AMD-Vi:   DEV_SPECIAL(IOAPIC[255])                devid: 00:00.1

Maybe you can suggest something else?

Last edited by Duelist (2015-07-10 16:13:39)

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5542 2015-07-10 16:14:06

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@Duelist

Suggestion: Working motherboard

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5543 2015-07-10 16:23:36

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

@Duelist

Suggestion: Working motherboard

Crap. Is his IVRS table messed up so bad?

@Zeorymer:
Can you tell us an exact model of your motherboard? I know that it's ASUS, but which one?

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5544 2015-07-10 16:24:41

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:
aw wrote:

@Duelist

Suggestion: Working motherboard

Crap. Is his IVRS table messed up so bad?

Dunno, but it's beyond the scope of this thread.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5545 2015-07-10 16:35:24

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
Duelist wrote:
aw wrote:

@Duelist

Suggestion: Working motherboard

Crap. Is his IVRS table messed up so bad?

Dunno, but it's beyond the scope of this thread.

I understand your tiredness of supporting countless people, but if i'll come into #kvm and ask everyone, including you, about that..:D


http://www.gossamer-threads.com/lists/xen/devel/274857
https://bbs.archlinux.org/viewtopic.php?id=163102

Suddenly, i've cleared my vision and other google results started to seem fitting.

Last edited by Duelist (2015-07-10 16:40:47)

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5546 2015-07-10 16:35:45

The_Moves
Member
Registered: 2015-01-06
Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@aw, What kernel are you currently running? I just updated to FC22 and stock 4.0.6-300 kernel without issues so far. Do you stay with Rawhide or within the FC releases for kernel packages? Or just compile your own :-)

Offline

#5547 2015-07-10 16:42:42

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:
aw wrote:

Have you tried disabling hugepages for the IOMMU yet? (modprobe.d: options vfio_iommu_type1 disable_hugepages=1)  What's the processor?  This seems like a common problem on APUs but seems to work fine on non-APU processors.

He has a family 15h CPU with

00:13.2 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]

alike device, so i think it's an FX platform(not APU, APUs don't work without their specific southbridges) and the initial problem was 

[Firmware Bug]: AMD-Vi: IOAPIC[9] not in IVRS table
[Firmware Bug]: AMD-Vi: IOAPIC[10] not in IVRS table
[Firmware Bug]: AMD-Vi: No southbridge IOAPIC found
AMD-Vi: Disabling interrupt remapping

I've suggested mapping ioapic directly via commandline, like

ivrs_ioapic[7]=00:14.0 ivrs_ioapic[8]=00:00.1

but that(after address adjustments) moved him to IOTLB_INV_TIMEOUT error.

The iommu dump he posted is:

AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
AMD-Vi:        mmio-addr: 00000000feb20000
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:00.2
AMD-Vi:   DEV_SELECT                         devid: 00:02.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 01:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 01:00.1
AMD-Vi:   DEV_SELECT                         devid: 00:04.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 02:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:05.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 03:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:06.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 04:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:07.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 05:00.0 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:11.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:12.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:12.2
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:13.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:13.2
AMD-Vi:   DEV_SELECT                         devid: 00:14.0 flags: d7
AMD-Vi:   DEV_SELECT                         devid: 00:14.3 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:14.4 flags: 00
AMD-Vi:   DEV_ALIAS_RANGE                 devid: 06:00.0 flags: 00 devid_to: 00:14.4
AMD-Vi:   DEV_RANGE_END                 devid: 06:1f.7
AMD-Vi:   DEV_SELECT                         devid: 00:14.5 flags: 00
AMD-Vi:   DEV_SELECT                         devid: 00:15.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 07:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 07:00.1
AMD-Vi:   DEV_SELECT_RANGE_START         devid: 00:16.0 flags: 00
AMD-Vi:   DEV_RANGE_END                 devid: 00:16.2
AMD-Vi:   DEV_SPECIAL(IOAPIC[0])                devid: 00:14.0
AMD-Vi:   DEV_SPECIAL(HPET[0])                devid: 00:14.0
AMD-Vi:   DEV_SPECIAL(IOAPIC[255])                devid: 00:00.1

Maybe you can suggest something else?

ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1

Last edited by nbhs (2015-07-10 16:43:04)

Offline

#5548 2015-07-10 16:47:33

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:
ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1
Zeorymer wrote:
/init/ line 1 = ivrs_ioapic[9]=00:14.0  : Command not found
/init/ line 1 = ivrs_ioapic[10]=00:00.1   : Command not found

or something like that.

Wait, how could i miss that part of his message?>_<
Sorry for bothering guys.

@Zeorymer:
Are you sure you've put that command line switches into the right... place?
Cause if you're running kernel newer than 3.19(where there's a patch for that) - that line is your only solution apart from aw's ultimate one.

Last edited by Duelist (2015-07-10 16:48:37)

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5549 2015-07-10 17:55:46

Zeorymer
Member
Registered: 2015-07-03
Posts: 7

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:
nbhs wrote:
ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1
Zeorymer wrote:
/init/ line 1 = ivrs_ioapic[9]=00:14.0  : Command not found
/init/ line 1 = ivrs_ioapic[10]=00:00.1   : Command not found

or something like that.

Wait, how could i miss that part of his message?>_<
Sorry for bothering guys.

@Zeorymer:
Are you sure you've put that command line switches into the right... place?
Cause if you're running kernel newer than 3.19(where there's a patch for that) - that line is your only solution apart from aw's ultimate one.

Kernel was posted several posts ago: 4.0.7
Processor is an 8350.

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="iommu=debug,verbose amd_iommu_dump=1 ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1"
GRUB_CMDLINE_LINUX=""
#

# Preload both GPT and MBR modules so that they are not missed
GRUB_PRELOAD_MODULES="part_gpt part_msdos"

# Uncomment to enable Hidden Menu, and optionally hide the timeout count
#GRUB_HIDDEN_TIMEOUT=5
#GRUB_HIDDEN_TIMEOUT_QUIET=true

# Uncomment to use basic console
GRUB_TERMINAL_INPUT=console

# Uncomment to disable graphical terminal
#GRUB_TERMINAL_OUTPUT=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
GRUB_GFXMODE=auto

# Uncomment to allow the kernel use the same resolution used by grub
GRUB_GFXPAYLOAD_LINUX=keep

# Uncomment if you want GRUB to pass to the Linux kernel the old parameter 
# format "root=/dev/xxx" instead of "root=/dev/disk/by-uuid/xxx" 
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY=true

# Uncomment and set to the desired menu colors.  Used by normal and wallpaper 
# modes only.  Entries specified as foreground/background.
GRUB_COLOR_NORMAL="light-blue/black"
GRUB_COLOR_HIGHLIGHT="light-cyan/blue"

# Uncomment one of them for the gfx desired, a image background or a gfxtheme
#GRUB_BACKGROUND="/path/to/wallpaper"
#GRUB_THEME="/path/to/gfxtheme"

# Uncomment to get a beep at GRUB start
#GRUB_INIT_TUNE="480 440 1"

#GRUB_SAVEDEFAULT="true"

This is where I put it, is this wrong?  I know there are other things in there that I recently added to see if I can get other information (first one doesn't seem to do anything new).

What I find odd id that I get the command not found messages but I don't get the IVRS messages being mapped wrong...which leads me to believe that the kernel THINKS the command is being passed(?)

So...a new board is the only solution?  Damn you Asus.

Last edited by Zeorymer (2015-07-10 17:59:26)

Offline

#5550 2015-07-10 18:17:38

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@Zeorymer

You can file a bug or post to the iommu mailing list

@The_Moves

Whatever I need for the situation.  Often rc kernels for development and testing, latest Fedora kernel otherwise.

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

Pages: 1 … 220 221 222 223 224 … 234

Topic closed

Board footer

Powered by FluxBB