You are not logged in.
Even though it complains about ioapic[9] and ioapic[10]? Then again, iommu is not my expertise.
Kernel is -ck with required options enabled for passthrough.
Everything else was in Arch repos, though I haven't really gotten that far thanks to Asus not knowing how to make a bios correctly.
I said adjust accordingly.. I'd try mapping ioapic[10] to 00:14.0 and ioapic[9] to 00:01.0. Maybe vice-versa... I can't tell for sure.
Yeah, 00:14.0 to ioapic[9], 00:01.0 to ioapic[10]. That should do the thing.
Tell the real versions.
Last edited by Duelist (2015-07-03 17:48:18)
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
Zeorymer wrote:Even though it complains about ioapic[9] and ioapic[10]? Then again, iommu is not my expertise.
Kernel is -ck with required options enabled for passthrough.
Everything else was in Arch repos, though I haven't really gotten that far thanks to Asus not knowing how to make a bios correctly.
I said adjust accordingly.. I'd try mapping ioapic[10] to 00:14.0 and ioapic[9] to 00:01.0. Maybe vice-versa... I can't tell for sure.
Yeah, 00:14.0 to ioapic[9], 00:01.0 to ioapic[10]. That should do the thing.Tell the real versions.
Whell booting with that caused a panic...I'm going to assume first that I messed something up.
pastebin because of the size of the log.
It seems to be related to the 7950 I'm trying to pass. I doubt the card itself, it was working yesterday in Windows (before I nuked it)
Kernel version is at the top of the log.
extra/qemu 2.3.0-3
community/libvirt 1.2.17-1
ovmf-fishman-git <-SVN version was marked out of date, if this version will be a problem, I can change it.
Anything else?
Offline
So I've had Windows 8.1 running for quite some time now in a VM and almost everything is working fine. However, I still feel like I'm missing something as far a CPU tweaking goes.
I have a Xeon with 8 threads (4 cores + HT). I launch with the qemu arg "cpu -host" and use libvirt to set the topology to 3 cores 6 thread. Then I use cpuset to create a cpuset and move all tasks (and kernel threads) to cpu 0-1. Then I have my vcpus pinned to CPUs 2-7. I've also enabled the Hyper-V enlightenments. However, I still feel like I'm getting questionable CPU performance. The most notable example is Borderlands: PS, which has really poor performance in the VM. I am passing through a 270X and play a lot of GPU intesive games just fine, but BL gives me issues like no other game.
Do anyone have any tips on how I could further optimize my CPU performance?
Offline
@MonopolyMan
What host kernel? A while back we added lazy debug register handling, primarily due to BL2 use of debug registers.
EDIT: kernel 3.15 includes lazy debug register handling
Last edited by aw (2015-07-05 13:25:07)
http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses? Try https://www.redhat.com/mailman/listinfo/vfio-users
Offline
So, my VDIs are working almost fine, they are stable and the performance is good. There are two small problems though
keyboard is not available during initial phase of Windows startup (selections in text mode, before Windows logo)
quite often during startup (animated Windows logo) the virtual machine will enter "paused" state and there is nothing to indicate why. The only way to recover is "virsh destroy", rollback its system volume (* because of above) and then start again
Here is current virsh definition of one of the affected VDIs.
<domain type='kvm' id='3'>
<name>gdynia</name>
<uuid>7a76b37e-3604-4ab1-84ef-69de09123ea3</uuid>
<memory unit='KiB'>16777216</memory>
<currentMemory unit='KiB'>16777216</currentMemory>
<memoryBacking>
<hugepages>
<page size='2048' unit='KiB'/>
</hugepages>
</memoryBacking>
<vcpu placement='static'>8</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='2'/>
<vcpupin vcpu='1' cpuset='3'/>
<vcpupin vcpu='2' cpuset='4'/>
<vcpupin vcpu='3' cpuset='5'/>
<vcpupin vcpu='4' cpuset='14'/>
<vcpupin vcpu='5' cpuset='15'/>
<vcpupin vcpu='6' cpuset='16'/>
<vcpupin vcpu='7' cpuset='17'/>
</cputune>
<resource>
<partition>/machine</partition>
</resource>
<os>
<type arch='x86_64' machine='pc-i440fx-2.2'>hvm</type>
<loader type='rom'>/usr/share/qemu/bios.bin</loader>
</os>
<features>
<acpi/>
<apic/>
<pae/>
<hyperv>
<relaxed state='on'/>
<vapic state='on'/>
<spinlocks state='on' retries='4096'/>
</hyperv>
</features>
<cpu mode='host-passthrough'>
<topology sockets='1' cores='4' threads='2'/>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
<timer name='hypervclock' present='yes'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64.xvga03.sh</emulator>
<disk type='block' device='disk'>
<driver name='qemu' type='raw'/>
<source dev='/dev/zvol/zdata/vdis/gdynia'/>
<backingStore/>
<target dev='vda' bus='virtio'/>
<boot order='1'/>
<alias name='virtio-disk0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw'/>
<source dev='/dev/zvol/zdata/protected'/>
<backingStore/>
<target dev='vdc' bus='virtio'/>
<alias name='virtio-disk2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</disk>
<disk type='block' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source dev='/dev/disk/by-partuuid/c5a05f68-9f2c-4ad6-96c5-a5fc85d73d7c'/>
<backingStore/>
<target dev='vdd' bus='virtio'/>
<alias name='virtio-disk3'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source file='/root/virtio-win.iso'/>
<backingStore/>
<target dev='hdd' bus='ide'/>
<readonly/>
<alias name='ide0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<alias name='usb'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<alias name='usb'/>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/>
</controller>
<controller type='ide' index='0'>
<alias name='ide'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='pci' index='0' model='pci-root'>
<alias name='pci.0'/>
</controller>
<controller type='virtio-serial' index='0'>
<alias name='virtio-serial0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</controller>
<interface type='bridge'>
<mac address='52:54:01:34:56:e7'/>
<source bridge='br0'/>
<target dev='vnet1'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<channel type='unix'>
<source mode='bind' path='/var/lib/libvirt/qemu/gdynia.agent'/>
<target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
<alias name='channel0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x0853'/>
<product id='0x0124'/>
<address bus='2' device='3'/>
</source>
<alias name='hostdev0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
<address domain='0x0000' bus='0x84' slot='0x00' function='0x0'/>
</source>
<alias name='hostdev1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x0f' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<driver name='vfio'/>
<source>
<address domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
</source>
<alias name='hostdev2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</hostdev>
<memballoon model='virtio'>
<alias name='balloon0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</memballoon>
</devices>
</domain>and some information about the host
root@gdansk ~ # uname -a
Linux gdansk 4.0.7-1-ARCH #1 SMP PREEMPT Sat Jul 4 15:22:43 BST 2015 x86_64 GNU/Linux
root@gdansk ~ # qemu-system-x86_64 --version
QEMU emulator version 2.2.1, Copyright (c) 2003-2008 Fabrice Bellard
root@gdansk ~ # virsh --version
1.2.16
root@gdansk ~ # cat /proc/cmdline
BOOT_IMAGE=../vmlinuz-linux zfs=zroot rw spl.spl_hostid=0xa8c03302 console=ttyS0,115200N8R nomodeset udev.children-max=32 edac_core.edac_mc_panic_on_ue=1 nohz=off intel_iommu=on iommu=pt pci-stub.ids=8086:1d20,1002:6707,1002:aa80,1002:67b0,1002:aac8,104c:8241,1033:0194 isolcpus=2,3,4,5,6,7,8,9,10,11,14,15,16,17,18,19,20,21,22,23 initrd=../initramfs-linux.img
root@gdansk ~ # zgrep VFIO /proc/config.gz
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO=m
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_KVM_VFIO=y
root@gdansk ~ # free -gwt :(
total used free shared buffers cache available
Mem: 125 51 58 0 6 9 65
Swap: 159 0 159
Total: 285 51 218
root@gdansk /etc/sysctl.d # cat 80-hugepages.conf
# Reserve this many 2MB pages for virtual machine
vm.nr_hugepages = 25000
root@gdansk ~ # grep "processor" /proc/cpuinfo | wc -l
24
root@gdansk ~ # cat /proc/cpuinfo | tail -28
processor : 23
vendor_id : GenuineIntel
cpu family : 6
model : 62
model name : Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz
stepping : 4
microcode : 0x427
cpu MHz : 1268.617
cache size : 15360 KB
physical id : 1
siblings : 12
core id : 5
cpu cores : 6
apicid : 43
initial apicid : 43
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm ida arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt
bugs :
bogomips : 5204.72
clflush size : 64
cache_alignment : 64
address sizes : 46 bits physical, 48 bits virtual
power management:
root@gdansk ~ # cat /usr/bin/qemu-system-x86_64.xvga03.sh
#!/bin/sh
exec nice --adjustment=-5 /usr/bin/qemu-system-x86_64 `echo "$@" | sed 's/vfio-pci,host=03:00.0/vfio-pci,host=03:00.0,x-vga=on/g'`
root@gdansk ~ # lspci -vnn -s 0000:03:00
03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cayman LE GL [FirePro V5900] [1002:6707] (prog-if 00 [VGA controller])
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:0b06]
Flags: bus master, fast devsel, latency 0, IRQ 128
Memory at c0000000 (64-bit, prefetchable) [size=256M]
Memory at dfe20000 (64-bit, non-prefetchable) [size=128K]
I/O ports at 7000 [size=256]
Expansion ROM at dfe00000 [disabled] [size=128K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Kernel driver in use: vfio-pci
Kernel modules: radeon
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] [1002:aa80]
Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series] [1002:aa80]
Flags: bus master, fast devsel, latency 0, IRQ 10
Memory at dfe40000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [50] Power Management version 3
Capabilities: [58] Express Legacy Endpoint, MSI 00
Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
Capabilities: [150] Advanced Error Reporting
Kernel driver in use: pci-stub
Kernel modules: snd_hda_intelLast edited by Bronek (2015-07-05 08:22:29)
Offline
@MonopolyMan
What host kernel? A while back we added lazy debug register handling, primarily due to BL2 use of debug registers.
EDIT: kernel 3.15 includes lazy debug register handling
I just installed 3.16 and that seemed to fix it! It made both Dirty Bomb and Borderlands go from completely unplayable to 60+ FPS.
Thank you very much!
Offline
Anything else?
Well... try swapping BDFs in the commandline, making ioapic[9] 00:00.1 and vice versa.
Also you can try swapping the host and the guest cards, placing your HD7950 into 01:00.0(first pci-e slot).
BTW, the card itself should be fine, because, afair, 270x is rebadged HD7950, and as you can see there and along the thread, people have no problems using it.
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
One other thing. For sound, I am using -soundhw hda. This works really well for output, but I have an odd issue with input. For some reason, my microphone is delayed by about 20-25 second on the guest. However, there is no issues on the host. I can't seem to find any reason why this might be happening. Does anyone have any susgestions?
Note: I'm using:
<qemu:env name='QEMU_PA_SAMPLES' value='4096'/>
<qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
Offline
Zeorymer wrote:Anything else?
Well... try swapping BDFs in the commandline, making ioapic[9] 00:00.1 and vice versa.
Tried that, froze on "booting kernel"
I noticed a line appear while booting that doesn't seem to get logged:
/init/ line 1 = ivrs_ioapic[9]=00:14.0 : Command not found
/init/ line 1 = ivrs_ioapic[10]=00:00.1 : Command not foundor something like that.
It seems to work anyway as the table error doesn't show
Also you can try swapping the host and the guest cards, placing your HD7950 into 01:00.0(first pci-e slot).
I attempted to do this as i was setting up the system, but it wants to diosplay everything out the first card in the system, even if the monitors are in the 2600XT in the lower slots. Any way I can force it to?
BTW, the card itself should be fine, because, afair, 270x is rebadged HD7950, and as you can see there and along the thread, people have no problems using it.
The R9 270X = 7870
The R9 280 = 7950
However there are quite a few 7950s on the list being passed sucessfully.
Additionally, I built the linux-vfio package to see if it made any difference, system booted with the same timeout messages and locked up when I tried the vfio-bind command.
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out
Jul 06 18:30:02 JK-Desktop kernel: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=07:00.0 address=0x00000002451e49e0]
Jul 06 18:30:02 JK-Desktop kernel: BUG: unable to handle kernel paging request at ffffffffffffc039
Jul 06 18:30:02 JK-Desktop kernel: IP: [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: PGD 180e067 PUD 1810067 PMD 0
Jul 06 18:30:02 JK-Desktop kernel: Oops: 0000 [#1] PREEMPT SMP
Jul 06 18:30:02 JK-Desktop kernel: Modules linked in: nls_utf8 isofs vfio_pci vfio_iommu_type1 vfio it87 hwmon_vid eeepc_wmi asus_wmi sparse_keymap led_class rfkill video kvm_amd kvm fuse crct10dif_pclmu
Jul 06 18:30:02 JK-Desktop kernel: hid_logitech ff_memless hid_generic hid_microsoft usbhid hid uas usb_storage atkbd libps2 ahci libahci libata ohci_pci xhci_pci ohci_hcd ehci_pci xhci_hcd ehci_hcd scs
Jul 06 18:30:02 JK-Desktop kernel: CPU: 4 PID: 720 Comm: cinnamon Not tainted 4.0.7-2-vfio #1
Jul 06 18:30:02 JK-Desktop kernel: Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO, BIOS 1604 10/16/2012
Jul 06 18:30:02 JK-Desktop kernel: task: ffff88023db2b2a0 ti: ffff8800a4144000 task.ti: ffff8800a4144000
Jul 06 18:30:02 JK-Desktop kernel: RIP: 0010:[<ffffffff81576b77>] [<ffffffff81576b77>] schedule+0x37/0x90
Jul 06 18:30:02 JK-Desktop kernel: RSP: 0018:ffff8800a4147a08 EFLAGS: 00010292
Jul 06 18:30:02 JK-Desktop kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00000000c0000100
Jul 06 18:30:02 JK-Desktop kernel: RDX: ffff8800a4147fd8 RSI: ffff88023db2b2a0 RDI: ffff88024ed13e40
Jul 06 18:30:02 JK-Desktop kernel: RBP: ffff8800a4147a18 R08: ffff8800a4144000 R09: ffff88024ed13e70
Jul 06 18:30:02 JK-Desktop kernel: R10: 0000000000000020 R11: 0000000000000004 R12: 000000000006224f
Jul 06 18:30:02 JK-Desktop kernel: R13: ffff8800a4147b80 R14: ffff8800a4147bec R15: 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: FS: 00007f532c8c39c0(0000) GS:ffff88024ed00000(0000) knlGS:0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 06 18:30:02 JK-Desktop kernel: CR2: ffffffffffffc039 CR3: 00000000a4ca2000 CR4: 00000000000407e0
Jul 06 18:30:02 JK-Desktop kernel: Stack:
Jul 06 18:30:02 JK-Desktop kernel: ffff8800a4147bec 0000000000000000 ffff8800a4147ab8 ffffffff815798fc
Jul 06 18:30:02 JK-Desktop kernel: ffff8800a4147a28 0000000000000000 0000000000000000 000000a9a6b041b6
Jul 06 18:30:02 JK-Desktop kernel: 000000a9a6aa1f67 ffffffff810e0bd0 ffff88024ed0e408 0000000000000000
Jul 06 18:30:02 JK-Desktop kernel: Call Trace:
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff815798fc>] schedule_hrtimeout_range_clock.part.6+0xfc/0x120
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff810e0bd0>] ? __run_hrtimer+0x250/0x250
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579891>] ? schedule_hrtimeout_range_clock.part.6+0x91/0x120
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579939>] schedule_hrtimeout_range_clock+0x19/0x50
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579983>] schedule_hrtimeout_range+0x13/0x20
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ec8f4>] poll_schedule_timeout+0x44/0x70
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ee111>] do_sys_poll+0x531/0x600
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ee2e1>] SyS_poll+0x71/0x130
Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff8157a889>] system_call_fastpath+0x12/0x17
Jul 06 18:30:02 JK-Desktop kernel: Code: b8 00 00 48 89 e5 53 48 83 ec 08 48 8b 10 48 85 d2 74 0a 48 83 b8 a8 07 00 00 00 74 27 65 48 8b 1c 25 60 b8 00 00 e8 99 f5 ff ff <48> 8b 83 38 c0 ff ff a8 08 75 f
Jul 06 18:30:02 JK-Desktop kernel: RIP [<ffffffff81576b77>] schedule+0x37/0x90Offline
I attempted to do this as i was setting up the system, but it wants to diosplay everything out the first card in the system, even if the monitors are in the 2600XT in the lower slots. Any way I can force it to?
There's three ways of changing this:
1. Making the host use UEFI too and ignore VGA. That way you'll see UEFI setup menu on all video outputs, but the GPU will be free of any extra burden like VGA.
2. On some Gigabyte motherboards it's possible to select the primary GPU device, to which all VGA stuff will be directed.
3. Live with it, but the VM should use OVMF. Currently my host system is legacy-booted with all VGA stuff routed to 01:00.0(so if i will push ctrl-alt-f1 while VM is off, i'll see a console on the screen), but when the system loads, my nvidia driver kicks in, loads the 04:00.0 GT610 card, and 01:00.0 is not used by host system(as i have my radeon blacklisted, but it's not prone to bind itself when not asked)...
Oh wait, you don't even need OVMF, if x-vga=on works correctly on your platform. For me it doesn't, so i use OVMF.
Anyway, i have the following lines in my xorg.conf line, maybe you'll like it too:
Section "Device"
Identifier "Device0"
Driver "nvidia"
VendorName "NVIDIA Corporation"
BoardName "GeForce GT 610"
BusID "PCI:04:00:00"
EndSectionAs you can see, i specify the device be it's BDF address, so if i would have multiple nvidia GPUs, normally only 04:00.0 would work.
The R9 270X = 7870
The R9 280 = 7950However there are quite a few 7950s on the list being passed sucessfully.
The card's just rare, i guess. Anyway, i don't remember them being any special, just another radeon southern(right?) islands GPU with reset problems that were fixed some time ago.
Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:01 JK-Desktop kernel: AMD-Vi: Completion-Wait loop timed out Jul 06 18:30:02 JK-Desktop kernel: AMD-Vi: Event logged [IOTLB_INV_TIMEOUT device=07:00.0 address=0x00000002451e49e0] Jul 06 18:30:02 JK-Desktop kernel: BUG: unable to handle kernel paging request at ffffffffffffc039 Jul 06 18:30:02 JK-Desktop kernel: IP: [<ffffffff81576b77>] schedule+0x37/0x90 Jul 06 18:30:02 JK-Desktop kernel: PGD 180e067 PUD 1810067 PMD 0 Jul 06 18:30:02 JK-Desktop kernel: Oops: 0000 [#1] PREEMPT SMP Jul 06 18:30:02 JK-Desktop kernel: Modules linked in: nls_utf8 isofs vfio_pci vfio_iommu_type1 vfio it87 hwmon_vid eeepc_wmi asus_wmi sparse_keymap led_class rfkill video kvm_amd kvm fuse crct10dif_pclmu Jul 06 18:30:02 JK-Desktop kernel: hid_logitech ff_memless hid_generic hid_microsoft usbhid hid uas usb_storage atkbd libps2 ahci libahci libata ohci_pci xhci_pci ohci_hcd ehci_pci xhci_hcd ehci_hcd scs Jul 06 18:30:02 JK-Desktop kernel: CPU: 4 PID: 720 Comm: cinnamon Not tainted 4.0.7-2-vfio #1 Jul 06 18:30:02 JK-Desktop kernel: Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 EVO, BIOS 1604 10/16/2012 Jul 06 18:30:02 JK-Desktop kernel: task: ffff88023db2b2a0 ti: ffff8800a4144000 task.ti: ffff8800a4144000 Jul 06 18:30:02 JK-Desktop kernel: RIP: 0010:[<ffffffff81576b77>] [<ffffffff81576b77>] schedule+0x37/0x90 Jul 06 18:30:02 JK-Desktop kernel: RSP: 0018:ffff8800a4147a08 EFLAGS: 00010292 Jul 06 18:30:02 JK-Desktop kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 00000000c0000100 Jul 06 18:30:02 JK-Desktop kernel: RDX: ffff8800a4147fd8 RSI: ffff88023db2b2a0 RDI: ffff88024ed13e40 Jul 06 18:30:02 JK-Desktop kernel: RBP: ffff8800a4147a18 R08: ffff8800a4144000 R09: ffff88024ed13e70 Jul 06 18:30:02 JK-Desktop kernel: R10: 0000000000000020 R11: 0000000000000004 R12: 000000000006224f Jul 06 18:30:02 JK-Desktop kernel: R13: ffff8800a4147b80 R14: ffff8800a4147bec R15: 0000000000000000 Jul 06 18:30:02 JK-Desktop kernel: FS: 00007f532c8c39c0(0000) GS:ffff88024ed00000(0000) knlGS:0000000000000000 Jul 06 18:30:02 JK-Desktop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 06 18:30:02 JK-Desktop kernel: CR2: ffffffffffffc039 CR3: 00000000a4ca2000 CR4: 00000000000407e0 Jul 06 18:30:02 JK-Desktop kernel: Stack: Jul 06 18:30:02 JK-Desktop kernel: ffff8800a4147bec 0000000000000000 ffff8800a4147ab8 ffffffff815798fc Jul 06 18:30:02 JK-Desktop kernel: ffff8800a4147a28 0000000000000000 0000000000000000 000000a9a6b041b6 Jul 06 18:30:02 JK-Desktop kernel: 000000a9a6aa1f67 ffffffff810e0bd0 ffff88024ed0e408 0000000000000000 Jul 06 18:30:02 JK-Desktop kernel: Call Trace: Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff815798fc>] schedule_hrtimeout_range_clock.part.6+0xfc/0x120 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff810e0bd0>] ? __run_hrtimer+0x250/0x250 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579891>] ? schedule_hrtimeout_range_clock.part.6+0x91/0x120 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579939>] schedule_hrtimeout_range_clock+0x19/0x50 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff81579983>] schedule_hrtimeout_range+0x13/0x20 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ec8f4>] poll_schedule_timeout+0x44/0x70 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ee111>] do_sys_poll+0x531/0x600 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811eca70>] ? poll_select_copy_remaining+0x150/0x150 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff811ee2e1>] SyS_poll+0x71/0x130 Jul 06 18:30:02 JK-Desktop kernel: [<ffffffff8157a889>] system_call_fastpath+0x12/0x17 Jul 06 18:30:02 JK-Desktop kernel: Code: b8 00 00 48 89 e5 53 48 83 ec 08 48 8b 10 48 85 d2 74 0a 48 83 b8 a8 07 00 00 00 74 27 65 48 8b 1c 25 60 b8 00 00 e8 99 f5 ff ff <48> 8b 83 38 c0 ff ff a8 08 75 f Jul 06 18:30:02 JK-Desktop kernel: RIP [<ffffffff81576b77>] schedule+0x37/0x90
Actually, this looks very bad. I think only aw or someone his level can give you a proper advice. Strange thing those ioapic remapping commands didn't work.
Last edited by Duelist (2015-07-07 01:57:35)
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
Actually, this looks very bad. I think only aw or someone his level can give you a proper advice. Strange thing those ioapic remapping commands didn't work.
So, in simple terms, I'm screwed because Asus messed up something.
I guess the easy way of fixing this is to replace the board with a known working one, which is hard to do with minimal funds.
Could power problems cause this? I've had suspicions that my PSU is on it's way out for a bit now because of some strange behavior with Windows and that both my previous 8320 and the 7950 had to be RMA'd last year (roughly around the same time as well).
My sensors show this, but I don't know how accurate these sensors are.
it8721-isa-0290
Adapter: ISA adapter
in0: +2.81 V (min = +2.63 V, max = +2.75 V) ALARM
in1: +2.87 V (min = +2.84 V, max = +3.05 V)
in2: +0.80 V (min = +0.00 V, max = +2.72 V)
+3.3V: +3.31 V (min = +5.90 V, max = +2.74 V) ALARM
in4: +2.27 V (min = +2.95 V, max = +1.50 V) ALARM
in5: +2.50 V (min = +1.06 V, max = +2.35 V) ALARM
in6: +1.54 V (min = +0.56 V, max = +2.53 V)
3VSB: +5.40 V (min = +3.02 V, max = +6.10 V)
Vbat: +3.38 V
fan1: 944 RPM (min = 20 RPM)
fan2: 0 RPM (min = 291 RPM) ALARM
fan3: 0 RPM (min = 66 RPM) ALARM
temp1: +30.0°C (low = +97.0°C, high = -18.0°C) ALARM sensor = thermistor
temp2: +27.0°C (low = -56.0°C, high = +94.0°C) sensor = thermistor
temp3: -128.0°C (low = -1.0°C, high = +127.0°C) sensor = disabled
intrusion0: OKI do see some obvious garbage, like fans 2 and 3, that have nothing in them.
Last edited by Zeorymer (2015-07-07 04:27:34)
Offline
Hi guys
I've been working at this all day, but no matter what I do, IOMMU doesn't seem to ever enable, at the kernel level. My processor and mobo both (supposedly) have vt-d support, but dmesg | grep iommu only returns the boot parameters themselves; almost as if they are being completely ignored. My boot options are
intel_iommu=on,igfx_on,pass-through i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbAnd my build is:
ASUS Maximus VII Ranger
Intel i7 4970K (ark.intel does claim this K edition had VT-d support)
Intel HD 4600 iGPU (arch host)
Nvidia GTX 980 (passthrough)
I am so lost, google has been no help.
Update:
Made a bit of a mistake, dmesg | grep IOMMU (as opposed to iommu) returns [0.000000] Intel-IOMMU: enabled, and nothing else. /sys/kernel/iommu_groups is completely empty.
Last edited by morat (2015-07-07 08:38:34)
Offline
Could power problems cause this? I've had suspicions that my PSU is on it's way out for a bit now because of some strange behavior with Windows and that both my previous 8320 and the 7950 had to be RMA'd last year (roughly around the same time as well).
Use the PSU calculator, like outervision's one, and compare calculations with your PSU.
Do not trust software voltmeters, use a physical one and push it into molex(IDE power) or pci-e power connector to find out 12V voltage.
...the motherboards for AM3+ are kinda cheap, however i understand your concern. Try joining #kvm and ask there. I feel an easy fix somewhere near.
And i've forgot, did you update your bios? It'd be useful to try every revision supporting IOMMU.
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
Hi guys
I've been working at this all day, but no matter what I do, IOMMU doesn't seem to ever enable, at the kernel level. My processor and mobo both (supposedly) have vt-d support, but dmesg | grep iommu only returns the boot parameters themselves; almost as if they are being completely ignored. My boot options areintel_iommu=on,igfx_on,pass-through i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbAnd my build is:
ASUS Maximus VII Ranger
Intel i7 4970K (ark.intel does claim this K edition had VT-d support)
Intel HD 4600 iGPU (arch host)
Nvidia GTX 980 (passthrough)I am so lost, google has been no help.
Update:
Made a bit of a mistake, dmesg | grep IOMMU (as opposed to iommu) returns [0.000000] Intel-IOMMU: enabled, and nothing else. /sys/kernel/iommu_groups is completely empty.
What does lsmod | grep vfio says?
It should be like
vfio_iommu_type1 20480 1
vfio_pci 36864 4
vfio 24576 9 vfio_iommu_type1,vfio_pcibut only when vfio is used. Try loading it manually via modprobe.
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
morat wrote:Hi guys
I've been working at this all day, but no matter what I do, IOMMU doesn't seem to ever enable, at the kernel level. My processor and mobo both (supposedly) have vt-d support, but dmesg | grep iommu only returns the boot parameters themselves; almost as if they are being completely ignored. My boot options areintel_iommu=on,igfx_on,pass-through i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbAnd my build is:
ASUS Maximus VII Ranger
Intel i7 4970K (ark.intel does claim this K edition had VT-d support)
Intel HD 4600 iGPU (arch host)
Nvidia GTX 980 (passthrough)I am so lost, google has been no help.
Update:
Made a bit of a mistake, dmesg | grep IOMMU (as opposed to iommu) returns [0.000000] Intel-IOMMU: enabled, and nothing else. /sys/kernel/iommu_groups is completely empty.What does lsmod | grep vfio says?
It should be likevfio_iommu_type1 20480 1 vfio_pci 36864 4 vfio 24576 9 vfio_iommu_type1,vfio_pcibut only when vfio is used. Try loading it manually via modprobe.
vfio is irrelevant until the iommu gets squared away. Is VT-d enabled in the BIOS? Do you have a DMAR line like this in dmesg?
ACPI: DMAR 0x00000000DAFDB000 0000B8 (v01 INTEL SNB 00000001 INTL 00000001)It's always a good idea to pastebin dmesg even if you don't know what you're looking for.
http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses? Try https://www.redhat.com/mailman/listinfo/vfio-users
Offline
Hi there.
I wonder if there's a way to use VFIO-PCI passthrough under Libvirt user session (qemu://session)?
It's been a huge success so far, but I'm using it via Libvirt system session, but it's keeping me from using PulseAudio, so far I keep getting the message that 'PCI device <ID> is not assignable>, is there any way around that?
Offline
Hi there.
I wonder if there's a way to use VFIO-PCI passthrough under Libvirt user session (qemu://session)?
It's been a huge success so far, but I'm using it via Libvirt system session, but it's keeping me from using PulseAudio, so far I keep getting the message that 'PCI device <ID> is not assignable>, is there any way around that?
Not currently
http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses? Try https://www.redhat.com/mailman/listinfo/vfio-users
Offline
What does lsmod | grep vfio says?
It should be likevfio_iommu_type1 20480 1 vfio_pci 36864 4 vfio 24576 9 vfio_iommu_type1,vfio_pcibut only when vfio is used. Try loading it manually via modprobe.
I have loaded vfio, and it does appear
vfio is irrelevant until the iommu gets squared away. Is VT-d enabled in the BIOS? Do you have a DMAR line like this in dmesg?
ACPI: DMAR 0x00000000DAFDB000 0000B8 (v01 INTEL SNB 00000001 INTL 00000001)It's always a good idea to pastebin dmesg even if you don't know what you're looking for.
Something has screwed my iGPU, and I can't get any output from it anymore; so I reset the CMOS, and now IOMMU seems to work. That would be the end of this, but now I cant use the onboard graphics as the display for arch, so can't bind the gpu, as its the only other output. Unfortunately I already have set it up to use the gpu. How can I roll back the changes I made from the OP?
Update:
I can use vga to get output from arch, so here is the dmesg
Last edited by morat (2015-07-08 00:56:41)
Offline
[ 0.000000] Command line: initrd=\initramfs-linux.img iommu=force,calgary intel_iommu=on root=PARTUUID=dc06f381-717e-4143-8e0e-a2ee69b1649c rw i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbSo we're just making up kernel boot option now, right? (calgary)
http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses? Try https://www.redhat.com/mailman/listinfo/vfio-users
Offline
[ 0.000000] Command line: initrd=\initramfs-linux.img iommu=force,calgary intel_iommu=on root=PARTUUID=dc06f381-717e-4143-8e0e-a2ee69b1649c rw i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbSo we're just making up kernel boot option now, right? (calgary)
We are? I thought Calgary was the option to select Intel support.
Offline
aw wrote:[ 0.000000] Command line: initrd=\initramfs-linux.img iommu=force,calgary intel_iommu=on root=PARTUUID=dc06f381-717e-4143-8e0e-a2ee69b1649c rw i915.enable_hd_vgaarb=1 pci-stub.ids=10de:13c0,10de:0fbbSo we're just making up kernel boot option now, right? (calgary)
We are? I thought Calgary was the option to select Intel support.
Ok, even though it's not documented in kenel-parameters.txt calgary is a valid option there... but you don't have a calgary system, I hopedefinitely.
config CALGARY_IOMMU
bool "IBM Calgary IOMMU support"
select SWIOTLB
depends on X86_64 && PCI
---help---
Support for hardware IOMMUs in IBM's xSeries x366 and x460
systems. Needed to run systems with more than 3GB of memory
properly with 32-bit PCI devices that do not support DAC
(Double Address Cycle). Calgary also supports bus level
isolation, where all DMAs pass through the IOMMU. This
prevents them from going anywhere except their intended
destination. This catches hard-to-find kernel bugs and
mis-behaving drivers and devices that do not use the DMA-API
properly to set up their DMA buffers. The IOMMU can be
turned off at boot time with the iommu=off parameter.
Normally the kernel will make the right choice by itself.
If unsure, say Y.Last edited by aw (2015-07-08 16:27:38)
http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses? Try https://www.redhat.com/mailman/listinfo/vfio-users
Offline
Ok, even though it's not documented in kenel-parameters.txt calgary is a valid option there... but you don't have a calgary system,
I hopedefinitely.config CALGARY_IOMMU bool "IBM Calgary IOMMU support" select SWIOTLB depends on X86_64 && PCI ---help--- Support for hardware IOMMUs in IBM's xSeries x366 and x460 systems. Needed to run systems with more than 3GB of memory properly with 32-bit PCI devices that do not support DAC (Double Address Cycle). Calgary also supports bus level isolation, where all DMAs pass through the IOMMU. This prevents them from going anywhere except their intended destination. This catches hard-to-find kernel bugs and mis-behaving drivers and devices that do not use the DMA-API properly to set up their DMA buffers. The IOMMU can be turned off at boot time with the iommu=off parameter. Normally the kernel will make the right choice by itself. If unsure, say Y.
Ah, right, OK. Well that's good to know, but IOMMU is already functioning properly, as per my last post. It looks like the dvi port on the mobo is busted, for reasons unknown, VGA and HDMI work fine, but running the vfio-bind script on the GPU crashes the computer, and seems to completely halt the CPU (motherboard opcode). Any ideas?
Last edited by morat (2015-07-09 01:17:32)
Offline
Ah, right, OK. Well that's good to know, but IOMMU is already functioning properly, as per my last post. It looks like the dvi port on the mobo is busted, for reasons unknown, VGA and HDMI work fine, but running the vfio-bind script on the GPU crashes the computer, and seems to completely halt the CPU (motherboard opcode). Any ideas?
IOMMU doesn't work right, and vfio-bind script sucks.
A fresh guide to follow.
@nbhs: mark the op-post as somewhat outdated and post a huge link to aw's blog, that'll help a lot, thanks!
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline
IOMMU doesn't work right, and vfio-bind script sucks.
A fresh guide to follow.@nbhs: mark the op-post as somewhat outdated and post a huge link to aw's blog, that'll help a lot, thanks!
Excellent, thanks; and even moreso to @aw. I'll let you know if it works.
Will I need to rollback what I did from OP's guide, and if so, how?
Offline
Excellent, thanks; and even moreso to @aw. I'll let you know if it works.
Will I need to rollback what I did from OP's guide, and if so, how?
Depending on where you've stopped. Disable the vfio-bind systemd service, delete its' remains, use a recent qemu and kernel - and you should be good to go for following aw's guide.
Gratitude is always welcome, but try reading modinfo vfio, he wrote that...
Last edited by Duelist (2015-07-09 14:41:17)
The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.
Offline