You are not logged in.

#1 2018-07-05 04:37:34

Toolybird
Member
Registered: 2017-09-30
Posts: 72

[SOLVED] virgl with libvirt (QEMU/KVM)

Can anyone get virgl working with libvirt?

virgl works great when using plain qemu (even with spice gl !)

But for the life of me, I just cannot get it working via libvirt (virsh / virt-manager)

VM fails to boot and I'm left with a zombie qemu process.

Logged an upstream bug report but no response so far:

https://bugzilla.redhat.com/show_bug.cgi?id=1594456

Also tried self compiled latest release libvirt-4.5.0 but no change.

Anyone?

Last edited by Toolybird (2018-07-10 21:44:16)

Offline

#2 2018-07-05 08:58:03

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

It would really help if you actually posted the error messages, maybe some logs, your QEMU cmdline call, your libvirt configuration etcetc.
Without any of that information this post is like you have a problem and just want us to magically fix it for you.
https://wiki.archlinux.org/index.php/Co … ow_to_post

Assuming the libvirt services are running properly and as libvirt is basically just a front to QEMU (and other hypervisors), I'd suspect you just have a difference in your QEMU cmdline vs your libvirt config.
No way to say for sure without any information whatsoever though.

Last edited by Omar007 (2018-07-05 08:58:38)

Offline

#3 2018-07-05 20:51:32

Toolybird
Member
Registered: 2017-09-30
Posts: 72

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

Omar007 wrote:

It would really help if you actually posted the error messages

Er, full xml dump, log files, error details etc are contained within the linked upstream bug report which apparently you completely missed?

I'll re-phrase the question in order to make it 100% clear.

Can anyone successfully boot a virgl enabled VM when using libvirt (ie: virt-manager or virsh)?

For example, use virt-manager to:

  • create a Linux guest VM using latest live iso from Fedora or Ubuntu

  • change Video Model to Virtio with 3D acceleration ticked

  • change Display Spice to Listen type None with OpenGL ticked

The XML should contain something like the following:

    <graphics type='spice'>
      <listen type='none'/>
      <image compression='off'/>
      <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:0a:00.0-render'/>
    </graphics>
    <video>
      <model type='virtio' heads='1' primary='yes'>
        <acceleration accel3d='yes'/>
      </model>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>

virgl is confimed as working if:

  • the VM actually boots smile

  • "dmesg | grep virgl" from within booted VM shows as enabled

Thanks

Offline

#4 2018-07-06 09:01:55

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

You're right, I missed attachments you never mentioned would be on that link.
Even then I'm still missing the command you use for QEMU that you say is working fine so it's still not possible to determine possible config differences that may trigger this for you.
I suspect at the very least the user running it is different as you're probably running it under your own user when you run it manually, while libvirt runs it under nobody:kvm by default.

As such, for the permission error it throws, you could try to set MESA_GLSL_CACHE_DIR to a location that actually exists and it has access to.
It normally default to $XDG_CACHE_HOME/mesa or to .cache/mesa in the user's home if XDG_CACHE_HOME is not set but this fails here because for the nobody user this is '/', for which you obv. do not have permission.
That should at the very least get rid of this problem and clean up these errors. If it still fails, provide the logs again.

Offline

#5 2018-07-07 03:19:40

Toolybird
Member
Registered: 2017-09-30
Posts: 72

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

Thanks for responses so far.

Please forget about the user account and the mesa cache thing. Now that I've established that the problem also occurs in the Libvirt User Session
permissions basically don't matter because QEMU is running as your normal user when using qemu:///session

If anyone is playing along at home and would like to test this, all you need is virt-manager and a Fedora 28 iso file.

For reference, the following QEMU command works fine. There is a tiny amount of screen glitch during bootup and the mouse is a little wonky (but switching back and forth between full screen and windowed seems to sort it):

qemu-system-x86_64 \
  -accel kvm -machine pc -smp 2 -m 2G \
  -cdrom Fedora-Workstation-Live-x86_64-28-1.1.iso \
  -usb -device usb-tablet \
  -vga virtio -display gtk,gl=on

This next QEMU command is employing SPICE using Spice User Manual as a guide. You have to use remote-viewer to connect to the display (see below for the command). When connected via remote-viewer with SPICE I cannot see the bootloader screen and there is all sorts of screen corruption during the bootup. But after a while the graphical drivers get loaded and the display comes good:

QEMU_AUDIO_DRV=spice \
qemu-system-x86_64 \
  -machine pc,vmport=off \
  -accel kvm -smp 2 -m 2G \
  -cdrom Fedora-Workstation-Live-x86_64-28-1.1.iso \
  -usb -device usb-tablet \
  -soundhw hda \
  -device virtio-vga,virgl=on \
  -spice gl=on,unix,addr=/tmp/spice.sock,disable-ticketing \
  -device virtio-serial -chardev spicevmc,id=vdagent,debug=0,name=vdagent \
  -device virtserialport,chardev=vdagent,name=com.redhat.spice.0
remote-viewer spice+unix:///tmp/spice.sock

Now for the whole point of this exercise - you need to have the QEMU User Session added as a connection in virt-manager (File/Add Connection/ Hypervisor QEMU/KVM user session). Grab the XML from https://ptpb.pw/Eatl, call it virgl.xml, then execute the following:

virsh -c qemu:///session define virgl.xml

Now go to virt-manager and adjust any needed VM settings (possibly CPU type, connect the Fedora iso etc). You might also have to go into "Display Spice" and click on the dropdown to change from "Auto" to actual render node.

Does it boot for you? Is virgl active?

Thanks

Offline

#6 2018-07-10 06:28:29

Toolybird
Member
Registered: 2017-09-30
Posts: 72

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

Made a breakthrough!

Here's the distilled test case. This command boots:

qemu-system-x86_64 -vga virtio -display gtk,gl=on -sandbox on,resourcecontrol=allow

This one doesn't:

qemu-system-x86_64 -vga virtio -display gtk,gl=on -sandbox on,resourcecontrol=deny

Could someone please run those commands and confirm? Thanks!

Offline

#7 2018-07-10 21:43:47

Toolybird
Member
Registered: 2017-09-30
Posts: 72

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

Finally got some response from upstream.

Short story - QEMU bug.

Longer story - libvirt calls qemu with seccomp enabled which blocks kernel syscall involved in MESA shader cache

Patch has been proposed:

http://lists.nongnu.org/archive/html/qe … 02411.html

Marking this as solved.

Offline

#8 2018-07-11 12:50:24

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

So it was not a libvirt problem.
Nice troubleshooting, Toolybird.


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#9 2018-08-05 13:30:05

bergentroll
Member
Registered: 2016-08-23
Posts: 30

Re: [SOLVED] virgl with libvirt (QEMU/KVM)

Thanks, Toolybird!
I have installed qemu-git, and gl works now.

Offline

Board footer

Powered by FluxBB