[solved] Both vdpauinfo and vainfo dump core with mesa drivers 19.1.x

Jason P. · 2019-07-02 15:51:19

Hi,

This issue is probably the root cause for https://bbs.archlinux.org/viewtopic.php?id=247216. I prefer to abandon the previous thread since the problem is not directly related nor specific of VLC.

These are the video drivers, libraries and firmware installed on my system:

mesa 19.1.1-1
mesa-vdpau 19.1.1-1
libva-mesa-driver 19.1.1-1
libvdpau 1.2-1
libva 2.4.1-1
nouveau-fw 325.15-1

So, when i try to test vdpau and libva setup both test tools exit dumping core:

vdpauinfo dump: https://termbin.com/qp52
vainfo dump: https://termbin.com/se6n

These are my environment variables for video drivers:

➜  $ ~ cat .pam_environment 
LIBVA_DRIVER_NAME=nouveau
VDPAU_DRIVER=nouveau

What can be happening? So far I can play videos within the browser but not any movie or stream with mplayer or VLC.

Many thanks.

Last edited by Jason P. (2019-09-27 10:49:30)

Lone_Wolf · 2019-07-02 16:38:56

post full terminal output of those commands please.

Also your xorg log, glxinfo -B and lspci -k

Is xf86-video-nouveau installed ?

Last edited by Lone_Wolf (2019-07-02 16:39:34)

Jason P. · 2019-07-02 17:43:39

Terminal output

➜  $ ~ vdpauinfo 
display: :1   screen: 0
[1]    6656 segmentation fault (core dumped)  vdpauinfo

➜  $ ~ vainfo 
[1]    6715 segmentation fault (core dumped)  vainfo

➜  $ ~ glxinfo -B
name of display: :1
display: :1  screen: 0
direct rendering: Yes
Extended renderer info (GLX_MESA_query_renderer):
    Vendor: nouveau (0x10de)
    Device: NVC3 (0xdc4)
    Version: 19.1.1
    Accelerated: yes
    Video memory: 1007MB
    Unified memory: no
    Preferred profile: core (0x1)
    Max core profile version: 4.3
    Max compat profile version: 4.3
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.1
OpenGL vendor string: nouveau
OpenGL renderer string: NVC3
OpenGL core profile version string: 4.3 (Core Profile) Mesa 19.1.1
OpenGL core profile shading language version string: 4.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile

OpenGL version string: 4.3 (Compatibility Profile) Mesa 19.1.1
OpenGL shading language version string: 4.30
OpenGL context flags: (none)
OpenGL profile mask: compatibility profile

OpenGL ES profile version string: OpenGL ES 3.1 Mesa 19.1.1
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10

➜  $ ~ lspci -v -s 01:00     
01:00.0 VGA compatible controller: NVIDIA Corporation GF106 [GeForce GTS 450] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: Gigabyte Technology Co., Ltd GF106 [GeForce GTS 450]
	Flags: bus master, fast devsel, latency 0, IRQ 29
	Memory at f4000000 (32-bit, non-prefetchable) [size=16M]
	Memory at e0000000 (64-bit, prefetchable) [size=128M]
	Memory at e8000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: <access denied>
	Kernel driver in use: nouveau
	Kernel modules: nouveau

01:00.1 Audio device: NVIDIA Corporation GF106 High Definition Audio Controller (rev a1)
	Subsystem: Gigabyte Technology Co., Ltd GF106 High Definition Audio Controller
	Flags: bus master, fast devsel, latency 0, IRQ 17
	Memory at f6080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: <access denied>
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

Logs

Lone_Wolf wrote:

Is xf86-video-nouveau installed ?

Nope. I've been using just the mesa driver alone for several months and so far it was good. Problems with video playback appeared recently. Latest mesa updates are causing troubles for some reason...

Is xf86-video-intel still needed? question on Reddit wrote:

modesetting uses glamor for 2D acceleration which uses OpenGL (Mesa). So on Arch specifically you only install Mesa on a new system and "it just works".

Reddit discussion.

Lone_Wolf · 2019-07-02 18:53:35

That reddit poster assumes a lot of things.
To make glamor "just work" xorg modesetting (xorg-server package) needs to be installed, mesa is indeed needed but the hardware/software platfrom also needs to support EGL.
EGL doesn't work well on some platforms, especially older ones.

[   105.241] (II) Loading sub module "glamoregl"
[   105.241] (II) LoadModule: "glamoregl"
[   105.241] (II) Loading /usr/lib/xorg/modules/libglamoregl.so
[   105.244] (II) Module glamoregl: vendor="X.Org Foundation"
[   105.244] 	compiled for 1.20.5, module version = 1.0.1
[   105.244] 	ABI class: X.Org ANSI C Emulation, version 0.4
[   105.348] (II) modeset(0): glamor X acceleration enabled on NVC3
[   105.349] (II) modeset(0): glamor initialized

In my opinion intel /nouveau users should start with xorg modesetting, but if that gives problems try xf86-video-* as alternative.

I haven' followed the other thread, have you tried booting with the LTS kernel (if you're using lts kernel now, try the stock archlinux kernel )
Did the troubles start before or after you switched to mesa 19.1.x ?

Jason P. · 2019-07-03 15:20:48

Thanks for your reply Lone_Wolf.

Is there anything wrong with the piece of Xorg log you've pointed out?

In my opinion intel /nouveau users should start with xorg modesetting, but if that gives problems try xf86-video-* as alternative.

Well, my path was the other way around. For years I've had installed on my Arch PC the xf86-video-nouveau package. Several months ago I discovered that I could live without it and the alternative (Xorg modesetting) was more actively developed and meant to be the go-to solution. Also I've noticed better overall experience (less tearing glitches).

➜  $ ~ uname -a
Linux altair 5.1.15-arch1-1-ARCH #1 SMP PREEMPT Tue Jun 25 04:49:39 UTC 2019 x86_64 GNU/Linux

I'm running stock Arch Linux kernel. I can boot into another version I have installed and try. It's 4.19.25.rt16-9. Let's see.

Did the troubles start before or after you switched to mesa 19.1.x ?

I cannot pinpoint to a particular version but I'm sure the problem didn't exist when I moved to Xorg modesetting. It must have happened a few versions ago.

Jason P. · 2019-07-03 15:36:09

I'm running stock Arch Linux kernel. I can boot into another version I have installed and try. It's 4.19.25.rt16-9. Let's see.

Same problem booting into that kernel.

How could I get more debugging info about what's crashing?

V1del · 2019-07-03 15:47:32

I really don't think you are far off with your initial assumption of a mesa regression, what happens if you downgrade that to a pre 19.1 release?

Jason P. · 2019-07-03 17:31:25

Well, I've tried downgrading these three packages together to their previous versions 19.1.0-3 and 19.1.0-2:

mesa
mesa-vdpau
libva-mesa-driver

Results were the same. Apart from that I've found some Xorg errors and warnings in boot logs. First three look particularly strange...

jul 03 19:14:22 gsd-xsettings[757]: Failed to get current display configuration state: GDBus.Error:org.freedesktop.DBus.Error.NameHasNoOwner: Name "org.gnome.Mutter.DisplayConfig" does not exist
jul 03 19:14:27 /usr/lib/gdm-x-session[664]: (EE) modeset(0): failed to set mode: Permission denied
jul 03 19:14:27 /usr/lib/gdm-x-session[917]: _XSERVTransSocketUNIXCreateListener: ...SocketCreateListener() failed
jul 03 19:14:27 /usr/lib/gdm-x-session[917]: (WW) Failed to open protocol names file lib/xorg/protocol.txt
jul 03 19:14:27 /usr/lib/gdm-x-session[917]: (WW) Open ACPI failed (/var/run/acpid.socket) (No such file or directory)
jul 03 19:14:27 /usr/lib/gdm-x-session[917]: (EE) Failed to load module "nouveau" (module does not exist, 0)
jul 03 19:14:27 /usr/lib/gdm-x-session[917]: (EE) Failed to load module "nv" (module does not exist, 0)
jul 03 19:14:28 /usr/lib/gdm-x-session[917]: (EE) Failed to load module "fbdev" (module does not exist, 0)
jul 03 19:14:28 /usr/lib/gdm-x-session[917]: (EE) Failed to load module "vesa" (module does not exist, 0)

Update: in the next boot the line complaining about "failed to set mode: Permission denied" does not appear anymore.

Last edited by Jason P. (2019-07-03 17:40:21)

Jason P. · 2019-07-05 17:41:31

I've finally discovered which exact package introduced the regression: mesa-19.1.0-1. With the version previous to that one I had no crash using vdpauinfo or vainfo.

I've checked upstream repository and this minor version bump had introduced a lot of changes. Where can I start looking?

Lone_Wolf · 2019-07-05 21:38:02

mesa 19.0.x to mesa 19.1.x is NOT a minor version bump.

mesa 19.0 was [url=https://cgit.freedesktop.org/mesa/mesa/commit/?h=19.0-branchpoint&id=90a7a9c973193929ec758057f82c612fb69890a1]
branched of from master on jan 29 2019[/url]

From then on mesa 19.0.x had its own tree.

Mesa master continued being development and only cherry-picked commits were backported from master to 19,0.x

On may 4 2019 mesa 19,1 was branched of from master.

Between those 2 branchpoints mesa gained over 3000 commits, most of which were not backported to 19.0 .

You'll have to use Git bisecting .
I estimate you'll have to build mesa atleast 12 times to pinpoint the first bad commit .

If you go this route I suggest you first build/test my aur mesa-git or Lordheavy's mesa-git (from his unofficial repo ) to see if the issue is already solved in mesa master.

liewkj · 2019-07-08 06:33:28

I have had the same issue for some time. Kudos to Jason P. for narrowing down the affected 3 mesa packages!

I signed up the forum to post this in hope that we will get the solution from upstream eventually, and to say thanks for Jason P.

I also want to add a few details:
- I have 2 PCs running ArchLinux at the latest with upstream kernel. Both with NVIDIA graphics but different family. The old PC has GeForce 9400 and nouveau with the latest mesa-19.1-x work just great. GeForce 9400 no longer gets NVIDIA propriety binary driver support and the last version nvidia-341xxx is less polished than the open-source nouveau. It is missing RenderNode support in recent DRI2 implementation. On the other hand, nouveau supports DRI2/RenderNode for GeForce 9400, all the rightful video acceleraion (VAAPI/VDPAU) for the supported video formats by the hardware and OpenGL/GLES performance figures from glmark2 are very close to nvidia-341xx. The new PC has GeForce GT730 (GK208B) and this was the problem with nouveau. VAAPI/VDPAU simply segfault'ed and refused to function. Although GK208B also no longer gets NVIDIA propriety binary driver support, it has more recent nvidia-390xxx which provides proper DRI2/RenderNode support. All video acceleration (VAAPI/VDPAU) works with nvidia-390xxx. OpenGL/GLES performance figures are much better than nouveau (even after reclocking). So, it is very likely that people would simply use nvidia-390xxx and stop the hassle of reporting and get the issue fixed upstream.
- Even though VAAPI/VDPAU segfault'ed with nouveau, OpenGL/GLES are fine with glxgears and glmark2.
- I can confirm that the interim solution of downgrading the 3 mesa packages back to 19.0-x works.
- dmesg has the code byte stream upon segfault. Both vainfo and vdpauinfo showed the same code byte stream

[ 4767.737193] vainfo[4651]: segfault at 38 ip 00007f903a4cded6 sp 00007fff69472a40 error 4 in nouveau_drv_video.so[7f903a236000+489000]
[ 4767.737198] Code: 48 81 fa e0 01 00 00 0f 8f 47 01 00 00 48 8b 40 08 48 8b 50 40 48 83 c0 40 48 39 c2 0f 84 4a 98 d6 ff 48 8b 42 10 48 8b 68 10 <f7> 45 38 ff ff 10 00 75 08 8b 45 28 39 45 24 74 29 48 8b 44 24 28
[ 4841.393458] vdpauinfo[4686]: segfault at 38 ip 00007f3ffddf6cd6 sp 00007ffc3d66b720 error 4 in libvdpau_nouveau.so.1.0.0[7f3ffdb28000+4c5000]
[ 4841.393465] Code: 48 81 fa e0 01 00 00 0f 8f 47 01 00 00 48 8b 40 08 48 8b 50 40 48 83 c0 40 48 39 c2 0f 84 90 2a d3 ff 48 8b 42 10 48 8b 68 10 <f7> 45 38 ff ff 10 00 75 08 8b 45 28 39 45 24 74 29 48 8b 44 24 28

I am interested to be able to switch to open source nouveau driver for my GeForce GT730, too, since nvidia-390xxx does not get any future improvements. By staying close to upstream kernels and mesa stacks in ArchLinux, hopefully nouveau will achieve 90% of nvidia-390xxx in OpenGL performance, just like the older GeForce 9400. For non-gaming use cases, such as DRI2/RenderNode and native Wayland support, nouveau has an upper hand just by being open source driver.

Last edited by liewkj (2019-07-08 06:55:49)

Jason P. · 2019-07-09 15:37:58

Lone_Wolf wrote:

mesa 19.0.x to mesa 19.1.x is NOT a minor version bump.

I was referring to minor version as in the semantic versioning scheme, not because there were no important changes. Thanks for your suggestions on how to dig deeper to fix this I will try whenever I can afford some time for this. How long does it usually take to build mesa? My PC it's somehow modest in resources.

liewkj wrote:

I signed up the forum to post this in hope that we will get the solution from upstream eventually, and to say thanks for Jason P.

You're welcome . This is a collaborative effort.

valera_cr · 2019-07-10 02:38:38

The latest versions don't work either.

Lordheavy's mesa-git (last updated 08-Jul-2019 06:42):
https://pkgbuild.com/~lcarlier/mesa-git/x86_64/

mesa 19.1.2-1 (last updated 2019-07-09 19:54 UTC)
https://www.archlinux.org/packages/extra/x86_64/mesa/

Lone_Wolf · 2019-07-10 09:18:16

Before 2017 mesa did use semantic versioning (or something similar) and incremented the major version only when something significant happened.

examples :
Mesa 12.x brought OpenGL 4.3 support, mesa 13.x brought OpenGL 4.5 support.

The mesa developers discussed release scheduling and versioning and settled on quarterly releases without fixed release dates..
Since then x.y.z has been used like this : x is the year in 2 digits, y is the quarter in 1 digt, z is the suffix .
mesa 17 is the first that uses this scheme.

As for building mesa trunk :
llvm trunk is by far the most time consuming, but for bisecting I suggest you build against extra llvm .

My build system has a 12 core / 24 threads processor, 16 GiB ram and an nvme2 SSD.
I use ccache so sourcecode that hasn't changed will not be rebuilt but taken from cache, speeding up things a lot.
If I disable ccache mesa-git takes less then 10 minutes to build.

On my previous system, an 8 core opteron from 2009 building mesa took 20 to 60 minutes depending on how much was changed since last build.

For a big bisect like this one I strongly advise to setup ccache to speed up building and also set SRCDEST in makepkg.conf to avoid having to clone mesa every build (see man makepkg) .
Apart from the flags for bisecting, I also recommend using --cleanbuild to remove $srcdir .

Keep in mind that as far as I know ccache and $SRCDEST are not usable with devtools.

Last edited by Lone_Wolf (2019-07-10 09:18:56)

valera_cr · 2019-07-12 11:40:11

After removal:

sudo pacman -R  lib32-mesa-vdpau libva-mesa-driver mesa-vdpau

And updates:

sudo pacman -S  mesa lib32-mesa opencl-mesa

to versions 19.1.2-1 vlc and mplayer work fine.

liewkj · 2019-07-13 06:19:05

valera_cr wrote:

After removal:
sudo pacman -R  lib32-mesa-vdpau libva-mesa-driver mesa-vdpau
And updates:
sudo pacman -S  mesa lib32-mesa opencl-mesa
to versions 19.1.2-1 vlc and mplayer work fine.

This is not a solution. You simply removed the vaapi/vdpau drivers and the media players will be running without video acceleration. Please use vainfo and vdpauinfo to verify if video acceleration is actually available.

liewkj · 2019-07-14 05:54:49

Found the code that SIGSEGV, and here's the temporary patch for mesa-19.1.2:

diff -ru ../orig/mesa-19.1.2/src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp ./src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp
--- ../orig/mesa-19.1.2/src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp	2019-07-09 02:10:44.000000000 -0700
+++ ./src/gallium/drivers/nouveau/codegen/nv50_ir_peephole.cpp	2019-07-13 22:18:05.291514392 -0700
@@ -2080,7 +2080,11 @@
 AlgebraicOpt::handleCVT_CVT(Instruction *cvt)
 {
    Instruction *insn = cvt->getSrc(0)->getInsn();
-   RoundMode rnd = insn->rnd;
+   RoundMode rnd;
+
+   if (insn == NULL)
+       return;
+   rnd = insn->rnd;
 
    if (insn->saturate ||
        insn->subOp ||

I did not dig into why insn would receive a NULL pointer. Other instruction handlers do the same without the need to check insn for NULL before dereferencing the pointer. I would probably leave it to upstream to figure out.

With this simple patch, both vainfo and vdpauinfo work.

$ vainfo
vainfo: VA-API version: 1.5 (libva 2.5.0)
vainfo: Driver version: Mesa Gallium driver 19.1.2 for NV106
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

I would simply remove mesa-vdpau and libva-mesa-driver altogether then copy and restore nouveau libva/vdpau drivers and their symlinks by hand for the meantime. Once this is fixed in upstream, then this can be cleaned up easily before reinstalling the packages.

Lone_Wolf · 2019-07-14 11:05:20

Nice work, iiewkj .

Could you create a bug report about this so mesa developers become aware of the problem and patch ?
https://bugs.freedesktop.org/

Jason P. · 2019-07-15 09:52:45

Many thanks liewkj for finding the problematic code. Cheers.

ewaller · 2019-07-15 14:20:13

Closing.

Jason P., Should you need this reopened, please use the report link and drop the moderators a note. Thanks.

ewaller · 2019-07-23 14:12:00

Reopened by request

Jason P. · 2019-07-23 16:24:29

liewkj, have you opened a bug on the issue tracker? I cannot find it. If mesa developers are not aware of this issue maybe we will not see an upstream fix any time soon.

I can file the bug myself but wanted to ask before.

Jason P. · 2019-07-23 17:25:07

Btw, just updated mesa to 19.1.3-1 and bug was not solved.

liewkj · 2019-07-24 09:17:25

No I didn't file the bug yet. I think libva-mesa-driver has serious issues with vaapi. Even though vainfo reported things looked OK, Chromium-vaapi and totem (using gstreamer-vaapi) did not work as expected. On Geforce 9400, h.264 video played as junk and crashed both Chromium and totem. On Geforce GT730 (GK208B), totem refused to play the same h.264 video while Chromium fell back to FFmpegVideoDecoder from MojoVideoDecoder. I am investigating this on other GPUs at my disposal, Radeon HD 6290, Ryzen Mobile Vega 8 and Intel Haswell HD 4400.

Lone_Wolf · 2019-07-24 10:42:13

You are aware that chromium-vaapi has its own problems and mpv with hwdec
tends to be advised as the most reliable test for hw acceleration ?

Also everything in this thread and the previous one indicates Jason P. found a bug in mesa and you found a workaround.
That bugs needs to be reported to mesa devs so they can investigate further and solve the underlying issue.

Arch Linux

#1 2019-07-02 15:51:19