You are not logged in.

#1 2015-04-23 10:21:18

wildtron
Member
Registered: 2015-04-23
Posts: 4

Coredumps on most applications

I'm receiving errors from applications that i'm currently running. This all started with vim.
vim
Screenshot

I traced the problem to be from glibc-2.21-3 as libpthread is from there. 

I trield downgrading the kernel, and downgrading glibc but the problem still perisists.

with using gdb to try to debug i got
gdb trace

Last edited by wildtron (2015-04-23 10:53:48)

Offline

#2 2015-04-23 11:36:59

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

This reminds me of this problem. What packages did you update besides the kernel and glibc? Could you post your pacman.log?

Offline

#3 2015-04-23 14:19:46

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

I'm getting this every 9 to 10 seconds

Apr 23 15:13:23 arch64 kernel: traps: nvidia-settings[1870] general protection ip:7efe59926150 sp:7fff2087cac8 error:0 in libpthread-2.21.so[7efe59914000+18000]
Apr 23 15:13:23 arch64 kernel: traps: nvidia-settings[1872] general protection ip:7fb4d0695150 sp:7fff87a2b0b8 error:0 in libpthread-2.21.so[7fb4d0683000+18000]
Apr 23 15:13:23 arch64 kernel: traps: nvidia-settings[1874] general protection ip:7f4c6c031150 sp:7ffdd86f9af8 error:0 in libpthread-2.21.so[7f4c6c01f000+18000]
Apr 23 15:13:23 arch64 systemd-coredump[1871]: Process 1870 (nvidia-settings) of user 1000 dumped core.
Apr 23 15:13:23 arch64 systemd-coredump[1873]: Process 1872 (nvidia-settings) of user 1000 dumped core.
Apr 23 15:13:23 arch64 systemd-coredump[1875]: Process 1874 (nvidia-settings) of user 1000 dumped core.

I don't know how long it's been going on as I only noticed it this morning. Coincidentally I've just received a glibc update

[2015-04-23 15:00] [ALPM] upgraded glibc (2.21-2 -> 2.21-3)

but this has made no difference (I'm aware of), It's the same output as before.

I've had a few instances with roxterm also but otherwise it's nvidia-settings constantly.

Edit: add Firefox (once) and Emerald (twice) to that list.

Last edited by fabertawe (2015-04-23 15:18:19)


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#4 2015-04-23 14:41:42

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

It's entirely possible that the "bug" isn't actually in glibc (see the linked post in my first comment) but in another package. So what packages besides glibc have been updated on your machines lately?

Last edited by MK13 (2015-04-23 14:42:43)

Offline

#5 2015-04-23 14:52:38

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

@MK13 - fair enough, you shouldn't have to repeat yourself, apologies. Here's the last couple of days...

[2015-04-22 16:32] [ALPM] upgraded lib32-systemd (218-1 -> 219-1)
[2015-04-22 16:32] [ALPM] upgraded wine (1.7.40-1 -> 1.7.41-1)
[2015-04-23 09:29] [ALPM] upgraded git (2.3.5-1 -> 2.3.6-1)
[2015-04-23 09:29] [ALPM] upgraded nvidia-utils (346.59-1 -> 349.16-1)
[2015-04-23 09:29] [ALPM] upgraded nvidia-libgl (346.59-1 -> 349.16-1)
[2015-04-23 09:29] [ALPM] upgraded hplip (3.15.2-3 -> 3.15.4-1)
[2015-04-23 09:29] [ALPM] upgraded lib32-nvidia-utils (346.59-1 -> 349.16-1)
[2015-04-23 09:29] [ALPM] upgraded lib32-nvidia-libgl (346.59-1 -> 349.16-1)
[2015-04-23 09:29] [ALPM] upgraded libinput (0.13.0-2 -> 0.14.1-1)
[2015-04-23 09:29] [ALPM] upgraded man-pages (3.82-1 -> 3.83-1)
[2015-04-23 09:29] [ALPM] upgraded supertuxkart (0.9rc1-2 -> 0.9-1)
[2015-04-23 09:31] [ALPM] upgraded arj (3.10.22-8 -> 3.10.22-10)
[2015-04-23 15:00] [ALPM] upgraded glibc (2.21-2 -> 2.21-3)
[2015-04-23 15:00] [ALPM] upgraded gpac (5324-1 -> 1:0.5.2-1)

If it's not glibc then what could affect multiple seemingly disparate packages?

Edit: well it was the glaringly obvious in this case - the nvidia packages. Downgrading to 346.59-1 stopped the errors. I should have tried that before posting!

Last edited by fabertawe (2015-04-23 15:17:23)


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#6 2015-04-23 15:25:24

igork
Member
Registered: 2015-04-23
Posts: 3

Re: Coredumps on most applications

Hello,

I've the same problem on my fresh install, every desktop application crashes on exit.
I did some research and looks like it happens in nvidia drivers, stack trace is always similar for any app:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7bd0150 in __lll_unlock_elision () from /usr/lib/libpthread.so.0
(gdb) bt full
#0  0x00007ffff7bd0150 in __lll_unlock_elision () from /usr/lib/libpthread.so.0
No symbol table info available.
#1  0x00007fffe85e212c in ?? () from /usr/lib/libEGL.so.1
No symbol table info available.
#2  0x00007fffe85748c2 in ?? () from /usr/lib/libEGL.so.1
No symbol table info available.
#3  0x00007fffffffe8c0 in ?? ()
No symbol table info available.
#4  0x00007fffe85f3831 in ?? () from /usr/lib/libEGL.so.1
No symbol table info available.
#5  0x00007fffffffe8c0 in ?? ()
No symbol table info available.
#6  0x00007ffff7dea815 in _dl_fini () from /lib64/ld-linux-x86-64.so.2
No symbol table info available.
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

After googling, i've found that this problem occurs when:
* software runs on intel haswell cpu (you can check for 'hle' flag in /proc/cpuinfo)
* and glibc configured to use elision locks for mutexes
* and some software trying to unlock mutex that doesn't locked

this case looks like libEGL trying to do that.

i've no idea how to workaround that (i'm arch newbie), probably to rollback nvidia drivers or build custom glibc or disable hardware lock elision in cpu somehow?

Offline

#7 2015-04-23 15:37:45

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

fabertawe wrote:

@MK13 - fair enough, you shouldn't have to repeat yourself, apologies.

I have to admit that my post sounded a bit rude, that wasn't my intention, so no need for an apology tongue

fabertawe wrote:

If it's not glibc then what could affect multiple seemingly disparate packages?

In the topic I referred to the problem was that after a suspend/resume-cycle the microcode update wasn't applied to all CPU cores (which was also reported via /proc, but glibc uses its own feature-detection mechanism and the discrepancy caused the error) which led to invalid opcode traps during execution. It only happened on certain CPU models though.

fabertawe wrote:

Edit: well it was the glaringly obvious in this case - the nvidia packages. Downgrading to 346.59-1 stopped the errors. I should have tried that before posting!

That would have been my guess as well, maybe the OP can confirm that it fixes the errors for him too?

igork wrote:

i've no idea how to workaround that (i'm arch newbie), probably to rollback nvidia drivers or build custom glibc or disable hardware lock elision in cpu somehow?

Rolling back the nVidia drivers should be alright as a workaround but it doesn't fix the actual problem. You could try to install the intel-ucode package since (AFAIR) it disables HLE.

Last edited by MK13 (2015-04-23 15:42:06)

Offline

#8 2015-04-23 15:51:25

igork
Member
Registered: 2015-04-23
Posts: 3

Re: Coredumps on most applications

MK13 wrote:

Rolling back the nVidia drivers should be alright as a workaround but it doesn't fix the actual problem. You could try to install the intel-ucode package since (AFAIR) it disables HLE.

Disabling HLE with microcode update helped, thanks a lot! smile

Offline

#9 2015-04-23 15:52:24

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

MK13 wrote:

You could try to install the intel-ucode package since (AFAIR) it disables HLE.

It wasn't installed so I tried it but it hasn't worked. Thanks for the suggestion though.


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#10 2015-04-23 15:58:38

igork
Member
Registered: 2015-04-23
Posts: 3

Re: Coredumps on most applications

fabertawe wrote:

It wasn't installed so I tried it but it hasn't worked. Thanks for the suggestion though.

fabertawe, you need also modify boot options to load microcode at startup - https://wiki.archlinux.org/index.php/Microcode
in my case i'm using grub and modified grub.conf like that:


from :

initrd   /boot/initramfs-linux.img

to:

initrd   /boot/intel-ucode.img /boot/initramfs-linux.img

Last edited by igork (2015-04-23 15:59:15)

Offline

#11 2015-04-23 17:40:45

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

There already seems to be a bug report for the error: https://bugs.archlinux.org/task/44693

Offline

#12 2015-04-23 18:28:54

guzzard
Member
Registered: 2014-07-09
Posts: 3

Re: Coredumps on most applications

Thanks MK13!

Installing the intel-ucode package (https://www.archlinux.org/packages/extr … tel-ucode/),
and after that updating grub as instructed in the wiki (https://wiki.archlinux.org/index.php/Microcode) solved the issue for me as well.

No issues with nvidia 349.16-1 installed.

Offline

#13 2015-04-24 01:29:23

wildtron
Member
Registered: 2015-04-23
Posts: 4

Re: Coredumps on most applications

Installing the microcode fixed my problems. What actually happened here MK13? It only appeared just this week and I had no problems like that before. Was it because of an update? big_smile

Offline

#14 2015-04-24 06:42:43

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

This is just guess, but I think that the update of nvidia-libgl to 349.16-1 introduced some kind of feature that (perhaps not even explicit) uses HLE instructions. Installing intel-ucode disables this instructions so glibc (specifically libpthread) don't use this instructions anymore.

Links for further reading:
Wikipedia on GPF
Wikipedia on HLE
Techreport on HLE error
Intel on TSX

Offline

#15 2015-04-24 09:21:19

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

igork wrote:

fabertawe, you need also modify boot options to load microcode at startup - https://wiki.archlinux.org/index.php/Microcode
in my case i'm using grub and modified grub.conf...

Thanks for that, I hadn't realised. This is a new Intel CPU, I've always had AMD in the past.

Anyway... I'm getting a kernel panic with stock kernel and linux-ck -

VFS: Unable to mount root fs on unknown-block(0,0)

I'm using BURG (/boot/burg/burg.cfg) and this may be the problem. It appears outdated but I left it as my system was still booting fine after recently updating the motherboard, and I'm still somewhat confused by EFI. Any suggestions as to which bootloader I should be using?

Thanks all.


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#16 2015-04-24 09:38:22

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

fabertawe wrote:

Anyway... I'm getting a kernel panic with stock kernel and linux-ck -

VFS: Unable to mount root fs on unknown-block(0,0)

I'm using BURG (/boot/burg/burg.cfg) and this may be the problem. It appears outdated but I left it as my system was still booting fine after recently updating the motherboard, and I'm still somewhat confused by EFI. Any suggestions as to which bootloader I should be using?

Thanks all.

Not sure about the kernel panic, but it looks like something went wrong at the configuration of BURG. Could you post your cfg file (new thread, link that here, we don't want to hijack tongue)?

As for an alternative bootloader I recommend gummiboot for UEFI or grub or syslinux for MBR. (totally subjective though)

Last edited by MK13 (2015-04-24 09:44:19)

Offline

#17 2015-04-24 10:38:36

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

@MK13 - unlike grub I needed to place the "/boot/intel-ucode.img" initrd entry on a separate line, so now I can boot again.

There was no microcode update (from dmesg) which probably means my CPU didn't need one. But I'm still getting the kernel traps so I've reverted the nvidia packages again.

Unfortunately... I'm now getting

Apr 24 11:27:00 arch64 kernel: grub-symdb[12719]: segfault at 143a2d0 ip 00007fd11ecfa20a sp 00007fffe7b6b498 error 4 in libc-2.21.so[7fd11ec7a000+199000]
Apr 24 11:27:00 arch64 kernel: grub-symdb[12721]: segfault at 25bda90 ip 00007fc409f3320a sp 00007ffe86c08cd8 error 4 in libc-2.21.so[7fc409eb3000+199000]
Apr 24 11:27:00 arch64 systemd-coredump[12724]: Process 12721 (grub-symdb) of user 1000 dumped core.
Apr 24 11:27:00 arch64 systemd-coredump[12723]: Process 12719 (grub-symdb) of user 1000 dumped core.

Could this mean a continuing problem with my bootloader and the microcode update actually not getting applied? I will have to try a new bootloader and report back roll


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#18 2015-04-24 10:54:31

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

Even in case that your CPU doesn't received an update there should be messages in dmesg, see Verifying that microcode got updated on boot

Offline

#19 2015-04-24 14:18:11

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

@MK13 - the messages you get from dmesg, even without an actual update, I was getting anyway - without intel-ucode. I've just got nothing extra, as in the microcode getting updated.


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#20 2015-04-24 14:47:21

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

fabertawe wrote:

@MK13 - the messages you get from dmesg, even without an actual update, I was getting anyway - without intel-ucode. I've just got nothing extra, as in the microcode getting updated.

Ok, my bad. I'm sitting in front of a windows machine right now, so unfortunately there was no way of testing it...

What CPU are you running on? Is it the i7 4790K like in you signature? Have a look at Intel Donwloadcenter for a complete list of CPU models that are affected by this microcode update. It shows the i7 series as well. Are you sure that the ucode image gets loaded during startup?

E: As your CPU is from the Haswell series it should definitely receive an update.

Last edited by MK13 (2015-04-24 14:49:24)

Offline

#21 2015-04-24 15:17:17

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

Yes, the i7 4790K. If anyone else who has one is reading this, perhaps we could compare output and see if I'm actually up to date or not...

$ dmesg | grep microcode
[    0.265661] microcode: CPU0 sig=0x306c3, pf=0x2, revision=0x19
[    0.265663] microcode: CPU1 sig=0x306c3, pf=0x2, revision=0x19
[    0.265667] microcode: CPU2 sig=0x306c3, pf=0x2, revision=0x19
[    0.265671] microcode: CPU3 sig=0x306c3, pf=0x2, revision=0x19
[    0.265675] microcode: CPU4 sig=0x306c3, pf=0x2, revision=0x19
[    0.265679] microcode: CPU5 sig=0x306c3, pf=0x2, revision=0x19
[    0.265683] microcode: CPU6 sig=0x306c3, pf=0x2, revision=0x19
[    0.265687] microcode: CPU7 sig=0x306c3, pf=0x2, revision=0x19
[    0.265703] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba

Last edited by fabertawe (2015-04-24 15:17:30)


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#22 2015-04-24 15:23:35

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

You can check it yourself:

cat /sys/devices/system/cpu/cpu*/microcode/version

Compare with ucode initrd and without

Last edited by MK13 (2015-04-24 15:32:47)

Offline

#23 2015-04-24 15:46:34

fabertawe
Member
From: Lloegr
Registered: 2009-11-24
Posts: 275

Re: Coredumps on most applications

I already posted it above tongue

If someone with the same CPU could tell me what theirs is then I'll know if the microcode needs updating and if it does, then I know the ucode initrd is failing somehow.


i7 4790K 4.4GHz, ASRock Z97 Extreme6, 16GB 2400MHz, GTX 970, Arch x86_64

Offline

#24 2015-04-24 15:47:59

Scimmia
Fellow
Registered: 2012-09-01
Posts: 8,584

Re: Coredumps on most applications

Offline

#25 2015-04-24 15:51:38

MK13
Member
From: Germany
Registered: 2014-04-12
Posts: 78

Re: Coredumps on most applications

MK13 wrote:

Compare with ucode initrd and without

If they differ you'll know that the update took place, if not your initrd is not working as you may think it does. Since I'm at home now I checked it with my machine.

Offline

Board footer

Powered by FluxBB