System freezes

mich41 · 2015-09-01 05:41:53

I'm not sure if you doing that can help anybody besides you

It's just a temporary "solution" to get Arch running reliably without disabling power management.

Last edited by mich41 (2015-09-01 05:42:34)

GourdCaptain · 2015-09-01 05:46:53

Fair enough. I'd rather have a fully functional Fedora install than a kludged together Frankenstein of an Arch Install I have to worry about accidentally breaking. I just figure it might be worth testing to isolate causes for sure.

seqizz · 2015-09-01 06:18:13

I have the same laptop with kris7t , which has 5700hq cpu. Sometimes I got random crashes, which system just stops responding and waits 10 seconds before blinking caps lock key for kernel panic. I never saw any records in logs/dmesg. My main suspicion nowadays is integrated graphics chip. Next step is setting up a kernel for crash dumps. I need my laptop to be reliable on high load and I too prefer to use Arch if it's possible. If I can find something, I'll share.

nashamri · 2015-09-01 17:55:31

seqizz wrote:

I have the same laptop with kris7t , which has 5700hq cpu. Sometimes I got random crashes, which system just stops responding and waits 10 seconds before blinking caps lock key for kernel panic. I never saw any records in logs/dmesg. My main suspicion nowadays is integrated graphics chip. Next step is setting up a kernel for crash dumps. I need my laptop to be reliable on high load and I too prefer to use Arch if it's possible. If I can find something, I'll share.

The issue is most certainly the intel broadwell CPU. Check the full post and the phoronix articles mentioned previously.

V1del · 2015-09-01 18:45:05

Have any of you done the Intel microcode update?

GourdCaptain · 2015-09-01 18:46:29

V1del wrote:

Have any of you done the Intel microcode update?

I had, before I gave up and switched to Fedora on the machine. It didn't help.

seqizz · 2015-09-01 18:46:55

nashamri wrote:

The issue is most certainly the intel broadwell CPU. Check the full post and the phoronix articles mentioned previously.

I guess mine was related with SNA, because after switching to UXA I'm mostly stable now. Uptime is 22 hours (which passed a load sensitive workday with 4-5 times sleep-wakeup and virtualbox). I don't know if this is an exception or something. I'll still try different stress tests & compiling with multicore to catch a panic.

What is your cpu governor by the way? I remember some crash with ondemand, learned it's not supported on speedstep anymore.

GourdCaptain · 2015-09-01 18:49:44

seqizz wrote:

nashamri wrote:
The issue is most certainly the intel broadwell CPU. Check the full post and the phoronix articles mentioned previously.
I guess mine was related with SNA, because after switching to UXA I'm mostly stable now. Uptime is 22 hours (which passed a load sensitive workday with 4-5 times sleep-wakeup and virtualbox). I don't know if this is an exception or something. I'll still try different stress tests & compiling with multicore to catch a panic.
What is your cpu governor by the way? I remember some crash with ondemand, learned it's not supported on speedstep anymore.

I hadn't changed it, so it was the default Intel governor for recent chips, which is where the problem code is. And it's pretty much the only game in town for CPU power management on these chips, hence the issues.

seqizz · 2015-09-01 18:53:45

GourdCaptain wrote:

I hadn't changed it, so it was the default Intel governor for recent chips, which is where the problem code is. And it's pretty much the only game in town for CPU power management on these chips, hence the issues.

I suggest you to check:

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

I'm using TLP for power management on laptop, which skips between powersave and performance governors by AC/BAT status.

GourdCaptain · 2015-09-01 18:59:19

seqizz wrote:

GourdCaptain wrote:
I hadn't changed it, so it was the default Intel governor for recent chips, which is where the problem code is. And it's pretty much the only game in town for CPU power management on these chips, hence the issues.
I suggest you to check:
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
I'm using TLP for power management on laptop, which skips between powersave and performance governors by AC/BAT status.

Huh, the different governor might be why you're not having the same issues, but the intel_pstate driver basically disregards most governor settings and if I remember correctly, doesn't show the exact results to the OS. Interestingly, Fedora doesn't have that file available to read, so I had to do it with cpupower.

seqizz · 2015-09-01 19:02:57

GourdCaptain wrote:

Huh, the different governor might be why you're not having the same issues, but the intel_pstate driver basically disregards most governor settings and if I remember correctly, doesn't show the exact results to the OS. Interestingly, Fedora doesn't have that file available to read, so I had to do it with cpupower.

Didn't know that. I just see the effects of whatever I set on tlp config (freq drops to 800 if I switch to powersave etc.).

GourdCaptain · 2015-09-01 19:24:26

seqizz wrote:

GourdCaptain wrote:
Huh, the different governor might be why you're not having the same issues, but the intel_pstate driver basically disregards most governor settings and if I remember correctly, doesn't show the exact results to the OS. Interestingly, Fedora doesn't have that file available to read, so I had to do it with cpupower.
Didn't know that. I just see the effects of whatever I set on tlp config (freq drops to 800 if I switch to powersave etc.).

Yeah. But recent intel chips basically do their own power management mostly on the hardware level, irregardless of what the OS tells them to do, and fudge the numbers a bit so the OS thinks everything's going as it wants. (I may not be absolutely accurate, but I read the kernel documentation for this pstate driver a while back.)

GourdCaptain · 2015-09-03 22:13:16

And in utter suprise, I just found out you can cause the crash by running an affected kernel IN VIRTUALBOX on Fedora 22. Woah.

nashamri · 2015-09-10 01:12:31

The issue is still there with linux 4.2-3 from the testing repo.

nashamri · 2015-09-11 23:50:53

Disabling intel's speedstep from the BIOS and disabling fast boot and secure boot and running the 4.2.0 kernel seems to be stable and no issues so far. I Will report here if I encounter anything.

jonessen · 2015-09-12 13:49:18

Hi! I didn't find this forum post until now, I'm really relieved my hardware isn't broken. I also have an i7-5775C, not overclocked and the cooling can't be the problem either, I was able to compile a new kernel with -j8 without a crash, then it crashed on installation. My system was absolutely unusable until I added `cpufreq.governor=performance` to the kernel command line. It still freezes, but much less frequently. I've also tried the kernel in core and in testing, without that changing anything.

And I tried https://wiki.archlinux.org/index.php/In … re-M_chips and switching to UXA, neither of which helped. If I had more free space on my SSD I would immediately install Fedora, but I think my semi-stable arch is the easiest thing to use at the moment. I'll check if my EFI has the option to disable SpeedStep.

mich41 · 2015-09-12 15:36:53

You can try the Fedora/Arch Frankenstein

jonessen · 2015-09-12 23:12:39

Okay, I'm now on Linux 4.1.6-201.fc22.x86_64 ^^

Let's see how this goes. So far I've tried a few things that made it freeze sometimes before, and also I haven't started with the performance cpufreq governor. Seems to be working!

jonessen · 2015-09-13 14:26:37

Didn't help... First time I booted today, the system immediately froze. Then I re-enabled cpufreq.governor=performance for the Fedora kernel as well, and the system froze again when I tried building a package from AUR.

It seems the most reliable way of freezing my system is running gcc. I had it before that the system froze when trying to compile my own C++ program. It worked every single time, so I'm really lucky that that project isn't something I actively work on at the moment.

GourdCaptain · 2015-09-13 16:37:15

jonessen wrote:

Didn't help... First time I booted today, the system immediately froze. Then I re-enabled cpufreq.governor=performance for the Fedora kernel as well, and the system froze again when I tried building a package from AUR.
It seems the most reliable way of freezing my system is running gcc. I had it before that the system froze when trying to compile my own C++ program. It worked every single time, so I'm really lucky that that project isn't something I actively work on at the moment.

Huh. We're kind of at the drawing board again on why Fedora's stable on the same hardware for me - I've run 24+ hour chains of encode jobs without crashes, lengthy compiles, and such.

jonessen · 2015-09-13 16:54:48

FWIW, I just booted into Fedora, chrooted into Arch from there and attempted to build the same AUR package (glade-git) again. During configure, the system froze, just like on Arch.

So... The Fedora Kernel, at least for me, is *not* more stable than the arch kernel, and if Fedora is more stable at all, then because of other low-level things, e.g.... uhhh, a different version of coreutils or something?!
EDIT: gcc might actually be a thing to cause this. I'll try manually going through what makepkg does, on Fedora.

Last edited by jonessen (2015-09-13 17:00:30)

jonessen · 2015-09-13 18:29:10

Well, who would have guessed.. Compiling glade on Fedora with its gcc works like a charm!
As arch very often also crashed during the compilation phase, not only configure (with runs gcc for some tests as well), this particular issue where the system runs fine for a long time and then freezes later when in use seems very likely to be caused by gcc 5.2.

Of course, we still have no clue why Fedora runs well with the powersave cpufreq governor and arch doesn't... The first thing that would come to my mind would be autostart programs / DE. I'm using i3, what are you using as a DE?

Last edited by jonessen (2015-09-13 18:30:24)

mich41 · 2015-09-13 20:20:58

Some time ago, Haswell cores turned out to contain a silicon bug which broke software using the hardware lock elision functionality. I wonder if it's possible that HLE or some other exotic instructions used by Arch and Ubuntu glibc, but not by Fedora glibc, are messing up these CPUs.

As for HLE, it can be disabled at runtime by setting

export GLIBC_PTHREAD_MUTEX=none
export GLIBC_PTHREAD_RWLOCK=none

before launching an affected program. I'm not sure if there is any way to disable it globally besides recompiling glibc without --enable-lock-elision.

Last edited by mich41 (2015-09-13 20:59:46)

jonessen · 2015-09-13 22:24:07

I tried disabling HLE, that didn't help. Also after the freeze I got from trying that out, my system froze right on boot again, with the normal arch kernel + cpufreq.governor=performance. I booted the Fedora kernel afterwards...

Anyway, I think some exotic instrucion triggering this is the best theory we got so far.

Has anybody created a bug on the kernel bugtracker? I gave it a shot about two weeks ago, but it was probably the wrong category and didn't have much useful information, so I closed it myself after nobody had replied a few days later.

GourdCaptain · 2015-09-13 22:58:16

jonessen wrote:

Well, who would have guessed.. Compiling glade on Fedora with its gcc works like a charm!
As arch very often also crashed during the compilation phase, not only configure (with runs gcc for some tests as well), this particular issue where the system runs fine for a long time and then freezes later when in use seems very likely to be caused by gcc 5.2.
Of course, we still have no clue why Fedora runs well with the powersave cpufreq governor and arch doesn't... The first thing that would come to my mind would be autostart programs / DE. I'm using i3, what are you using as a DE?

I'm using XFCE from the Fedora XFCE spin.

jonessen wrote:

Has anybody created a bug on the kernel bugtracker? I gave it a shot about two weeks ago, but it was probably the wrong category and didn't have much useful information, so I closed it myself after nobody had replied a few days later.

I posted on the bug I'm guessing you made earlier in the thread, but I don't think any other bug tracker entries have been created.

Arch Linux

#26 2015-09-01 05:41:53

Re: System freezes

#27 2015-09-01 05:46:53

Re: System freezes

#28 2015-09-01 06:18:13

Re: System freezes

#29 2015-09-01 17:55:31

Re: System freezes

#30 2015-09-01 18:45:05

Re: System freezes

#31 2015-09-01 18:46:29

Re: System freezes

#32 2015-09-01 18:46:55

Re: System freezes

#33 2015-09-01 18:49:44

Re: System freezes

#34 2015-09-01 18:53:45

Re: System freezes

#35 2015-09-01 18:59:19

Re: System freezes

#36 2015-09-01 19:02:57

Re: System freezes

#37 2015-09-01 19:24:26

Re: System freezes

#38 2015-09-03 22:13:16

Re: System freezes

#39 2015-09-10 01:12:31

Re: System freezes

#40 2015-09-11 23:50:53

Re: System freezes

#41 2015-09-12 13:49:18

Re: System freezes

#42 2015-09-12 15:36:53

Re: System freezes

#43 2015-09-12 23:12:39

Re: System freezes

#44 2015-09-13 14:26:37

Re: System freezes

#45 2015-09-13 16:37:15

Re: System freezes

#46 2015-09-13 16:54:48

Re: System freezes

#47 2015-09-13 18:29:10

Re: System freezes

#48 2015-09-13 20:20:58

Re: System freezes

#49 2015-09-13 22:24:07

Re: System freezes

#50 2015-09-13 22:58:16

Re: System freezes

Board footer