Kernel 3.6.2 Power Regressions (Sandy Bridge)

KaiSforza · 2012-12-13 20:08:31

foxxx wrote:

Hey guys,
just tried out arch linux for the first time and i'm quite happy so far. Quite soon I got really annoyed by that power regression bug in the current core repo kernel so I tried different things with laptop-mode-tools, tlp, cpupower, etc.
At one point I got the cpu scaling to work and got the idle power consumption down to ~9W. But after resuming from suspend that wasn't working anymore.
Just some moments away from compiling my own kernel I discovered this thread and your linux-mainline repo KaiSforza. Right now I want to thank you for your great work. Your kernel is running really smooth, even if I can't get the power consumption to below 11W, but it's way better than the core repo one.
If I got some spare time this weekend I'll try using the mentioned patch myself and compile some different kernel-versions. I'll let you know about my results

Thank you for trying out my repo, I hope you can get your power usage down again.

I also see that you have a Lenovo T420. Before I think of releasing a 3.7 kernel, would you mind testing the one in [testing] for me? I've run across a bug where plugging in the power supply causes the computer to freeze up. I'd like to know if it's something I did to my computer or if it's the fault of my hardware.

foxxx0 · 2012-12-13 20:18:59

I'd appreciate it if I could support you in developing/testing new kernel releases.

As you already spotted I own a T420 too, so I'm available for your testing versions Just eMail me with download details and information/logs you want from me while testing.

PS: Are you guys hanging out somewhere in IRC ?

Last edited by foxxx0 (2012-12-13 20:38:49)

mabl · 2012-12-15 12:08:23

KaiSforza wrote:

I also see that you have a Lenovo T420. Before I think of releasing a 3.7 kernel, would you mind testing the one in [testing] for me? I've run across a bug where plugging in the power supply causes the computer to freeze up. I'd like to know if it's something I did to my computer or if it's the fault of my hardware.

I see the same bug on my Lenovo X220 with the 3.7.0-1-ck kernel. I could accept the power consumption after standby bug - but this is just too much. Why aren't those bugs discovered in the RCs? I know I did not test it - but I'd expect Lenovos to be used by quite a lot of kernel devs...

EDIT:
Ok there are bugreports:
https://bugzilla.kernel.org/show_bug.cgi?id=51661
https://bugzilla.kernel.org/show_bug.cgi?id=51721

Last edited by mabl (2012-12-15 13:00:43)

KaiSforza · 2012-12-15 16:12:00

mabl wrote:

KaiSforza wrote:
I also see that you have a Lenovo T420. Before I think of releasing a 3.7 kernel, would you mind testing the one in [testing] for me? I've run across a bug where plugging in the power supply causes the computer to freeze up. I'd like to know if it's something I did to my computer or if it's the fault of my hardware.
I see the same bug on my Lenovo X220 with the 3.7.0-1-ck kernel. I could accept the power consumption after standby bug - but this is just too much. Why aren't those bugs discovered in the RCs? I know I did not test it - but I'd expect Lenovos to be used by quite a lot of kernel devs...
EDIT:
Ok there are bugreports:
https://bugzilla.kernel.org/show_bug.cgi?id=51661
https://bugzilla.kernel.org/show_bug.cgi?id=51721

I actually took the time to find someone with similar hardware (foxxx with his T420) to test it, and he couldn't reproduce the results. If you are, then please by all means contribute to the bug report that I submitted.

Note that I did a kernel bisect, and it was introduced right after the rc kernel line was finished. Here are the bisects that I did, and here is the log: (Note, the bisects are only going to work with Intel chips.)

git bisect start
# bad: [29594404d7fe73cd80eaa4ee8c43dcc53970c60e] Linux 3.7
git bisect bad 29594404d7fe73cd80eaa4ee8c43dcc53970c60e
# good: [b69f0859dc8e633c5d8c06845811588fe17e68b3] Linux 3.7-rc8
git bisect good b69f0859dc8e633c5d8c06845811588fe17e68b3
# bad: [04c5decdc0aecde43bf44860484f26ee0691335f] Merge branch 'upstream' of git://git.linux-mips.org/pub/scm/ralf/upstream-linus
git bisect bad 04c5decdc0aecde43bf44860484f26ee0691335f
# good: [df2fc246c8ee8b6067af1fa55d3bc23107457f61] Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux
git bisect good df2fc246c8ee8b6067af1fa55d3bc23107457f61
# bad: [54d1ae492f724f103a07f69c8e429a35d43b5c52] Merge branch 'fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/rusty/linux
git bisect bad 54d1ae492f724f103a07f69c8e429a35d43b5c52
# bad: [cfd1f032f98e5ab3a04f23a0adbd53ff8744827d] Merge branch 'core-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect bad cfd1f032f98e5ab3a04f23a0adbd53ff8744827d
# bad: [8d4516904b39507458bee8115793528e12b1d8dd] watchdog: Fix CPU hotplug regression
git bisect bad 8d4516904b39507458bee8115793528e12b1d8dd

8d4516 was the first commit that caused my computer to do this, and it also the first commit added after rc8.

mabl · 2012-12-15 16:40:37

Sure, I know how git bisect works - but is there a nice guide on how to do it on an arch system without polluting my system with files outside of package management? I'm quite new to arch.

Last edited by mabl (2012-12-15 16:40:56)

KaiSforza · 2012-12-15 17:51:21

Check the pkgbuild I used for bisecting.

Note, it uses pacman-git, but it installs everything nice and separate without files being scattered. Or you can test out the 5 kernels I have already built. (They can be installed next to a kernel not called 'linux-mainline-bisect')

Learning to use makepkg and PKGBUILDs will help in the future of using Arch.

Last edited by KaiSforza (2012-12-15 17:54:39)

el mariachi · 2012-12-16 12:22:26

I found this topic while searching for the reason why my laptop was running hotter with 3.6.10

It's a Dell E5400 with intel 4500MHD graphics (G45).
There was a time in which it ran at 35-40ºC idle, now it's 50-60ºC even with KaiSforza's mainline kernel.
I guess this applies to more hardware than just sandy bridge.

donniezazen · 2012-12-16 17:31:45

@el mariachi It sounds like a different bug. The bug we are discussing here seems to be triggered by suspend and temperature lies in tune to 90C or more. You might want to report another bug upstream or comment of already filed bug.

KaiSforza · 2012-12-16 18:03:53

el mariachi wrote:

I found this topic while searching for the reason why my laptop was running hotter with 3.6.10
It's a Dell E5400 with intel 4500MHD graphics (G45).
There was a time in which it ran at 35-40ºC idle, now it's 50-60ºC even with KaiSforza's mainline kernel.
I guess this applies to more hardware than just sandy bridge.

I'd be willing to say this is similar if you can give us some powertop output. Just

sudo powertop --html

and upload it somewhere that we can access.

el mariachi · 2012-12-17 08:52:55

http://ompldr.org/vZ3E2dQ

☁  ~  sensors
acpitz-virtual-0
Adapter: Virtual device
temp1:        +60.5°C  (crit = +102.0°C)

coretemp-isa-0000
Adapter: ISA adapter
Core 0:       +60.0°C  (high = +100.0°C, crit = +100.0°C)
Core 1:       +61.0°C  (high = +100.0°C, crit = +100.0°C)

Uptime: 3min

KaiSforza · 2012-12-17 18:19:21

I'm now applying two patches to my mainline kernels (3.8-rc1 should be tagged today or tomorrow). You can check my github to see what the patches do.

el mariachi · 2012-12-17 18:34:08

So you confirm I'm also affected KaiSforza?

KaiSforza · 2012-12-17 18:42:32

el mariachi wrote:

So you confirm I'm also affected KaiSforza?

Honestly, I have no idea. If the 3.7-rc8 kernel in my repo doesn't seem to do anything then you may be affected by a different bug.

flockyrocky · 2012-12-18 10:49:14

Just tried your mainline kernel on my T420s. hmm, it's hard to say if the issue is fixed. Actually I'm a bit confused by current situation.
Let's say when we were in the perfect world, my laptop used to spend around 10w when there's no stress on it. the temperature is around 40C.
While when the bug happens, I never see the power consumption less than 25w. temperature is about 60C.
Now with mainline kernel from your repository, it's 18w/50C.
It's more likely the bug is partly fixed.
I'll wait and try the 3.8-rc1 when it's ready. From the bug tracking I think the it's officially fixed in this version. Worth trying.
Anyway, thanks for the efforts.

hendry · 2012-12-18 15:38:28

I'm having crazy heat problems too with 3.6.10 from boot.
http://r2d2.webconverger.org/2012-12-18/

KaiSforza · 2012-12-18 16:21:48

hendry wrote:

I'm having crazy heat problems too with 3.6.10 from boot.
http://r2d2.webconverger.org/2012-12-18/

Now that looks exactly like the issues we're talking about. Would you mind trying out the mainline kernel, hendry? There's a repository for them in my signature.

hendry · 2012-12-19 06:43:31

KaiSforza wrote:

Now that looks exactly like the issues we're talking about. Would you mind trying out the mainline kernel, hendry? There's a repository for them in my signature.

I've been testing your kernels from time to time, but I had a bad heat experience and went back to LTS.

Now using 3.7.0-3-mainline, it seems fine, but I still have my confidence rapped.
http://r2d2.webconverger.org/2012-12-19/powertop.html

Hope the solution is in sight. It's shocking how 3.6.10 actually managed to get WORSE.

KaiSforza · 2012-12-22 02:34:14

I'm in the process of building 3.8-rc1. I'll test it out myself with and without patches to see if they fixed all the i915 stuff. Will also upload an x86_64 kernel before I head off to sleep.

KaiSforza · 2012-12-22 03:15:02

Just built and booted into an un-patched kernel, suspended a few times and restarted once and it's still running smooth and cold. I don't think there's a need to patch for this bug anymore. Seems the i915 guys got their fixes pushed to the mainline. I also won't be uploading since this seems to be resolved with the 3.8 series. miffe's repository should work for you now.

EDIT: Yeah...so I did build a 3.8-rc1 for you guys.

Last edited by KaiSforza (2012-12-22 20:20:06)

hendry · 2012-12-22 04:08:36

Be good if one could reference the bug where this was fixed. Most frustrating thing about this "Sandy bridge saga" is having no idea how to track this issue.

KaiSforza · 2012-12-22 04:29:10

hendry wrote:

Be good if one could reference the bug where this was fixed. Most frustrating thing about this "Sandy bridge saga" is having no idea how to track this issue.

It's on the first page along with a very incomplete list of affected hardware.

(also here for convenience: https://bugzilla.kernel.org/show_bug.cgi?id=48721 )

Note: I'm not saying that the bug is up to date, but that was the bug report I sent way back when this started. I could probably go through a bunch of commits and look for the i915 changes...

Last edited by KaiSforza (2012-12-22 04:33:09)

KaiSforza · 2012-12-22 18:12:04

So it's been a little over 12 hours of using an un-patched 3.8-rc1 (minus the patch included in the -ARCH kernel), and so far none of the bugs from 3.6/3.7 in terms of power usage. I'm running at 40°C rignt now after quite a few reboots and suspend-resume cycles. I'd not mark this solved till it hits the repos, but there is a solution.

I hope this fixes things for the rest of you!

Starting soon I'll be building un-signed kernels on my vps for those of you that need more generic kernels.

Rolinh · 2012-12-22 18:37:31

Well, I'm glad to hear that.

I'll try out 3.8-rc1 then and tell you if the bug is fixed for me too (but as I own the same laptop, it should be fixed too ^^).

KaiSforza · 2012-12-22 20:13:15

Binaries released (unsigned, sorry) at http://sakuscans.com/pacmanpkg/x86_64/

Rolinh · 2012-12-22 21:03:56

Question: why .tar.gz instead of .tar.xz ?

PS: 3.7-rc8 is still on your repo but... pretty useless now

Arch Linux

#226 2012-12-13 20:08:31

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#227 2012-12-13 20:18:59

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#228 2012-12-15 12:08:23

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#229 2012-12-15 16:12:00

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#230 2012-12-15 16:40:37

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#231 2012-12-15 17:51:21

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#232 2012-12-16 12:22:26

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#233 2012-12-16 17:31:45

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#234 2012-12-16 18:03:53

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#235 2012-12-17 08:52:55

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#236 2012-12-17 18:19:21

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#237 2012-12-17 18:34:08

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#238 2012-12-17 18:42:32

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#239 2012-12-18 10:49:14

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#240 2012-12-18 15:38:28

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#241 2012-12-18 16:21:48

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#242 2012-12-19 06:43:31

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#243 2012-12-22 02:34:14

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#244 2012-12-22 03:15:02

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#245 2012-12-22 04:08:36

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#246 2012-12-22 04:29:10

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#247 2012-12-22 18:12:04

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#248 2012-12-22 18:37:31

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#249 2012-12-22 20:13:15

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

#250 2012-12-22 21:03:56

Re: Kernel 3.6.2 Power Regressions (Sandy Bridge)

Board footer