High load averages and interrupts with 2.6.36/2.6.37

Ramses de Norre · 2011-01-26 19:36:55

Holy crap, powertop-git reports about 240 wake-ups/sec here!
I tried stopping all daemons to no avail... The top causes are "Timer" and "KWork".

bernarcher · 2011-01-26 19:55:12

This could be caused by the kernel highres timer. Does it help to switch it off at boot time (appending "nohz=off highres=off" to the grub boot command)?

karol · 2011-01-26 20:03:23

Ramses de Norre wrote:

The top causes are "Timer" and "KWork".

Those are categories, what's the description saying?

Ramses de Norre · 2011-01-26 21:43:21

@karol: the description field is empty for all instances... I should just run "powertop" as root, right? (there is also no manpage nor a -h or --help switch, argh...)

@bernarcher: I take that's a response to my post? I'll try it tomorrow and post the results!

karol · 2011-01-27 14:42:27

Ramses de Norre wrote:

@karol: the description field is empty for all instances... I should just run "powertop" as root, right?

Are you running powertop-git?

                Usage        Events/s  Category       Description
616,2 µs/s   42,3      Interrupt      PS/2 Touchpad / Keyboard / Mouse
23,5 ms/s    24,8      Process        firefox
9,2 ms/s     17,4      Process        /usr/bin/X -nolisten tcp :0 -auth /tmp/serverauth.S6t9LIIirO
2,8 ms/s     5,6       Process        urxvtd -q -o -f
476,6 µs/s   5,2       Timer          tick_sched_timer
10,7 ms/s    0,6       Process        htop
148,0 µs/s   4,4       Timer          hrtimer_wakeup
6,8 µs/s     0,8       kWork          cfq_kick_queue
7,7 µs/s     0,6       kWork          flush_to_ldisc
2,3 µs/s     0,4       kWork          intel_idle_update
44,1 µs/s    0,4       Process        /usr/sbin/gpm -m /dev/psaux -t ps2
83,7 µs/s    0,3       Interrupt      [4] block(softirq)
0,9 ms/s     0,0       Process        powertop

Ramses de Norre · 2011-01-27 19:13:36

karol wrote:

Ramses de Norre wrote:

@karol: the description field is empty for all instances... I should just run "powertop" as root, right?

Are you running powertop-git?

                Usage        Events/s  Category       Description
616,2 µs/s   42,3      Interrupt      PS/2 Touchpad / Keyboard / Mouse
23,5 ms/s    24,8      Process        firefox
9,2 ms/s     17,4      Process        /usr/bin/X -nolisten tcp :0 -auth /tmp/serverauth.S6t9LIIirO
2,8 ms/s     5,6       Process        urxvtd -q -o -f
476,6 µs/s   5,2       Timer          tick_sched_timer
10,7 ms/s    0,6       Process        htop
148,0 µs/s   4,4       Timer          hrtimer_wakeup
6,8 µs/s     0,8       kWork          cfq_kick_queue
7,7 µs/s     0,6       kWork          flush_to_ldisc
2,3 µs/s     0,4       kWork          intel_idle_update
44,1 µs/s    0,4       Process        /usr/sbin/gpm -m /dev/psaux -t ps2
83,7 µs/s    0,3       Interrupt      [4] block(softirq)
0,9 ms/s     0,0       Process        powertop

Yes, the package from AUR:

Summary: 256.2 wakeups/second,  36.9 GPU ops/second and 0.0 VFS ops/sec

                Usage       Events/s    Category       Description
            30268645 ms/s    0.0        Timer
            30264416 ms/s    0.0        Timer
            30264072 ms/s    1.0        Timer
            30263990 ms/s    0.0        Timer
            30263899 ms/s    1.0        Timer
            30263894 ms/s    0.0        Timer
            30263745 ms/s    0.0        Timer
            30263737 ms/s    1.0        Timer
            30263708 ms/s    1.0        Timer
            30263698 ms/s    1.0        Timer
            30263632 ms/s    0.0        Timer
            30263618 ms/s    1.0        Timer
            30263538 ms/s    1.0        Timer
            30263460 ms/s    1.0        Timer
            30263457 ms/s    1.0        Timer
            30263437 ms/s    0.0        Timer
            30263377 ms/s    1.0        Timer
            30263365 ms/s    0.0        Timer
            30263343 ms/s    1.0        Timer
            30263297 ms/s    1.0        Timer
            30263258 ms/s    0.0        Timer
            30263242 ms/s    1.0        Timer
            30263223 ms/s    1.0        Timer
            30263217 ms/s    1.0        Timer
            30263168 ms/s    0.0        Timer

Ramses de Norre · 2011-01-27 19:30:43

Hmm, this looks quite bogus, booting with bernarcher's kernel options makes powertop-git say I have 0.0 wakeups/sec, I suppose that can't be correct?

l'pc · 2011-01-28 22:12:47

Same as Ramses de Norre; with nohz=off and highres=off I have 0 wakeups per second, with kernel 2.6.36.3-2 on x86_64.

Though it seems to be better than with the precedent version (-1) of the kernel, it still isn't good; I'm at 80-100 when idling. (without specials kernel arguments)

kokoko3k · 2011-01-30 09:19:32

I think 0 wakeups per second are just a powertop-jit bug, does powertop reports it too?

skunktrader · 2011-01-30 23:22:32

Seems to be finally fixed in 2.3.37

$ uname -a
Linux kahuna 2.6.37-ARCH #1 SMP PREEMPT Sat Jan 29 19:40:04 UTC 2011 i686 Intel(R) Xeon(TM) CPU 2.40GHz GenuineIntel GNU/Linux
$ uptime
 18:20:03 up 39 min,  2 users,  load average: 0.00, 0.02, 0.05

Edit: remove formatting

Last edited by skunktrader (2011-01-30 23:23:14)

Demon · 2011-01-31 23:57:57

skunktrader wrote:

Seems to be finally fixed in 2.3.37

2.3.37?!

Telkkar · 2011-02-01 13:52:54

Demon wrote:

skunktrader wrote:
Seems to be finally fixed in 2.3.37
2.3.37?!

I WISH it was fixed back then. I'll give 2.6.37-5 a go from [core]. I think the last time I used .37 was from [testing] and it was -1, soo, I hope what skunktrader said about it being fixed is true.

EDIT: It appears that skunktrader is right, and the issue (at least the one I was having) is fixed in the latest 2.6.37 kernel from [core].

If no one has any other issues, I'm going to put [SOLVED] in the title. I'll give it some time.

EDIT2: Yeah, no. I'm still having problems. Not as high wakeups as everyone else is having, but much more than I should be having for an idle system at the console.

Last edited by Telkkar (2011-02-28 13:33:14)

skunktrader · 2011-02-01 15:08:50

Sorry. Obviously I meant 2.6.37

fowler · 2011-02-01 15:39:33

This issue is not solved as I am still getting 250 wakeups per second with 2.6.37 according to powertop-git and my desktop performance is degraded.

OdinEidolon · 2011-02-01 16:07:39

fowler wrote:

This issue is not solved as I am still getting 250 wakeups per second with 2.6.37 according to powertop-git and my desktop performance is degraded.

same here. And high CPU averages too.

Telkkar · 2011-02-02 03:09:18

All right, guess it was only for me.

I'll leave it unmarked and see what else I can find around the forums.

Ramses de Norre · 2011-02-02 12:30:36

I still got the ~250 wakeups/sec on idle... The load averages are fine though. I'll keep watching this thread and I keep on googling this issue every now and then, in case someone comes up with a fix.

fowler · 2011-02-02 14:26:29

I'm getting around 500 wakeups/second here.

OdinEidolon · 2011-02-02 15:16:05

somehow load averages went down. This is while only downloading with ktorrent (which still uses some cpu according to htop).

load average: 0.01, 0.10, 0.11

Wakeups are still ~270/second. I had ~150 with previous kernel.

Ultraman · 2011-02-16 17:11:23

Seems there are multiple threads about this...

I am also still experiencing these problems, but _only_ when running on battery power.
With my laptop plugged in, my load drops below 1.00, kworker and [acpi] interrupts calm down in powertop.
And when I unplug they start going at it again. This seriously hurts battery life, dropped to 3.5 hours from 6-ish hours.
I experience this with both 2.6.37-ARCH and 2.6.36-ck kernels.

This is an Intel Core2 U7300, 4GB DDR3, Intel GS/GM45 chipset. Sony Vaio Y11S1E laptop.
Also read somewhere this had to do with Intel and DRM polling. Unloading the 'intel_ips' module should remedy it, but this module is not loaded on my system.

Hope this issue gets fixed soon...

OdinEidolon · 2011-02-16 18:09:03

Ultraman wrote:

Seems there are multiple threads about this...
I am also still experiencing these problems, but _only_ when running on battery power.
With my laptop plugged in, my load drops below 1.00, kworker and [acpi] interrupts calm down in powertop.
And when I unplug they start going at it again. This seriously hurts battery life, dropped to 3.5 hours from 6-ish hours.
I experience this with both 2.6.37-ARCH and 2.6.36-ck kernels.
This is an Intel Core2 U7300, 4GB DDR3, Intel GS/GM45 chipset. Sony Vaio Y11S1E laptop.
Also read somewhere this had to do with Intel and DRM polling. Unloading the 'intel_ips' module should remedy it, but this module is not loaded on my system.
Hope this issue gets fixed soon...

What governor you use on battery ad on AC? Is the frequency auto-limited when on battery or on AC? (my laptop for example won't go over 1.6 GHz on battery, while it should be 2.53GHz. This can be fixed telling the kernel to ignore ppc - see wiki)

Ultraman · 2011-02-16 22:10:02

Governor on AC and battery is the same: 'conservative'. I found that one to work best for my CPU and needs.
So it's on 800MHz when it can and 1.3GHz when it needs to.
This always worked fine before. Do you expect the governor to be an issue?

When on battery, laptop-mode tools kick in to do some nice things. Increasing writeback time, hda and usb suspend options, hard drive spindown, that kind of stuff.

[Edit]
Strange this is that this behaviour does not always happen.
I pulled the AC just now, and kworker or [acpi] interrupts are not going through the roof. kworker does still cause some more wakeups than I would like, but this is acceptable now. No high load atm.
This is how it should behave more or less. Wasn't the case this afternoom for me...
I'll see if booting from battery or with AC plugged in makes a difference.

Last edited by Ultraman (2011-02-16 22:15:13)

OdinEidolon · 2011-02-17 11:25:27

Ultraman wrote:

Governor on AC and battery is the same: 'conservative'. I found that one to work best for my CPU and needs.
So it's on 800MHz when it can and 1.3GHz when it needs to.
This always worked fine before. Do you expect the governor to be an issue?

It could be but evidently it is not.

Ultraman wrote:

Strange this is that this behaviour does not always happen.
I pulled the AC just now, and kworker or [acpi] interrupts are not going through the roof. kworker does still cause some more wakeups than I would like, but this is acceptable now. No high load atm.
This is how it should behave more or less. Wasn't the case this afternoom for me...
I'll see if booting from battery or with AC plugged in makes a difference.

I really do not know, Kworker is funny here too, but only rarely. Also, I have kded going crazy and using 100% of the CPU about once a week.

kokoko3k · 2011-02-17 18:50:10

Users should kill processes one by one and see if problem persists.
That's the only way to be 'almost sure' that the problem lies in some kernel code.

Edit: poor my english...

Last edited by kokoko3k (2011-02-17 18:50:47)

Ultraman · 2011-02-20 23:21:44

I can confirm this happens when booting from battery power. When booting with AC plugged the problem does not appear or it does not appear in such extremes.
Plugging AC in after booting from battery power does alleviate the issue. Interesting...

So I experience this issue only when running on battery power after booting

Updating to kernel 2.6.37.1 now. I haven't checked out the changelog, so I do not know if this will fix it or not. Hoping for some magic.
If it fixes it, i'll report back. Otherwise you can assume the problem persists.

[Edit]
Booting off AC, all fine, pulling the AC and running on battery then, it stays OK.
Going to try booting from the battery tomorrow.

Last edited by Ultraman (2011-02-21 19:49:51)

Arch Linux

#26 2011-01-26 19:36:55

Re: High load averages and interrupts with 2.6.36/2.6.37

#27 2011-01-26 19:55:12

Re: High load averages and interrupts with 2.6.36/2.6.37

#28 2011-01-26 20:03:23

Re: High load averages and interrupts with 2.6.36/2.6.37

#29 2011-01-26 21:43:21

Re: High load averages and interrupts with 2.6.36/2.6.37

#30 2011-01-27 14:42:27

Re: High load averages and interrupts with 2.6.36/2.6.37

#31 2011-01-27 19:13:36

Re: High load averages and interrupts with 2.6.36/2.6.37

#32 2011-01-27 19:30:43

Re: High load averages and interrupts with 2.6.36/2.6.37

#33 2011-01-28 22:12:47

Re: High load averages and interrupts with 2.6.36/2.6.37

#34 2011-01-30 09:19:32

Re: High load averages and interrupts with 2.6.36/2.6.37

#35 2011-01-30 23:22:32

Re: High load averages and interrupts with 2.6.36/2.6.37

#36 2011-01-31 23:57:57

Re: High load averages and interrupts with 2.6.36/2.6.37

#37 2011-02-01 13:52:54

Re: High load averages and interrupts with 2.6.36/2.6.37

#38 2011-02-01 15:08:50

Re: High load averages and interrupts with 2.6.36/2.6.37

#39 2011-02-01 15:39:33

Re: High load averages and interrupts with 2.6.36/2.6.37

#40 2011-02-01 16:07:39

Re: High load averages and interrupts with 2.6.36/2.6.37

#41 2011-02-02 03:09:18

Re: High load averages and interrupts with 2.6.36/2.6.37

#42 2011-02-02 12:30:36

Re: High load averages and interrupts with 2.6.36/2.6.37

#43 2011-02-02 14:26:29

Re: High load averages and interrupts with 2.6.36/2.6.37

#44 2011-02-02 15:16:05

Re: High load averages and interrupts with 2.6.36/2.6.37

#45 2011-02-16 17:11:23

Re: High load averages and interrupts with 2.6.36/2.6.37

#46 2011-02-16 18:09:03

Re: High load averages and interrupts with 2.6.36/2.6.37

#47 2011-02-16 22:10:02

Re: High load averages and interrupts with 2.6.36/2.6.37

#48 2011-02-17 11:25:27

Re: High load averages and interrupts with 2.6.36/2.6.37

#49 2011-02-17 18:50:10

Re: High load averages and interrupts with 2.6.36/2.6.37

#50 2011-02-20 23:21:44

Re: High load averages and interrupts with 2.6.36/2.6.37

Board footer