CPU is used heavily, but htop/top don't show any process using it.

Qualsiasi · 2010-01-04 14:57:29

I'm using Arch x86_64, updating the full system 2 times a week, so my system is up to date.

Every 30-44 minutes my system becomes quite unresponsive, and the cpu-usage plasmoid (and KDE System Monitor) show CPU being used up to 70-80% or more, while the sistem is quite idle, and this goes on for 20 to 30 seconds, and then the CPU goes back to 0-1%, this is specially annoying when I'm looking some videos, that start to freeze, obviously when I'm watching 1080p videos there's no chance and I have to pause the playback, and go on after the CPU has gone back to 1%.

I'm using a two-screens layout in X, to have KDE on primary screen (DVI + LCD monitor) and no window manager on the second screen ( HDMI + Full Hd TV). Mplayer runs on DISPLAY=:0.1 fullscreen using VDPAU, so it takes abouy 30-to-50% of a core to decode 1080p.

When this "usage" occours I try to spot the "crazy" process with 'top' and 'htop' but they don't show any process taking more than 5-10% of a single core, having an i7 virtual cores are 8, so I'd expect to see some process using up to 800% CPU... but nothing.

I don't know how to find out what is taking all this cpu-time, load average, goes from 0.01 to 2-3 in a matter of seconds, I've also tried powertop, to see if it was something related to interrupts.

I'm using LUKS on all system, but it ran smooth on my old Pentium4, so I don't think an i7 has problems with it.

Background processes, and services, include mysql, apache, ktorrent, jDownloader, mldonkey (mlnet), hal, crond, apcupsd...

How can I find out what the problem is?

This are the graphs I got while playing a 1080p movie. Notice the intensive CPU using.

This is another strange screenshot, showing low user+sys+nice usage, but high system usage.

Thanks
--
Qualsiasi

Last edited by Qualsiasi (2010-01-04 14:58:39)

lucke · 2010-01-04 15:09:29

Get dstat from AUR and run something like "dstat -cdnmgsf --top-cpu --top-bio". Perhaps you will see something suspicious. Probably wait/interrupts contribute to the high load.

Qualsiasi · 2010-01-04 15:50:56

I ran dstat and this is the data I've collected:

http://pastebin.com/m623abbca

It looks like CPU is having very high "wait" percentage during the "slow-down", but don't know what's responsible for that.

UPDATE: I've run --top-io and X seems to to LOTS of i/o activity during those slowdowns,

Here's dstat output: http://pastebin.com/m330e8403

Last edited by Qualsiasi (2010-01-04 17:03:55)

TigTex · 2010-01-04 17:37:59

Maybe you are experiencing overheating issues, no?
Thermal throttling may do that.

Qualsiasi · 2010-01-04 17:42:46

TigTex wrote:

Maybe you are experiencing overheating issues, no?
Thermal throttling may do that.

No, temperature is below 40°C (104 °F) all the time.

Cyrusm · 2010-01-04 17:57:48

Is ACPI enabled on your motherboard? I had very similar symptoms a while ago and it turns out ACPI was disabled. re-enabling it fixed the problem

Qualsiasi · 2010-01-04 18:19:12

Cyrusm wrote:

Is ACPI enabled on your motherboard? I had very similar symptoms a while ago and it turns out ACPI was disabled. re-enabling it fixed the problem

Yes, ACPI is enabled ( /proc/acpi exists, and modules like thermal and processor are loaded), in BIOS there i have ACPI 2.0 support disabled, should I give it a try?

arkay · 2010-01-18 00:15:40

Hi All,

Sorry to tag onto an existing thread but I am also suffering from a similar problem. Mine started recently when I had a seagate drive die and I replaced it with a Western Digital. Using clonezilla to partition copy from the seagate (while it was still working), to the WD. Everything copied as expected. The other partitions on the disk work fine but my Arch 64 partition exhibits periods of extremely high wait IO. Locking 2-4 cores up and freezing the desktop for periods up to about a minute at a time. It happens only after boot for about 10 minutes then clears up and goes away.

I have a pastebin here with dstat showing the phenomena:

http://pastebin.com/m6d0697bd

Anyone have a clue as to what could cause this? ACPI is also enabled on my system.

Cheers,

Arkay.

Last edited by arkay (2010-01-18 00:49:39)

Qualsiasi · 2010-01-19 07:44:50

@Arkay

do you use LUKS/dm-crypt on your system?

My drive is:

# hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
Model Number: WDC WD5000AADS-00S9B0
Serial Number: WD-WCAV90862799
Firmware Revision: 01.00A01
Transport: Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6

I tried to enable ACPI 2.0, nothing. Tried to reinstall 32-bit Arch.. nothing. Next time I'll have some free time I'll switch back to Slackware, just to see if it's an hardware issue or a software one.

I'll update.

arkay · 2010-01-19 09:16:05

Hi,

No LUKS on my system. But with mine it only does it for about 5 minutes after boot and never again while it's up during that session.

It's quiet odd. I've noticed a couple of times lately it hasn't done it at all. Then others it's really bad... Too weird!

Cheers,

Arkay.

arkay · 2010-01-27 17:18:27

Just as a follow up mine is still doing it. Really badly now. Next step is re-installation for me. I have another test Mint 7 partition on this machine and it isn't happening there.

It's driving me nuts. I have 2 cores locked solid on 100% IO wait most of the time. Gets so bad the desktop freezes. Even the clock stops, then eventually it will burst back to life.

Cheers,

Arkay.

orphius1970 · 2010-01-28 00:09:30

sorry if this was stated earlier, but which desktop are you sing. I had the same problem with kde4
and the strigi/nepomuk. Disabling it helped. You may want to check your starting services/apps and see if on of
those is the culprit

arkay · 2010-01-28 08:04:26

Hi,

That was my initial thought as well so today I removed kde completely and installed gnome. But it hasn't fixed the problem.

I had a spare partition on the drive so I re-installed Arch via net install. Before I even got to the stage of installing X I saw the system doing exactly the same thing.

My thought at this time are that it is something directly related to the sata driver and the WD Green 1.5 TB drive. I'm yet to confirm it but I'm going to image my install on another seagate drive and see if it goes away. Between that and a possible motherboard bios upgrade I don't have any other avenues to explore.

It could be the 2.6.32 kernel as well but I can't think of any way of confirming that at this point either.

I noticed Qualsiasi is also using a Western Digital drive. Maybe that is the common factor.

Cheers,

Arkay.

arkay · 2010-01-28 08:56:47

Bit more digging on this. Seems the process in question is jbd2 and flush. Worse when firefox is active.

I found this which very much explains what I'm seeing in Arch:

http://ubuntuforums.org/showthread.php?t=1364973

Cheers,

Arkay.

orphius1970 · 2010-01-28 10:12:28

maybe you need to look into the tuning options for your filesystem? For example,
how if i put notail in my fstab for my reiser partitions, it runs better. Not too sure if that
will help.

arkay · 2010-01-28 10:29:59

Dunno. It wasn't previously a problem until I swapped to this current drive. I've just changed to the deadline scheduler to see if it will make a difference. All seems ok at the moment. Will keep an eye on it after a cold boot. Seems that once it settles it stays ok but then rears it's head again the next day.

Cheers,

Arkay.

arkay · 2010-01-31 08:34:06

Well I've sorted out the problem though I can't explain it. At first I thought it was ext4 causing the issues though specific to the WD drive. I've now cloned the WD back onto a Seagate and the problem has gone. Seems it's the way Arch and ext4, possibly 2.6.32 react with the Western Digital drive.

Nothing I did with that drive in the machine made any noticeable difference. Switching back to the Seagate and I've got no wait IO at all. As it should be.

So all I can suggest if you're having similar issues and you have a WD drive is to get rid of it.

Cheers,

Arkay

enrique · 2010-01-31 12:03:48

Looks like there are some issues with WD green drives, I know a guy who has 2 or 3 WD green drives in a NAS box (With embedded Linux), and performance is extremely poor, he has replaced them with another brand (I think it is seagate drives) and performance is back to what you would expect.

arkay · 2010-02-01 01:15:55

I wish I knew what to do to find the exact problem. As it is now I have a Seagate that I can't trust and a perfect (according to S.M.A.R.T) WD drive that I can't use. Has to be something to do with the internal cache on the drive.

Cheers,

Arkay.

arkay · 2010-02-01 03:58:32

Ok. I think I've found the root cause of this issue. It seems that the WD Green drives use a thing called intellipark that will park the drive heads every 8 seconds. When used under Linux the normal flush interval for a drive is 30 sec so what appears to be happening is that the heads on the drive are parking too quickly resulting in any flush to the drive having to re-spin up the drive to write, killing performance and killing the drive. I don't know if this is causing the IO waits on my system but it seems likely. Though it happens on mine with active IO so I can't see that it's the drive spinning down unless the idle detection on the drive is haywire. Still. It's something else to try.

This thread contains some information about it:

http://www.networkedmediatank.com/showt … 327&page=1

Apparently WD have a dos utility that can disable or reprogram the park interval on these drives. It's called wdidle3.exe and is hard to come by.

WD only hand it out per support request but I managed to find a link to the file on the web, I haven't used it yet so I can't vouch for it. But if you want to try you can find it here:

http://home.arcor.de/ghostadmin/wdidle3_1_00.zip

The next problem is requiring a dos boot disk. If I can find my USB key I'll be making a USB Freedos disk so I can attempt to turn this off on my drive, then I can re-test.

I can't believe how seemingly hard it is for drive manufacturers (both the majors Seagate and WD), to get firmware and functions on these drives right. If I could find a sales rep for seagate or WD I'd gladly return these drives... Rectally.

Cheers,

Arkay.

enrique · 2010-02-01 07:33:35

The funny thing is that I also have a 1TB WD green drive that I use for external backup, so I have not really paid any attention to how it performs, I just plug it in and run the backup script. But after this thread I noticed how slow it is... well that's what you get for trying to be a bit green

arkay · 2010-02-01 10:33:17

It may be because I was using this for an OS drive + data in a desktop machine. Probably not as noticeable if used as purely a data drive.

I tried running the util on it to see if I could turn of the parking but the utility didn't recognise that the drive was even plugged in.

Will be getting rid of this drive, can't see a use for it.

Cheers,

Arkay.

dmz · 2010-02-01 11:06:45

Guys, I have 17 WD Green and haven't noticed any problems..?

arkay · 2010-02-02 04:00:15

Are any of them an OS drive?

Cheers,

Arkay.

dmz · 2010-02-02 04:13:19

No, but I promise you that I really write a lot to them, all the time... When I read this thread I felt how my heart jumped, but I've checked every single drive now and... nothing seems wrong.

Edit: By the way, do anyone of you have some WD Elements (WD Green, USB) in posession?

Last edited by dmz (2010-02-02 04:14:45)

Arch Linux

#1 2010-01-04 14:57:29

CPU is used heavily, but htop/top don't show any process using it.

#2 2010-01-04 15:09:29

Re: CPU is used heavily, but htop/top don't show any process using it.

#3 2010-01-04 15:50:56

Re: CPU is used heavily, but htop/top don't show any process using it.

#4 2010-01-04 17:37:59

Re: CPU is used heavily, but htop/top don't show any process using it.

#5 2010-01-04 17:42:46

Re: CPU is used heavily, but htop/top don't show any process using it.

#6 2010-01-04 17:57:48

Re: CPU is used heavily, but htop/top don't show any process using it.

#7 2010-01-04 18:19:12

Re: CPU is used heavily, but htop/top don't show any process using it.

#8 2010-01-18 00:15:40

Re: CPU is used heavily, but htop/top don't show any process using it.

#9 2010-01-19 07:44:50

Re: CPU is used heavily, but htop/top don't show any process using it.

#10 2010-01-19 09:16:05

Re: CPU is used heavily, but htop/top don't show any process using it.

#11 2010-01-27 17:18:27

Re: CPU is used heavily, but htop/top don't show any process using it.

#12 2010-01-28 00:09:30

Re: CPU is used heavily, but htop/top don't show any process using it.

#13 2010-01-28 08:04:26

Re: CPU is used heavily, but htop/top don't show any process using it.

#14 2010-01-28 08:56:47

Re: CPU is used heavily, but htop/top don't show any process using it.

#15 2010-01-28 10:12:28

Re: CPU is used heavily, but htop/top don't show any process using it.

#16 2010-01-28 10:29:59

Re: CPU is used heavily, but htop/top don't show any process using it.

#17 2010-01-31 08:34:06

Re: CPU is used heavily, but htop/top don't show any process using it.

#18 2010-01-31 12:03:48

Re: CPU is used heavily, but htop/top don't show any process using it.

#19 2010-02-01 01:15:55

Re: CPU is used heavily, but htop/top don't show any process using it.

#20 2010-02-01 03:58:32

Re: CPU is used heavily, but htop/top don't show any process using it.

#21 2010-02-01 07:33:35

Re: CPU is used heavily, but htop/top don't show any process using it.

#22 2010-02-01 10:33:17

Re: CPU is used heavily, but htop/top don't show any process using it.

#23 2010-02-01 11:06:45

Re: CPU is used heavily, but htop/top don't show any process using it.

#24 2010-02-02 04:00:15

Re: CPU is used heavily, but htop/top don't show any process using it.

#25 2010-02-02 04:13:19

Re: CPU is used heavily, but htop/top don't show any process using it.

Board footer