IDE I/O takes 100% CPU!

rose · 2005-03-29 13:03:16

I have DMA turned on and get ~ 47MB/s hard disk reading speed tested with hdparm.
But whenever the hard disk or the cdrom is read/written to I get 100% CPU usage (inside kernel)! I tried everything, I googled and found threads about similar cases and I didn't find any solution!

Please, help me, before my disk eats my cpu!

PS: try doing (as root)

cat /dev/hda > /dev/null&
top

And tell me what cpu usage you get. Thanks.

i3839 · 2005-03-29 13:35:03

10% cpu usage for cat, my hdparm speed is 18 Mb/s and cpu 1200 Mhz, kernel 2.6.12-rc1.

rose · 2005-03-29 14:18:05

Being more specific, I get something like this from top:

top - 16:14:07 up 3 min,  1 user,  load average: 0.91, 0.52, 0.21
Tasks:  58 total,   1 running,  57 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.0% us,  9.0% sy,  0.0% ni,  0.0% id, 88.0% wa,  1.0% hi,  0.0% si
Mem:    905284k total,   896128k used,     9156k free,   776308k buffers
Swap:        0k total,        0k used,        0k free,    43148k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2151 root      18   0  1440  336  276 D  8.3  0.0   0:02.18 cat

Cat uses only 8.3% cpu.
But look at id(le) value - it's 0%.
"wa" jumped to 88.0%! This is where the cpu is eaten.

incinerator · 2005-03-29 14:44:14

Try the same stunt whilst running something like a niced cpuburn. Then watch top again.

rose · 2005-03-29 15:19:04

incinerator wrote:

Try the same stunt whilst running something like a niced cpuburn. Then watch top again.

You are right, the "wa" goes to 0 then.
However, I want to use SMP-enabled kernel with my hyper-threading P4. Unfortunately, kernel wastes time with "wa", decreasing my cpu speed. So I must disable SMP.
Any other way to solve this problem?

i3839 · 2005-03-29 15:22:22

"wa" stands for "wait" and means that the kernel is waiting for IO, it says nothing about the cpu usage.

rose · 2005-03-29 15:46:22

It is *actively* waiting.
If you have a ht cpu, then it wastes one virtual cpu (slowing down the other one, because the cpus are only virtual, sharing execution units).

i3839 · 2005-03-29 16:15:12

If you think it's busy looping until there's data, then think again. It just means that the CPU has nothing to do (and is doing nothing), because it's waiting on some IO to finish. It means it wants to do something, but can't thanks to the damn slow IO.

phrakture · 2005-03-29 16:28:53

yeah, you can try changing your scheduler... see if that helps... io is *always* slower than a cpu... it's up to the scheduler to try to figure things out to do while it waits for that io

i3839 · 2005-03-29 16:52:16

io isn't always slower than cpu, think about 1/10 Gbit ethernet... And the scheduler does just faithfully its work, and runs all the runnable tasks. If there are only IO waiting tasks then there's nothing to run and wa will become high.

incinerator · 2005-03-29 16:53:31

rose wrote:

It is *actively* waiting.
If you have a ht cpu, then it wastes one virtual cpu (slowing down the other one, because the cpus are only virtual, sharing execution units).

That is not entirely true. the "wa" field shows the percentage of time the os is spending waiting for I/O requests to finish. However, that is probably not "busy waiting", because the x86 architecture provides mechanisms that enables waiting for i/o without busy cpu waiting (interrupts etc.).

Anyway, my cpuburn example shows that this should not be a big problem. If the os gets something else to do it will preempt the waiting process and give another application the cpu. And if there's two applications wanting to do something instead of waiting, then both will get a virtual (or real) cpu each.

Btw, instead of using top to measure cpu usage you could just do

time cat /dev/hda >/dev/null

Cheers,
Dominik

i3839 · 2005-03-29 17:08:33

If a process is waiting it doesn't have the cpu, it can't have the cpu. It's either running, or standing still.

phrakture · 2005-03-29 17:12:10

i3839 wrote:

If a process is waiting it doesn't have the cpu, it can't have the cpu. It's either running, or standing still.

or swapped out...but that doesn't count...

kth5 · 2005-03-29 17:27:50

you may as well have a buggy driver or a generic one only. have a look at /proc/ide. on boards with via chipsets and the correct drivers loaded a file named via shows up. i dunno about other drivers but it might be that there can as well be intel or whatever.

i3839 · 2005-03-29 18:20:55

phrakture wrote:

or swapped out...but that doesn't count...

A process can run when it's partially swapped out. Swapping happens per memory page, not per process.

rose · 2005-03-29 19:02:25

I have proper chipset drivers compiled.

The problem only exists when i use HYPER-THREADING (tm).

Let me explain the problem by giving you an example. I wanted to rip cd's with grip. Under normal conditions, oggenc gives me at least 8x encoding speed. Ripping cd alone takes no more than 10% cpu. So, you should be really surprised if the ripping+encoding speed was < 7x.
But, surprise, with H-T enabled, I get < 5x!

Why? Because (I suppose) one cpu is busy waiting for io and slows down encoding on the other. The problem is that the two "virtual" cpus share ALU, FPU and so on, so busy waiting on one slows down the other.

phrakture · 2005-03-29 19:05:32

i3839 wrote:

phrakture wrote:
or swapped out...but that doesn't count...
A process can run when it's partially swapped out. Swapping happens per memory page, not per process.

i'm talking about those process that are bracketed when reported by "ps aux" - I was under the impression that those are swapped out due to extensive idle periods... I'm probably way off though

i3839 · 2005-03-29 20:10:05

rose wrote:

Why? Because (I suppose) one cpu is busy waiting for io and slows down encoding on the other. The problem is that the two "virtual" cpus share ALU, FPU and so on, so busy waiting on one slows down the other.

There is no busy waiting, if there is then it's grip which is doing it and then it's a bug in grip. But then it would also happen when HT is disabled.

So if you try it now without HT is't fast, but with HT it's slow? Did you verify that by only changing HT, and nothing else?

As for the tasks between []: Those seem to be tasks without virtual memory usage. That are kernel threads and zombies, not swapped out processes.

rose · 2005-03-29 21:32:47

i3839 wrote:

So if you try it now without HT is't fast, but with HT it's slow? Did you verify that by only changing HT, and nothing else?

Yes, yes. I'll repeat my test tomorrow (if I have time), and give you exact numbers.

RedShift · 2005-03-30 17:42:23

cat /dev/hda > /dev/null

about 5-7% CPU usage, nforce4 board with amd74xx IDE controller. Man I love hardware controllers. The problem could be that your IDE controller is teh suck.

incinerator · 2005-04-01 08:44:07

joy. finally, someone speaks who has a clue...

i3839 · 2005-04-01 20:45:22

It's not the speaking that matters, but the listening. :-/

Arch Linux

#1 2005-03-29 13:03:16

IDE I/O takes 100% CPU!

#2 2005-03-29 13:35:03

Re: IDE I/O takes 100% CPU!

#3 2005-03-29 14:18:05

Re: IDE I/O takes 100% CPU!

#4 2005-03-29 14:44:14

Re: IDE I/O takes 100% CPU!

#5 2005-03-29 15:19:04

Re: IDE I/O takes 100% CPU!

#6 2005-03-29 15:22:22

Re: IDE I/O takes 100% CPU!

#7 2005-03-29 15:46:22

Re: IDE I/O takes 100% CPU!

#8 2005-03-29 16:15:12

Re: IDE I/O takes 100% CPU!

#9 2005-03-29 16:28:53

Re: IDE I/O takes 100% CPU!

#10 2005-03-29 16:52:16

Re: IDE I/O takes 100% CPU!

#11 2005-03-29 16:53:31

Re: IDE I/O takes 100% CPU!

#12 2005-03-29 17:08:33

Re: IDE I/O takes 100% CPU!

#13 2005-03-29 17:12:10

Re: IDE I/O takes 100% CPU!

#14 2005-03-29 17:27:50

Re: IDE I/O takes 100% CPU!

#15 2005-03-29 18:20:55

Re: IDE I/O takes 100% CPU!

#16 2005-03-29 19:02:25

Re: IDE I/O takes 100% CPU!

#17 2005-03-29 19:05:32

Re: IDE I/O takes 100% CPU!

#18 2005-03-29 20:10:05

Re: IDE I/O takes 100% CPU!

#19 2005-03-29 21:32:47

Re: IDE I/O takes 100% CPU!

#20 2005-03-30 17:42:23

Re: IDE I/O takes 100% CPU!

#21 2005-04-01 08:44:07

Re: IDE I/O takes 100% CPU!

#22 2005-04-01 20:45:22

Re: IDE I/O takes 100% CPU!

Board footer