extremely unresponsive system during i/o load

something · 2010-03-27 10:26:54

hello,

I have been using arch for a few years now and have been very happy with it, bar for one recurring problem that i have yet to fix.
during data transfers i suffer extremely slow system response times.

i have looked in the forum already and found a few threads. but have yet to actually find a solution.

example is when i have begun a data transfer of files from one hard drive to another. of any size from 4GB to 1000GB(many files usually from 200MB to 8GB in size)
i have tried ionice on transfers but has not seemed to make system any more responsive. ive had this problem initially on NTFS mounted drives transfering between other NTFS drives.
Recently i decided to scrap NTFS as it has been a long time since i have used the drives directly on a windows machine.
I am still experiencing the problem on EXT4 drives.

my systems configuration:
--
[smelly@^archive ~]$ uname -a
Linux ^archive 2.6.32-ARCH #1 SMP PREEMPT Tue Feb 23 19:43:46 CET 2010 x86_64 Intel(R) Xeon(R) CPU X3350 @ 2.66GHz GenuineIntel GNU/Linux

Quad Core Xeon x3350(@3.6Ghz *seems to not matter if at stock or not)
8GB DDRII RAM
Asus Rampage Formula x48 M/B
nVidia 9600GT
1x250gb SATA WD WD2500AAKS-00VSA0(34GB ROOT,207GB HOME,8.2GB SWAP) - /dev/sda1,2,3
1x1000GB SATA WD WD10EADS-00L5B1(EXT4) - /dev/sdb1
1x1000GB SATA SG ST31000528AS(NTFS) - /dev/sdc1
1x1000GB SATA WD WD10EACS-00D6B0(EXT4) /dev/sdd1
1x1000GB SATA WD WD10EACS-00D6B0(NTFS) /dev/sde1
1xHotSwap SATA Bay that occasionally hosts a 1TB WD Drive.

I am currently changing over all drives to ext4 and am doing them one by one to allow for transferring data off and on again.

atm i am transferring off of one drive onto a free ext4 drive to allow me to convert the ntfs to ext4.

and am suffering this sluggishness. although i am doing the transfer in 50gb chunks. which seems to lower the pain.(although it may be in my head)

example of slowness. gnome-do takes 5-10seconds to appear where is would normally take less than ~1 second
opening pcmanfm takes around 5-10seconds also to open.(normally ~1s)
firefox may open within 10-20seconds whereas normally ~2-3s.

everything that i interact with is fairly slow. taking anywhere from 5s to 30+s to responde to my actions.
many times programs will claim unresponsive and will want to force quit. everything returns to speedy mc-speed-sys when data transfer finishes.

so hopefully anyone out there may be able to help me. as mentioned i have checked existing threads. *although i may have missed something.
i have tried swapoff(which seems to help the most, but it is still sluggish), tried different schedulers.anticipatory vs cfq, but that did little that i could tell.

transfer rates on these drives seem good. getting as high at 110MB/s read and about 80/90MB/s write.
io attempted to use ionice. but the problem still persisted. unless i was using it incorrectly..

hopefully someone out there has succeeded where i have not...

*i have had this problem since i first installed this system, i cant remember what kernel version that was on . but its been about 1 year + maybe a year and a half.

is there anybody else out there who has experienced/is experiencing the same probllems..?

*edit. using mix of xfce + lxde + compiz + gdm

Last edited by something (2010-03-27 10:31:56)

cinan · 2010-03-27 10:32:26

I have the same problem. I can't find any solution, just found this https://bugzilla.kernel.org/show_bug.cgi?id=12309

something · 2010-03-27 10:36:21

hopefully we can find out a solution.

whats your system specs? and what filesystems are you using?

what i find strange is that even though i am barely(if at all) touching the root drive i still slow down to a crawl.
i was thinking of trying out zen kernel. i read somewhere of a user with the same problem having it mostly fixed in zen. but i didnt follow up.

cinan · 2010-03-27 10:57:05

HW specs: Gigabyte P35-S3G, Core2Duo E6400, 2GiB RAM, SATA Seagate 160GB.
I have ext3 and ext4 filesystems, kernel26 (I've also tried ck and zen patchset, but I can see no difference). There's also no difference between 32 and 64bit system.
When I'm copying a big file (say hundreds MiBs) the system is totally unresponsible. Apps launching and graphics rendering is unbelievably slow.

agapito · 2010-03-27 11:06:03

I have to say this.. when there is a lot of i/o load LINUX SUCKS.

flamelab · 2010-03-27 11:17:16

Ionice anyone ?

something · 2010-03-27 11:37:40

i tried using ionice but it didnt seem to really do much..

ionice -c3 -pxxxx etc.

Mountainjew · 2010-03-27 12:40:46

I've had this issue for over a year also.

My current specs:
AMD 5600
Asus M3A78 Pro
2GB Ram

I've read that there are issues with the SB600/SB700 chipset, which i have. Not sure if they've been sorted in the last few kernel updates but sure doesn't feel like it. I've tried many different filesystems and tried changing swappiness and cache pressure. Neither of which have helped much. Getting a high cpu load on transferring to ntfs and other linux filesystems.

At the minute i'm using the Zen kernel and have tried all different io schedulers. At the minute i'm using deadline, but all schedulers are equally as bad. There was a huge thread on the gentoo forums about this but it seems to have ground to a halt. Haven't tried ionice, i'll look into that now.

ngoonee · 2010-03-27 12:57:05

Common issue with 64-bit systems, no-one has ever been able to actually track down the cause I believe. Not sure if -bfs will help you (I think it didn't for me, but I still use it though, and I don't have those sorts of loads anymore).

something · 2010-03-27 13:08:21

damn. i had hopes for zen/BFS, i had only put it off for the use of aur. ill look more into the 64bit thing.

something · 2010-03-27 13:57:19

is anyone with this problem running preload daemon??

brebs · 2010-03-27 18:51:39

Here's the huge Gentoo thread (split into 2 "parts" because it's so huge).

Last edited by brebs (2010-03-27 18:52:24)

agapito · 2010-03-31 07:24:22

something wrote:

is anyone with this problem running preload daemon??

Yes. I am running preload daemon (have 8 gigs of ram)

hungerfish · 2010-04-01 00:18:46

I'm also well accustomed to this behaviour. It 'started' maybe two years ago, I was using gentoo at that time. Now I'm with arch, and it hasn't gone away. Also 64bit Athlon 3000+ here, 5year old machine!

kjslag · 2010-04-13 00:33:28

it seems to be a little better for me with the new kernel

$ uname -a
Linux J 2.6.33-ARCH #1 SMP PREEMPT Sun Apr 4 10:27:30 CEST 2010 x86_64 Intel(R) Core(TM)2 Duo CPU T8300 @ 2.40GHz GenuineIntel GNU/Linux

HoboJ · 2010-04-13 02:09:58

I certainly get lots of delays in opening/using applications but my system has never ground to a halt while doing lots of IO intensive things. Use of the -ck patchset has helped but it's still very noticeable.

$ uname -a
Linux tj-computer 2.6.33-ck #1 SMP PREEMPT Sun Apr 11 18:03:18 EDT 2010 x86_64 AMD Phenom(tm) II X3 710 Processor AuthenticAMD GNU/Linux

Last edited by HoboJ (2010-04-13 02:10:10)

Zariel · 2010-04-13 07:40:50

Try using BFS and SQLB ( i use zen kernel ) it works a charm for me.

kjslag · 2010-04-16 03:43:02

Thanks! The -ck kernel is significantly faster (noticable on kde at least). The noticable differences for me are:
- the alt-2 program launcher is more responsive (especially just after boot). i used to have to wait a little before typing
- my system does grind to a halt when I run out of ram and start using swap. instead, it's just extremely slow (but still moving!)
- programs (especially dolphin) launch faster

$ uname -a
Linux J 2.6.33-ck #1 SMP PREEMPT Thu Apr 15 18:06:50 PDT 2010 x86_64 Intel(R) Core(TM)2 Duo CPU T8300 @ 2.40GHz GenuineIntel GNU/Linux

Procyon · 2010-04-16 08:26:43

For me it's Virtualbox. It uses kswapd even though I have tried to get rid of it for other processes by disabling swap.

According to iotop, in the first few seconds of launching VirtualBox kswapd0 uses about 20-40% IO. After that it goes to 0%. But some external heavy IO or something will make "Virtualbox's kswapd" go crazy and stay at 100% IO making the entire system unresponsive until Virtualbox is closed.

kernel26 2.6.32

I have used -ck for a week when it was first out, I don't remember exactly how that went, but I'm pretty sure it had some kswapd hits as well.

EDIT: I am going to try -ck again. I also found a comment about disabling kswapd, so I did that as well - its init will now return 0 immediately (can't hurt to try, right?)

EDIT2: Don't remove kswapd! It worked well for a while but when I launched Virtualbox some really weird stuff happened.

Last edited by Procyon (2010-04-16 21:59:25)

toad · 2011-09-09 06:23:38

Zariel wrote:

Try using BFS and SQLB ( i use zen kernel ) it works a charm for me.

The bfs kernel I found, but what is sqlb?

toad@archtop 504\4 ~ > packer -Ss sqlb

aur/sqlbrute 1.0-1
    SQLBrute is a tool for brute forcing data out of databases using blind SQL injection.
aur/automysqlbackup 2.5.1-1
    Automatic daily/weekly MySQL backup script
aur/sqlbuddy 1.3.3-1
    Fast web-based tool to administrate MySQL

masteryod · 2011-09-09 11:49:13

so what is the situation about that issue in linux 3.0 era?

I don't know if I understand correctly but while copying a lot of data (about 70GB, 2-4GB per file) from usb2.0 hdd drive to ext4 partition I'm supposed to see kde4 responsiveness slowdown? Because I'll be doing some data dumping next week and I can keep an eye on it

didn't we just necrobumped this thread?

cahr_g · 2011-09-09 22:12:07

with 3.0 system becomes very unresponsive while copying a lot of data, firefox becomes unusable
3.0-ck with bfs works very well while copying files (but I cant use it cause 720p playbacks sucks with bfs on my box)

Gullible Jones · 2011-09-10 01:25:55

In my experience changing the scheduler queue length has been somewhat helpful. The default length of 128 seems to be suboptimal for desktops.

This is done by changing the contents of /sys/block/$DRIVE/queue/nr_requests, like so:

echo 8192 > /sys/block/sda/queue/nr_requests

Don't expect miracles... OTOH, this seems to keep my netbook running smooth when installing updates.

BTW, I'm guessing it's one of those cases of the kernel devs trying to please everyone - what's good for desktops may not be good for server loads. I've heard that shorter queue lengths are better for large RAID arrays, so maybe the devs were trying to compromise.

brebs · 2011-09-10 05:10:48

cahr_g wrote:

720p playback sucks with bfs on my box

What, stuttering?

Try experimenting with rr_interval values, e.g.:

echo 9 > /proc/sys/kernel/rr_interval

toad · 2011-09-14 11:18:12

Sorry about necrobumping - here is the new thread: https://bbs.archlinux.org/viewtopic.php … 71#p990571

Arch Linux

#1 2010-03-27 10:26:54

extremely unresponsive system during i/o load

#2 2010-03-27 10:32:26

Re: extremely unresponsive system during i/o load

#3 2010-03-27 10:36:21

Re: extremely unresponsive system during i/o load

#4 2010-03-27 10:57:05

Re: extremely unresponsive system during i/o load

#5 2010-03-27 11:06:03

Re: extremely unresponsive system during i/o load

#6 2010-03-27 11:17:16

Re: extremely unresponsive system during i/o load

#7 2010-03-27 11:37:40

Re: extremely unresponsive system during i/o load

#8 2010-03-27 12:40:46

Re: extremely unresponsive system during i/o load

#9 2010-03-27 12:57:05

Re: extremely unresponsive system during i/o load

#10 2010-03-27 13:08:21

Re: extremely unresponsive system during i/o load

#11 2010-03-27 13:57:19

Re: extremely unresponsive system during i/o load

#12 2010-03-27 18:51:39

Re: extremely unresponsive system during i/o load

#13 2010-03-31 07:24:22

Re: extremely unresponsive system during i/o load

#14 2010-04-01 00:18:46

Re: extremely unresponsive system during i/o load

#15 2010-04-13 00:33:28

Re: extremely unresponsive system during i/o load

#16 2010-04-13 02:09:58

Re: extremely unresponsive system during i/o load

#17 2010-04-13 07:40:50

Re: extremely unresponsive system during i/o load

#18 2010-04-16 03:43:02

Re: extremely unresponsive system during i/o load

#19 2010-04-16 08:26:43

Re: extremely unresponsive system during i/o load

#20 2011-09-09 06:23:38

Re: extremely unresponsive system during i/o load

#21 2011-09-09 11:49:13

Re: extremely unresponsive system during i/o load

#22 2011-09-09 22:12:07

Re: extremely unresponsive system during i/o load

#23 2011-09-10 01:25:55

Re: extremely unresponsive system during i/o load

#24 2011-09-10 05:10:48

Re: extremely unresponsive system during i/o load

#25 2011-09-14 11:18:12

Re: extremely unresponsive system during i/o load

Board footer