unresponsive system when copying large files drive-to-drive (x86_64)

graysky · 2009-04-28 20:25:45

I'm running Arch x86_64 on a quad core Xeon processor/P35 motherboard w/ 4 gigs of RAM. I also have two SATAII, 7200 RPM HDDs. For the most part, Arch is a VERY responsive platform. I am noticing some really odd unresponsiveness that only happens when I copy large files from one hdd to another hdd.

Symptoms:
-Mouse cursor skips (doesn't move smoothly)
-Windows such as firefox or shell don't move smoothly across the screen if I move them
-Keyboard lag - as I type this post ) my fingers are going faster than the text is appearing

Once the copy is finished, everything is normal again. I swear I read someone else reporting this behavior, I just can't find it. Two questions:

1) Is normal for the x86_64 kernel?
2) If not, what can I tweak to remedy it?

EDIT: Ack! Found the post I remembered reading:

http://bbs.archlinux.org/viewtopic.php?id=70525

Last edited by graysky (2009-07-04 10:51:35)

graysky · 2009-07-04 10:50:51

I read somewhere that the 2.6.30 kernel contained some fixes for this issue under x64 Linux but after updating to it, I found that my system is still damn laggy when copying files drive-to-drive. Is there a fix for this uder Arch x86_64 or is it just a fact of life?

darthaxul · 2009-07-04 19:45:13

if the file is bigger than 4gig(uncompressed) then u need swap partition, as the first thing it does is copy the file to memory and if it overflows it is bad. then your server is 1/3 options, either server/middle/desktop. if it is server, it doesn't care what framerate u get on X. then there is the timer. I'm running mine at 300 but arch's default should be 1000. the higher it is the more it "checks" what needs to be done with files, events, etc. then there is the file system type and block size factor. if ur block size is really low like me(1024k) then ur hd has to write more times(on and off) just to write out the huge file. so for huge files, format with huge block size(4096k?) so that the hd does less work.

Last edited by darthaxul (2009-07-04 19:49:07)

R00KIE · 2009-07-05 03:04:26

Kernel 2.6.30 should help a bit but I've seen this behavior while copying 700M ~ 1G files (30G total) from HD to an external HD via USB (notebook to external box).

I Run my system on ram only as I don't have a swap partition mounted and apps like firefox and pcmanfm were already loaded and running and yet would lag a lot while the files were copying.
What helps just a little bit is to change the disk scheduler to deadline instead of the default cfq.

graysky · 2009-07-05 10:12:01

I have 8 gigs of RAM on this system and 2 gigs of swap. This happens when copying even a 1 gig file from say /dev/sda4 to /dev/sda2

metapy · 2009-07-06 08:27:17

I had the same problem, but it seems to be related to the swap file. Disabled swap and it all runs smoothly now. I was using tmpfs for swap which may have had something to do with it. I am going to run swapless for now and see if any other problems pop up.

schuay · 2009-07-06 10:53:56

metapy wrote:

I had the same problem, but it seems to be related to the swap file. Disabled swap and it all runs smoothly now. I was using tmpfs for swap which may have had something to do with it. I am going to run swapless for now and see if any other problems pop up.

Nope. Never had any swap file/partition and I also had/have this issue.

lilsirecho · 2009-11-04 03:42:20

Perhaps the problem is related to the size of tmpfs which would choke after reaching its partitioned limit if no swap were provided as a scapegoat. Thus either way the transfer would get mighty slow.................

Pierre · 2009-11-04 05:09:52

This is quite an old and known problem. But I had heard it will be fixed in 2.6.32.

ngoonee · 2009-11-04 07:18:16

Pierre wrote:

This is quite an old and known problem. But I had heard it will be fixed in 2.6.32.

Sources please? Very good news, if true.

Ranguvar · 2009-11-04 07:48:23

Also have the issue, no swap file, 6GiB RAM.

Anyone have the problems go away after switching to BFS?

shazeal · 2009-11-05 02:33:03

This is a really old issue has been around 2.6.19 I belive. Best way to solve this is to change your IO scheduler to something more suitable for a desktop, BFQ or the BFS both resolve it on machines I use reguarly, whenever I use a machine running CFQ (default scheduler) I notice the problem straight away. No idea about potential fixes in 2.6.32 I am still on 2.6.31 atm. But unless they have overhauled the IO scheduler I would doubt it.

Mountainjew · 2009-11-05 02:43:41

I've had this problem since switching from 32-bit to 64-bit Arch about 6 months ago. I've tried the BFS kernel and didn't notice much difference. I've yet to try the Zen kernel with BFS/BFQ since i can never get it to compile, always sticks on some module. Hope this is true that it's being addressed in 2.6.32. It would be kinda cool to have a fast linux box again

graysky · 2009-11-05 20:41:58

@MJ - I don't think this has been addressed in 2.6.32 at all... the speedup there as I understand it is only for multicore CPUs and how work gets scheduled. I don't think they've addressed the IO scheduler bug.

EDIT: I tried to test this booting into my 2.6.32-rc6 kernel, but I can't get X to come up since I haven't gone through the trouble of recompiling the nvidia driver. Maybe someone else can test this and post back?

Last edited by graysky (2009-11-05 21:02:05)

graysky · 2009-11-07 10:27:44

Finally got around to building a 2.6.32-rc6 kernel with nvidia driver (see [url=]this thread[/url] for instructions). Anyway, booted into it and moved an 8 gig file from partition to partition. The lag is still there, but it is greatly improved.

Mikko777 · 2009-11-07 12:30:32

With lvm+raid ext4 partitions the filecopy/ move lag is usually nonexistant.

Why... I have no idea.

iBertus · 2009-12-06 05:30:29

Mikko777 wrote:

With lvm+raid ext4 partitions the filecopy/ move lag is usually nonexistant.
Why... I have no idea.

I've got the same type of setup and still experience the lag. I'm going to update to 2.6.32 and see if that helps. I always feel cheated when transfering files on my machine (Intel Skulltrail w/ H2O cooling) and it lags when my wife can do this on her crappy Windows laptop and it isn't an issue.

I transfer lots of huge files from external media onto my RAID array and it's a pain in the butt to not be able to do anything else while waiting for the copy to complete.

iBertus · 2009-12-08 02:56:07

I updated to 2.6.32 and now my lvm array isn't detected on bootup :-(

graysky · 2009-12-08 08:38:33

iBertus wrote:

I updated to 2.6.32 and now my lvm array isn't detected on bootup :-(

U using the packages from [testing] or did you build your own? Did you mod the hooks in your /etc/mkinitcpio.conf for your lvm?

http://wiki.archlinux.org/index.php/Ins … igure_Arch
http://wiki.archlinux.org/index.php/Con … Using_raid

Arch Linux

#1 2009-04-28 20:25:45

unresponsive system when copying large files drive-to-drive (x86_64)

#2 2009-07-04 10:50:51

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#3 2009-07-04 19:45:13

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#4 2009-07-05 03:04:26

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#5 2009-07-05 10:12:01

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#6 2009-07-06 08:27:17

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#7 2009-07-06 10:53:56

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#8 2009-11-04 03:42:20

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#9 2009-11-04 05:09:52

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#10 2009-11-04 07:18:16

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#11 2009-11-04 07:48:23

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#12 2009-11-05 02:33:03

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#13 2009-11-05 02:43:41

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#14 2009-11-05 20:41:58

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#15 2009-11-07 10:27:44

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#16 2009-11-07 12:30:32

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#17 2009-12-06 05:30:29

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#18 2009-12-08 02:56:07

Re: unresponsive system when copying large files drive-to-drive (x86_64)

#19 2009-12-08 08:38:33

Re: unresponsive system when copying large files drive-to-drive (x86_64)

Board footer