System Laggs while copying files

K0tuk · 2009-04-20 09:59:35

Hi!

I`ve ran into such issue: when copying a large amount of files or one big file ( from one partition to another, from flash drive to partition, from DVD to partition and etc. ) my system starts lagging. Even though the copying process runs pretty fast, for that time working with OS becomes... difficult. What can be the problem? Hard drive issue? I had a similar problem in Linux Mint 6. All my partitions are in ext3 format. My laptop is Acer 7720g and i run Arch x86_64. no problems in Windows Vista, even though Vista sucks anyway

moljac024 · 2009-04-20 10:02:01

Google for swapiness

SiD · 2009-04-20 10:08:10

Hi, I have the same problem. I googled for swappiness.
Which value for swappiness should I use?

K0tuk · 2009-04-20 12:39:35

moljac024 wrote:

Google for swapiness

So... what you suggest is setting swappiness to 0 to use only RAM for copying? Or setting it to 100 to use SWAP as much as possible?

Anyway, i`ll experiment with values when i get back home.

Thanks for reply.

Wilco · 2009-04-20 13:19:20

This is a problem that is hard to overcome.

There are dual and quad core processors, but when copying large amounts of data, like a DVD from partition1 to partition2, SATA or IDE cause a very high system load. That's why the system becomes laggy or unresponsive while at the same time CPU usage is <20%.
The same issues exist with windows or FreeBSD. It's mainly a hardware limiting factor.

K0tuk · 2009-04-20 18:14:15

So no ways of overcoming that problem? Than how Windows handles it, if i don`t see any lagging in it while copying these files?

brebs · 2009-04-20 20:48:44

K0tuk wrote:

my system starts lagging.
x86_64

This is a known-and-moaned-about-for-ages problem with 64-bit Linux.

K0tuk · 2009-04-21 07:00:43

brebs wrote:

This is a known-and-moaned-about-for-ages problem with 64-bit Linux.

Hm... that sucks! My acer has Core 2 Duo processor...

Anyway, in this thread a guy says that

I noticed immediately that the system lags a lot, especially when under
load, sometimes when not under load. I figured bad scheduling at first.
Switching to ext4 helped.

Maybe this could help?

takedown · 2009-04-21 10:29:15

K0tuk wrote:

Anyway, in this thread a guy says that
I noticed immediately that the system lags a lot, especially when under
load, sometimes when not under load. I figured bad scheduling at first.
Switching to ext4 helped.
Maybe this could help?

No help

K0tuk · 2009-04-21 11:41:18

Hmm... i`ve found a similar topic about http://www.techspot.com/blog/224/slow-s … -2003-x64/

Last edited by K0tuk (2009-04-21 11:41:44)

sirocco · 2009-04-21 11:48:08

... and another topic http://bbs.archlinux.org/viewtopic.php?id=62985

graysky · 2009-04-28 20:29:52

takedown wrote:

K0tuk wrote:
Anyway, in this thread a guy says that
I noticed immediately that the system lags a lot, especially when under
load, sometimes when not under load. I figured bad scheduling at first.
Switching to ext4 helped.
Maybe this could help?
No help

I'm using ext4 and my system lags on Arch x86_64 when copying as well.

graysky · 2009-04-28 20:44:25

SiD wrote:

Hi, I have the same problem. I googled for swappiness.
Which value for swappiness should I use?

If you read the post I linked below, the author recommends the following to be added to your /etc/sysctl.conf which might be what moljac024 is talking about.

# tip from http://forums.debian.net/viewtopic.php?f=16&t=31275
vm.swappiness=20
vm.vfs_cache_pressure=50

Last edited by graysky (2009-04-28 20:45:11)

loudtiger · 2009-04-28 20:44:37

slightly unrelated, does anyone find that deleting a particularly large file takes a long time? i'm talking about 4-10gb files, on an ext3-> ext4 partition. say, wasn't there supposed to be a defrag tool for extents?

Wintervenom · 2009-04-28 20:45:39

It does the same for me, both natively and on a virtual machine, and on two different machines. Filesystem and sysctl tweaks didn't really matter.

Last edited by Wintervenom (2009-08-03 14:23:51)

graysky · 2009-04-28 20:53:39

loudtiger wrote:

slightly unrelated, does anyone find that deleting a particularly large file takes a long time? i'm talking about 4-10gb files, on an ext3-> ext4 partition. say, wasn't there supposed to be a defrag tool for extents?

I answered your questions in the following thread. Let's keep the replies in this thread related to the system lag while copying files topic

http://bbs.archlinux.org/viewtopic.php?pid=544695

dobedo · 2009-04-29 19:43:42

Just googled a bit on slowness and found info about ionice command (in Arch it's part of the util-linux-ng package).
Has someone tried it and found this as a solution to system lag ?

K0tuk · 2009-09-15 16:34:54

So... i hate to bring this old topic up, but has somebody found out the solution to this problem? Because i have just tested it (copied a 2,3Gb file from one partition to another, and whole system became slow as hell... But i saw a lot of Kernel updates since the last post. Any info on this issue?

Procyon · 2009-09-15 17:37:57

I found ionice and renice to be useless by themselves. The BFS thread has some hints to get better performance in CFS and they apply to this issue as well (as far as I noticed, the system doesn't cripple here and loadavg stays below 4). To summarize:

sudo sh -c 'echo 0 > /proc/sys/kernel/sched_wakeup_granularity_ns'
sudo sh -c 'echo 1000000 > /proc/sys/kernel/sched_latency_ns'
sudo sh -c 'echo 2000000 > /proc/sys/kernel/sched_min_granularity_ns'

And you can go even further with the NO_NEW_FAIR_SLEEPERS trick:

sudo mkdir /debug
sudo mount -t debugfs debugfs /debug
sudo sh -c 'echo NO_NEW_FAIR_SLEEPERS > /debug/sched_features'

(edit: added some more sudo)

Last edited by Procyon (2009-09-15 18:49:50)

K0tuk · 2009-09-18 21:21:34

Thanks, i`ll try that and report later about how it improved things.

K0tuk · 2009-09-20 11:13:36

Well, it didn`t improve anything... The system becomes almost unresponsive when copying a 2Gb file

Procyon · 2009-09-20 12:27:04

Could you try out this ionice and renice script with the above scheduler tweaks. With both good and bad I wonder if it will work better than ioreniced. You can add to the lists, if you don't add ( and ) it will match anywhere in the process name.

#! /bin/bash
BAD='(cp)
(mv)
(make)
cc1
(atool)
(easytag)
(unzip)
(unrar)'
GOOD='(mplayer)
(mpd)
(firefox)
(musca)
(openbox)
(awesome)
(dwm)
(Xorg)'
IFS="
"
while sleep 5; do
badpid=$(grep -Fh "$BAD" /proc/[0-9]*/stat 2>/dev/null | cut -d' ' -f1)
if [ -n "$badpid" ]; then
ionice -c 3 -t -p $badpid >/dev/null 2>&1
renice -n 20 $badpid >/dev/null 2>&1
fi

goodpid=$(grep -Fh "$GOOD" /proc/[0-9]*/stat 2>/dev/null | cut -d' ' -f1)
if [ -n "$goodpid" ]; then
ionice -c 2 -n 0 -t -p $goodpid >/dev/null 2>&1
renice -n -10 $goodpid >/dev/null 2>&1
fi
done

K0tuk · 2009-10-19 08:05:56

Sorry for a late reply.

I tried ioreniced and it helped a bit, but not by much... For example, it takes 30 seconds for Totem to start Going to try schedtoold from the AUR too.

sirocco · 2009-10-20 09:05:28

May be this is solution
http://bugzilla.kernel.org/show_bug.cgi?id=7372

Ok, wow...I just solved the problem here. Apparently write caching was
disabled. I'd never used sdparm before. Found a thread on Web Hosting Talk that
sounded like this issue and explained that fix.
sdparm -s WCE=1 /dev/sda
And everything's fine. If that helps others, I take no credit. If it doesn't,
I'm sorry for adding to the noise.

graysky · 2009-11-07 10:50:28

I think that's enabled by default:

# sdparm -a /dev/sda | grep WCE
  WCE         1

I believe this is a problem with the kernel scheduler. 2.6.32 will probably fix this somewhat based on my testing, but it is still present. Have a look at the following threads:

Multicore scheduler improvement in 2.6.32-rc6 - http://bbs.archlinux.org/viewtopic.php?id=83957
Compile you own kernel: http://bbs.archlinux.org/viewtopic.php?id=84133

Arch Linux

#1 2009-04-20 09:59:35

System Laggs while copying files

#2 2009-04-20 10:02:01

Re: System Laggs while copying files

#3 2009-04-20 10:08:10

Re: System Laggs while copying files

#4 2009-04-20 12:39:35

Re: System Laggs while copying files

#5 2009-04-20 13:19:20

Re: System Laggs while copying files

#6 2009-04-20 18:14:15

Re: System Laggs while copying files

#7 2009-04-20 20:48:44

Re: System Laggs while copying files

#8 2009-04-21 07:00:43

Re: System Laggs while copying files

#9 2009-04-21 10:29:15

Re: System Laggs while copying files

#10 2009-04-21 11:41:18

Re: System Laggs while copying files

#11 2009-04-21 11:48:08

Re: System Laggs while copying files

#12 2009-04-28 20:29:52

Re: System Laggs while copying files

#13 2009-04-28 20:44:25

Re: System Laggs while copying files

#14 2009-04-28 20:44:37

Re: System Laggs while copying files

#15 2009-04-28 20:45:39

Re: System Laggs while copying files

#16 2009-04-28 20:53:39

Re: System Laggs while copying files

#17 2009-04-29 19:43:42

Re: System Laggs while copying files

#18 2009-09-15 16:34:54

Re: System Laggs while copying files

#19 2009-09-15 17:37:57

Re: System Laggs while copying files

#20 2009-09-18 21:21:34

Re: System Laggs while copying files

#21 2009-09-20 11:13:36

Re: System Laggs while copying files

#22 2009-09-20 12:27:04

Re: System Laggs while copying files

#23 2009-10-19 08:05:56

Re: System Laggs while copying files

#24 2009-10-20 09:05:28

Re: System Laggs while copying files

#25 2009-11-07 10:50:28

Re: System Laggs while copying files

Board footer