Arch Linux

SyXbiT

Member

From: Seattle, WA

Registered: 2008-06-28

Posts: 177

Website

anyone noticed how copying GBs of data makes any linux distro crawl

I'm pretty sure it's cause of the Ingo Molnar's Completely Fair Scheduler. I don't remember this happening before, but can't be sure

If i copy 50GB of files from one SATA 7200rpm ext3 drive to another, all apps are excruciatingly slow (including firefox, nautilus etc..) until the copying is finished
whereas non-linux kernels (vista, OS X etc..) don't have this issues.
In vista, firefox seems pretty responsive even while copying.

has anyone else noticed?

Last edited by SyXbiT (2009-01-14 03:58:12)

Procyon

Member

Registered: 2008-05-07

Posts: 1,819

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Not noticed it and I have copied hundreds of GB at a time to an encrypted USB HDD and between internal HDDs.

Could use firefox without any problem at all -- didn't even notice whether it was done or not.

EDIT:

OK I am noticing it right now. I am copying two 7GB files from ext3 to XFS.

cpuload is 4 to 7 on dual core.

Last edited by Procyon (2009-01-31 16:55:37)

Wintervenom

Member

Registered: 2008-08-20

Posts: 1,011

Re: anyone noticed how copying GBs of data makes any linux distro crawl

[...] has anyone else noticed?

Yes, this happens to me, too.  When the I/O goes up, everything else goes down.[/i].

Last edited by Wintervenom (2009-08-03 14:12:51)

fukawi2

Ex-Administratorino

From: .vic.au

Registered: 2007-09-28

Posts: 6,224

Website

Re: anyone noticed how copying GBs of data makes any linux distro crawl

If i copy 50GB of files from one SATA 7200rpm ext3 drive to another

ext3 is a very reliable and well-tested file systen, but it's far from a good performer when it comes to speed.

I do similar data transfers with JFS without a problem.

---

Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?

BlueHackers // fscanary // resticctl

ak-89

Member

From: Finland

Registered: 2008-08-26

Posts: 86

Website

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I had this too when I was still using Arch's stock kernel, when disk I/O went up everything slowed down. Maybe my computer has something against CFQ scheduler(wasn't that Arch  default) now I am using BFQ and I haven't noticed any slowdowns.
BFQ isn't in stock kernel or is it?
You can try to change your scheduler by adding to grub's kernel line "elevator=deadline" or catting "deadline" to somewhere-i-cant-remember-maybe-/proc/..-or-/sys/.. You can replace "deadline" with whatever is supported by your kernel.
You can check what schedulers was compiled in
zcat /proc/config.gz | grep IOSCHED

archlinuxsagi

Member

Registered: 2008-09-12

Posts: 259

Re: anyone noticed how copying GBs of data makes any linux distro crawl

So far, the slow down in my computer is caused by either opera or vmware.
File transfers.. not really.
I have switch fam to garmin when i found out that fam cpu usage is 100% during file transfer.

Last edited by archlinuxsagi (2009-01-14 05:49:53)

skottish

Forum Fellow

From: Here

Registered: 2006-06-16

Posts: 7,942

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I'm pretty sure it's cause of the Ingo Molnar's Completely Fair Scheduler. I don't remember this happening before, but can't be sure

You're right, it was around the time of the new scheduler going into mainline. It was discussed here before quite a few times.

Dheart

Member

From: Sofia, Bulgaria

Registered: 2006-10-26

Posts: 956

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I think those slowdowns are related to the filesystem you use. It is true that JFS/XFS are the best for big files, but here on reiserfs I don't notice any slowdowns either, even though I'm on a 5200 RPM notebook HDD.

---

My victim you are meant to be
No, you cannot hide nor flee
You know what I'm looking for
Pleasure your torture, I will endure...

sirocco

Member

Registered: 2008-03-10

Posts: 149

Re: anyone noticed how copying GBs of data makes any linux distro crawl

May be similar this bug http://bugzilla.kernel.org/show_bug.cgi?id=12072

ArchArael

Member

Registered: 2005-06-14

Posts: 504

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I use deadline scheduler and xfs filesystem and I can copy any amount of GB and continue with my work without any noticeable slowdown.
I'm quite happy with my setup.

gorn

Member

Registered: 2008-02-01

Posts: 56

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I have this problem and it sounds like I need to look at my scheduler, but another related tip is that you can use ionice and nice to put some processes (like your backup) at a lower priority.

nice -n 19 ionice -c3 <backup program or cp or whatever>

<edit>
To play with the schedulers check:
/sys/block/<DriveBlockDevice>/queue/scheduler (such as /sys/block/sda/queue/scheduler)

cat it to see what schedulers you have, the current on is in []:
noop anticipatory deadline [cfq]

and:
echo deadline > scheduler
cat scheduler
noop anticipatory [deadline] cfq
</edit>

Last edited by gorn (2009-01-15 03:03:57)

Ranguvar

Member

Registered: 2008-08-12

Posts: 2,549

Re: anyone noticed how copying GBs of data makes any linux distro crawl

/me makes a mental note to try and benchmark each scheduler when I next compile a kernel

Is there anything about the CFS that seems like it could cause I/O problems?

pointone

Wiki Admin

From: Waterloo, ON

Registered: 2008-02-21

Posts: 379

Re: anyone noticed how copying GBs of data makes any linux distro crawl

http://bugzilla.kernel.org/show_bug.cgi?id=12309 perhaps?

---

M*cr*s*ft: Who needs quality when you have marketing?

lzs

Member

Registered: 2008-01-23

Posts: 39

Re: anyone noticed how copying GBs of data makes any linux distro crawl

It only happens to me when I write to the same physical disk my system lies on. Writing to other disks or reading from the system disk is fine though ... (etx3)

bangkok_manouel

Member

From: indicates a starting point

Registered: 2005-02-07

Posts: 1,556

Re: anyone noticed how copying GBs of data makes any linux distro crawl

http://bugzilla.kernel.org/show_bug.cgi?id=12309 perhaps?

http://it.slashdot.org/article.pl?sid=09/01/15/049201

buttons

Member

From: NJ, USA

Registered: 2007-08-04

Posts: 620

Re: anyone noticed how copying GBs of data makes any linux distro crawl

No.  I'm on ext3 and I've never noticed a problem.  I do copy a great deal of data as well.

Be sure that you're using noatime.  I also recommend dir_index and data=writeback, but noatime is the only thing I can imagine would cause something like that.

---

Cthulhu For President!

Ranguvar

Member

Registered: 2008-08-12

Posts: 2,549

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Hmmmm...

http://it.slashdot.org/it/09/01/15/049201.shtml

Maybe?

SLKDK

Member

Registered: 2008-08-11

Posts: 61

Re: anyone noticed how copying GBs of data makes any linux distro crawl

I also notice HEAVY slowdown when copying files to another network computer, or when downloading many big files with a torrent program like deluge.
Or when unrar'ing af large .rar file (like multiple rar files). Opening Opera/Firefox or something could take minutes..

Im on ext3 on a laptop.

Could be really nice if there was a fix for this.

Zariel

Member

Registered: 2008-10-07

Posts: 446

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Theres a regression which causes high latency when doing large IO, been in since like 2.6.18 or something

/e http://lkml.org/lkml/2009/1/16/487

Last edited by Zariel (2009-02-15 17:31:34)

SLKDK

Member

Registered: 2008-08-11

Posts: 61

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Zariel> So no fix for this? (Not that good at english).

Best regards.

PS. Im gonna reinstall Arch right now with EXT4 to see if it helps alittle.

adamlau

Member

Registered: 2009-01-30

Posts: 418

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Compression/decompression, yes. Gigabyte file transfers across XFS, not nearly as much.

Last edited by adamlau (2009-02-15 22:52:09)

---

Arch Linux + sway
Debian Testing + GNOME/sway
NetBSD 64-bit + Xfce

Zibi1981

Member

From: Poland

Registered: 2008-01-31

Posts: 644

Re: anyone noticed how copying GBs of data makes any linux distro crawl

It only happens to me when I write to the same physical disk my system lies on. Writing to other disks or reading from the system disk is fine though ... (etx3)

I've noticed similar issue in my case. My filesystem is JFS, and I'm using deadline elevator, of which I've read it's the best option for JFS. My HDD is WD Caviar WD2500BEVS (2,5", 250GB, 8MB, SATA/150, 5400rpm).

---

"... being a Linux user is sort of like living in a house inhabited by a large family of carpenters and architects. Every morning when you wake up, the house is a little different. Maybe there is a new turret, or some walls have moved. Or perhaps someone has temporarily removed the floor under your bed."

MSI Raider GE78HX 13VI-032PL

Wintervenom

Member

Registered: 2008-08-20

Posts: 1,011

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Perhaps I ought to try the Zen kernel.

Last edited by Wintervenom (2009-08-03 14:13:29)

jordi

Member

Registered: 2006-12-16

Posts: 103

Website

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Maybe you guys have the issue described here

---

Blog | Github | Twitter

lzs

Member

Registered: 2008-01-23

Posts: 39

Re: anyone noticed how copying GBs of data makes any linux distro crawl

Maybe you guys have the issue described here

Thanks, swapping indeed was the offender for me. Funny thing, I already stumbled across that site a year or so ago, but stopped reading after the problem description as it did not seem to apply to my problem ...