cp performance

scarypezsanta · 2011-09-07 06:47:25

When I copy a file foo to foo2 I get a transfer rate of X. If I then copy foo to foo3 and simultaneously copy foo2 to foo4 the transfer rates drop to about X/2 for both processes. I have not even come close to maxing out the IOPS from my SAN so I feel it must have something to do with how cp is written. I have tried dd with similar results. I can see in top that I have two cp processes running but is it really two threads or are the two processes sharing some kind of resource back and fourth that slows both down? Is there anything I can do to increase this?

tavianator · 2011-09-07 19:18:42

What is X approximately? The copy is either IO-bound, network-bound, or CPU-bound.

ethail · 2011-09-07 20:01:06

Sounds like it's not CPU-bounded task, I bet you copy but CPU usage never gets high on those tasks. In addition, CPU's are several orders of magnitude than IO or network.

If foo and foo2 read from the same drive, and foo3/4 write on the same drive, with network interfaces in the middle of it, your performance is bounded by the network, then IO (unless your network is faster than any of the drives in which case the drive is the culprit) Also, with HDD you have to take into consideration how sparse the files are.

Whatever (network or IO) the bounding of the task is, using one scheduler could represent in a gain against another.

Something that does not compute in my mind:

scarypezsanta wrote:

I feel it must have something to do with how cp is written. I have tried dd with similar results.

Why it should be cp the one to take the blame when you report that is not the only tool doing the same?

scarypezsanta · 2011-09-07 23:24:19

Thanks for your reply Ethail. The reason I am leaning towards cp and dd as the culprits is that I know that the Compellent SAN is optimized for multithreaded processes. I can get better IO than this with other types of processes and the network can handle faster traffic as well. I suspect that even when I have two cp or dd processes running they are waiting for each other on and off so the threads are not really running simultaneosly which would be an issue with how the code is written. On a windows server I can use robocopy to get true multithreaded copies completed which blows the single threaded copies out of the water. I am hoping to find a way to reproduce this on my Arch server.

lucke · 2011-09-08 00:04:15

You could try different io schedulers (elevators), including BFQ, which is available e.g. in the linux-pf kernel.

draugdel · 2011-09-08 15:23:57

What is the storage of the SAN like? RAID? Which type?

Multithreaded read of a file is not equal multiprocess read of two files.

Tell us the different scenarios you tested (with numbers!).

Arch Linux

#1 2011-09-07 06:47:25

cp performance

#2 2011-09-07 19:18:42

Re: cp performance

#3 2011-09-07 20:01:06

Re: cp performance

#4 2011-09-07 23:24:19

Re: cp performance

#5 2011-09-08 00:04:15

Re: cp performance

#6 2011-09-08 15:23:57

Re: cp performance

Board footer