Dumb question regarding dd performance tests

dsulli99 · 2013-07-04 19:56:04

Hi,

Would anybody be able to speculate on why the following invocations of the dd utility appear to produce linear results (in terms of data transfer rate) based on the count implemented? There is no practical reason why I need to know this other than pure curiosity :-)

http://pastebin.com/Mv7e05sB

I appreciate your taking the time to answer my question.

Dan

**Note: updated post to use pastebin for output and to change command from hdparm to dd as suggest by the command output

Last edited by dsulli99 (2013-08-04 02:43:22)

HalosGhost · 2013-07-04 19:57:54

Please read the Forum Etiquette, specifically, this section might be of interest.

All the best,

-HG

deepsoul · 2013-07-09 20:11:33

dsulli99 wrote:

Would anybody be able to speculate on why the following invocations of the hdparm utility appear to produce linear results (in terms of data transfer rate) ...

Is this a trick question? There is no hdparm invocation in the code snippets. The data rate is approximately constant with respect to wallclock time, which includes the second sync. This is just what one would expect.

dsulli99 · 2013-07-12 17:46:05

I apologize, I mean to say dd, not hdparm (I was using hdparm for something else, I have updated the post to reflect this). I agree with you that the data rate is approximately constant with respect to the wall time for each invocation. My concern is that the data rate varies depending on the amount of data being copied. For example, the first iteration yields a data rate of 71.7 MB/s; the second iteration yields 129 MB/s, and the third 984 MB/s. Since the data rate reported accounts for the time (i.e. divides by /s), I would think that these would be more consistent, i.e ~71.7 MB/s for all three iterations. The rate appears to grow linearly with the amount of data being copied, however I wouldn't expect this as this calculation factors time into the equation. Does this make sense?

Last edited by dsulli99 (2013-08-04 02:43:42)

alphaniner · 2013-07-12 18:25:17

From my minimal understanding of HDDs and filesystems, buffering/caching has to be part of the issue. There's also location on the disk. I believe writes to the 'beginning' (outer edge of platters) of the disk tend to be faster than those to the 'end' (inner edge of platters) of the disk. Still, it doesn't seem these things could cause such a discrepancy.

If you have a spare/unused HDD you can try dd'ing to the 'raw disk' (/dev/sdX). In my experience speeds [reported by dd] tend to be pretty consistent in that case, at least once you're beyond the disk cache.

Also, you should edit the first post to s/hdparm/dd. You can also change the thread title that way.

Last edited by alphaniner (2013-07-12 18:25:58)

R00KIE · 2013-07-13 22:06:22

Try something like 'dd if=/dev/sdx of=/dev/null bs=10M count=10 iflag=direct', adjust the bs and count values to your liking and add any other options you deem necessary (skip= might be useful).

Edit:
Ooops, you are benchmarking writes, just exchange the if and of parts. I didn't dare to try because I don't have any spare disks for testing right now but it should work.

Last edited by R00KIE (2013-07-13 22:08:47)

deepsoul · 2013-07-14 22:23:26

dsulli99 wrote:

For example, the first iteration yields a data rate of 71.7 MB/s; the second iteration yields 129 MB/s, and the third 984 MB/s. Since the data rate reported accounts for the time (i.e. divides by /s), I would think that these would be more consistent, i.e ~71.7 MB/s for all three iterations. The rate appears to grow linearly with the amount of data being copied,

dd does not include the time it takes to flush the data to disk in its calculation. So it consistently outputs a higher data rate than what one gets using the wallclock time. Also, the data rate actually falls for larger amounts of data, because more of it is written to disk while dd is running (rather then only on the final sync). It is the amount you write with a single dd command that causes the change, not the order in which you run them.

Arch Linux

#1 2013-07-04 19:56:04

Dumb question regarding dd performance tests

#2 2013-07-04 19:57:54

Re: Dumb question regarding dd performance tests

#3 2013-07-09 20:11:33

Re: Dumb question regarding dd performance tests

#4 2013-07-12 17:46:05

Re: Dumb question regarding dd performance tests

#5 2013-07-12 18:25:17

Re: Dumb question regarding dd performance tests

#6 2013-07-13 22:06:22

Re: Dumb question regarding dd performance tests

#7 2013-07-14 22:23:26

Re: Dumb question regarding dd performance tests

Board footer