You are not logged in.

#1 2013-07-04 19:56:04

dsulli99
Member
Registered: 2010-11-18
Posts: 31

Dumb question regarding dd performance tests

Hi,

Would anybody be able to speculate on why the following invocations of the dd utility appear to produce linear results (in terms of data transfer rate) based on the count implemented?  There is no practical reason why I need to know this other than pure curiosity :-)

http://pastebin.com/Mv7e05sB

I appreciate your taking the time to answer my question.

Dan

**Note: updated post to use pastebin for output and to change command from hdparm to dd as suggest by the command output

Last edited by dsulli99 (2013-08-04 02:43:22)

Offline

#2 2013-07-04 19:57:54

HalosGhost
Member
From: Twin Cities, MN
Registered: 2012-06-22
Posts: 1,486
Website

Re: Dumb question regarding dd performance tests

Please read the Forum Etiquette, specifically, this section might be of interest.

All the best,

-HG


"All errors are ᴘᴇʙᴋᴀᴄ errors—It's just a matter of narrowing down which keyboard and chair." -Trilby
\ldots

Offline

#3 2013-07-09 20:11:33

deepsoul
Member
From: Earth
Registered: 2012-12-23
Posts: 57
Website

Re: Dumb question regarding dd performance tests

dsulli99 wrote:

Would anybody be able to speculate on why the following invocations of the hdparm utility appear to produce linear results (in terms of data transfer rate) ...

Is this a trick question?  There is no hdparm invocation in the code snippets.  The data rate is approximately constant with respect to wallclock time, which includes the second sync.  This is just what one would expect.


Officer, I had to drive home - I was way too drunk to teleport!

Offline

#4 2013-07-12 17:46:05

dsulli99
Member
Registered: 2010-11-18
Posts: 31

Re: Dumb question regarding dd performance tests

I apologize, I mean to say dd, not hdparm (I was using hdparm for something else, I have updated the post to reflect this).  I agree with you that the data rate is approximately constant with respect to the wall time for each invocation.  My concern is that the data rate varies depending on the amount of data being copied.  For example, the first iteration yields a data rate of 71.7 MB/s; the second iteration yields 129 MB/s, and the third 984 MB/s.  Since the data rate reported  accounts for the time (i.e. divides by /s), I would think that these would be more consistent, i.e ~71.7 MB/s for all three iterations.  The rate appears to grow linearly with the amount of data being copied, however I wouldn't expect this as this calculation factors time into the equation.  Does this make sense?

Last edited by dsulli99 (2013-08-04 02:43:42)

Offline

#5 2013-07-12 18:25:17

alphaniner
Member
From: Ancapistan
Registered: 2010-07-12
Posts: 2,777

Re: Dumb question regarding dd performance tests

From my minimal understanding of HDDs and filesystems, buffering/caching has to be part of the issue. There's also location on the disk. I believe writes to the 'beginning' (outer edge of platters) of the disk tend to be faster than those to the 'end' (inner edge of platters) of the disk. Still, it doesn't seem these things could cause such a discrepancy.

If you have a spare/unused HDD you can try dd'ing to the 'raw disk' (/dev/sdX). In my experience speeds [reported by dd] tend to be pretty consistent in that case, at least once you're beyond the disk cache.

Also, you should edit the first post to s/hdparm/dd. You can also change the thread title that way.

Last edited by alphaniner (2013-07-12 18:25:58)


But whether the Constitution really be one thing, or another, this much is certain - that it has either authorized such a government as we have had, or has been powerless to prevent it. In either case, it is unfit to exist.
-Lysander Spooner

Offline

#6 2013-07-13 22:06:22

R00KIE
Forum Moderator
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 3,610

Re: Dumb question regarding dd performance tests

Try something like 'dd if=/dev/sdx of=/dev/null bs=10M count=10 iflag=direct', adjust the bs and count values to your liking and add any other options you deem necessary (skip= might be useful).

Edit:
Ooops, you are benchmarking writes, just exchange the if and of parts. I didn't dare to try because I don't have any spare disks for testing right now but it should work.

Last edited by R00KIE (2013-07-13 22:08:47)


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#7 2013-07-14 22:23:26

deepsoul
Member
From: Earth
Registered: 2012-12-23
Posts: 57
Website

Re: Dumb question regarding dd performance tests

dsulli99 wrote:

For example, the first iteration yields a data rate of 71.7 MB/s; the second iteration yields 129 MB/s, and the third 984 MB/s.  Since the data rate reported  accounts for the time (i.e. divides by /s), I would think that these would be more consistent, i.e ~71.7 MB/s for all three iterations.  The rate appears to grow linearly with the amount of data being copied,

dd does not include the time it takes to flush the data to disk in its calculation.  So it consistently outputs a higher data rate than what one gets using the wallclock time.  Also, the data rate actually falls for larger amounts of data, because more of it is written to disk while dd is running (rather then only on the final sync).  It is the amount you write with a single dd command that causes the change, not the order in which you run them.


Officer, I had to drive home - I was way too drunk to teleport!

Offline

Board footer

Powered by FluxBB