Arch Linux

Joffer

Member

Registered: 2006-01-12

Posts: 69

500GB for storage (iso/video) - ext3 or another fs?

I'm going to add a 500GB SATA disk to my system (Q6600 4GB RAM P35, Sysdisk=Raptor 74) and I will probably mount it as one large disk under /data or something. It will be used to store large moviefiles, typical 4-20 GB and probably some iso files too. Should I go with the "standard" ext3 or would it be better to go for reiserfs/xfs/jfs? (i'm only just a bit familiar with reiserfs, never used xfs/jfs...).

lilsirecho

Veteran

Registered: 2003-10-24

Posts: 5,000

Re: 500GB for storage (iso/video) - ext3 or another fs?

ext3

---

Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

VikM

Member

Registered: 2007-11-10

Posts: 50

Re: 500GB for storage (iso/video) - ext3 or another fs?

You're going to start a war with this kind o questions...;)
If you have enough time, test yourself, after all your subjective perception is what matters. Anyway take a look at:
http://linuxgazette.net/122/TWDT.html#piszcz
http://www.debian-administration.org/articles/388
http://www.linuxquestions.org/questions … tem-42929/
http://www.novell.com/documentation/sus … pas02.html
http://www.ffnn.nl/pages/articles/linux … arking.php

My favorite is jfs, try a google search for "jfs deadline scheduler".

test1000

Member

Registered: 2005-04-03

Posts: 834

Re: 500GB for storage (iso/video) - ext3 or another fs?

xfs is made for this kind of scenario and is the most stable fs after ext3 iv'e ever tried. However ext3 also got other advantages like more compatibility with standard system tools. I suppose it depends on what kind of performance you NEED as xfs will give you much better performance in that area than ext3.

<quote>My favorite is jfs, try a google search for "jfs deadline scheduler".</quote>

i think jfs is good mainly when you want to save cpu.. as it barely taxes it.

---

KISS = "It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience." - Albert Einstein

onearm

Member

From: Anywhere but here

Registered: 2006-07-06

Posts: 359

Website

Re: 500GB for storage (iso/video) - ext3 or another fs?

Try first one between jfs and xfs and see if there are problems/glitches (I experienced some with xfs); if yes, you can always go back to ext3

---

To get something done, a committee should consist of no more than three persons, two of them absent.
--
My Github

Joffer

Member

Registered: 2006-01-12

Posts: 69

Re: 500GB for storage (iso/video) - ext3 or another fs?

Have no intentions in starting a fs-war

test1000: Speed would be nice, I hate waiting more than I have to.

Anyway, I tested now, with ext3 on the big disk (with data=journal mode and  dir_index tuning), and reading a 4GB iso from my raptor and storing it on the 500GB took about 5 minutes which translates to under 15MB/s in transfeer speed. Does that seem right?

Update: I did a new test now. File was about 4,5GB

From data (500gb) to raptor:

# time cp /data/iso/testfile.iso ~/

real    1m33.246s
user    0m0.153s
sys     0m12.499s

From Raptor to Data:

# time cp ~/testfile.iso /data/iso/testfile2.iso

real    3m25.831s
user    0m0.243s
sys     0m11.679s

Seems there are some issues with writing on the data disk? It's like 48MB/s from data to raptor, but only 21MB/s the other way around.

/dev/sda is raptor and /dev/sdb is 500GB (wdc sata2):

[joffer@armor ~]$ sudo hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   8386 MB in  2.00 seconds = 4198.38 MB/sec
 Timing buffered disk reads:  188 MB in  3.01 seconds =  62.51 MB/sec
[joffer@armor ~]$ sudo hdparm -tT /dev/sdb

/dev/sdb:
 Timing cached reads:   8198 MB in  2.00 seconds = 4104.37 MB/sec
 Timing buffered disk reads:  254 MB in  3.02 seconds =  84.08 MB/sec

It looks like the WDC is faster than my 74GB raptor also. Hmmm.. Oh well, 60MB/s+ is good anyway.

More info:

[joffer@armor ~]$ sudo hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
        Model Number:       WDC WD740GD-00FLA2                      
        Serial Number:      WD-WMAKE1815086
        Firmware Revision:  31.08F31
Standards:
        Supported: 6 5 4 
        Likely used: 6
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  145226112
        LBA48  user addressable sectors:  145226112
        device size with M = 1024*1024:       70911 MBytes
        device size with M = 1000*1000:       74355 MBytes (74 GB)
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, with device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Recommended acoustic management value: 128, current value: 254
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4 
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Release interrupt
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
           *    READ/WRITE_DMA_QUEUED
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    Automatic Acoustic Management feature set
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    SATA-I signaling speed (1.5Gb/s)
           *    Host-initiated interface power management
           *    SMART Command Transport (SCT) feature set
           *    SCT Long Sector Access (AC1)
           *    SCT LBA Segment Access (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
Security: 
                supported
        not     enabled
        not     locked
                frozen
        not     expired: security count
        not     supported: enhanced erase
Checksum: correct

[joffer@armor ~]$ sudo hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
        Model Number:       WDC WD5000AAKS-00TMA0                   
        Serial Number:      WD-WMAPW1154563
        Firmware Revision:  12.01C01
Standards:
        Supported: 7 6 5 4 
        Likely used: 7
Configuration:
        Logical         max     current
        cylinders       16383   16383
        heads           16      16
        sectors/track   63      63
        --
        CHS current addressable sectors:   16514064
        LBA    user addressable sectors:  268435455
        LBA48  user addressable sectors:  976773168
        device size with M = 1024*1024:      476940 MBytes
        device size with M = 1000*1000:      500107 MBytes (500 GB)
Capabilities:
        LBA, IORDY(can be disabled)
        Queue depth: 32
        Standby timer values: spec'd by Standard, with device specific minimum
        R/W multiple sector transfer: Max = 16  Current = 16
        Recommended acoustic management value: 128, current value: 254
        DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
             Cycle time: min=120ns recommended=120ns
        PIO: pio0 pio1 pio2 pio3 pio4 
             Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
        Enabled Supported:
           *    SMART feature set
                Security Mode feature set
           *    Power Management feature set
           *    Write cache
           *    Look-ahead
           *    Host Protected Area feature set
           *    WRITE_BUFFER command
           *    READ_BUFFER command
           *    NOP cmd
           *    DOWNLOAD_MICROCODE
                Power-Up In Standby feature set
           *    SET_FEATURES required to spinup after power up
                SET_MAX security extension
           *    Automatic Acoustic Management feature set
           *    48-bit Address feature set
           *    Device Configuration Overlay feature set
           *    Mandatory FLUSH_CACHE
           *    FLUSH_CACHE_EXT
           *    SMART error logging
           *    SMART self-test
           *    General Purpose Logging feature set
           *    64-bit World wide name
           *    Segmented DOWNLOAD_MICROCODE
           *    SATA-I signaling speed (1.5Gb/s)
           *    SATA-II signaling speed (3.0Gb/s)
           *    Native Command Queueing (NCQ)
           *    Host-initiated interface power management
           *    Phy event counters
                DMA Setup Auto-Activate optimization
           *    Software settings preservation
           *    SMART Command Transport (SCT) feature set
           *    SCT Long Sector Access (AC1)
           *    SCT LBA Segment Access (AC2)
           *    SCT Error Recovery Control (AC3)
           *    SCT Features Control (AC4)
           *    SCT Data Tables (AC5)
                unknown 206[12]
Security: 
        Master password revision code = 65534
                supported
        not     enabled
        not     locked
                frozen
        not     expired: security count
        not     supported: enhanced erase
        122min for SECURITY ERASE UNIT. 
Checksum: correct

Last edited by Joffer (2007-11-26 17:39:38)

byte

Member

From: Düsseldorf (DE)

Registered: 2006-05-01

Posts: 2,046

Re: 500GB for storage (iso/video) - ext3 or another fs?

Why did you use data=journal when you want speed? That's by far the slowest mode for sequential write transfers, everything is written twice.

Last edited by byte (2007-11-27 16:48:31)

---

1000

Joffer

Member

Registered: 2006-01-12

Posts: 69

Re: 500GB for storage (iso/video) - ext3 or another fs?

Because of these articles mostly, and a few like it:
http://www.ibm.com/developerworks/linux … fs8.html#4
http://bbs.archlinux.org/viewtopic.php?id=40334

The results were astounding. data=journal mode allowed the 16-meg-file to be read from 9 to over 13 times faster than other ext3 modes, ReiserFS, and even ext2 (which has no journaling overhead):
Written-to-filesystem     16-meg-read-time (seconds)
ext2    78
ReiserFS    67
ext3 data=ordered     93
ext3 data=writeback     74
ext3 data=journal     7

Andrew repeated this test, but tried to read a 16Mb file from the test filesystem (rather than a different filesystem), and he got identical results. So, what does this mean? Somehow, ext3's data=journal mode is incredibly well-suited to situations where data needs to be read from and written to disk at the same time. Therefore, ext3's data=journal mode, which was assumed to be the slowest of all ext3 modes in nearly all conditions, actually turns out to have a major performance advantage in busy environments where interactive IO performance needs to be maximized. Maybe data=journal mode isn't so sluggish after all!

Reading it a bit closer makes me believe it's better for my OS disk which doesn't change that much where most I/O is reading..

Update: I changed back to default ordered mode and it looks like its back to nice speeds around 1min30s for 4GB+ (calculated). Will test with the large file tomorrow

Update follow-up: Same file, from raptor to datadisk:

real    1m37.606s
user    0m0.213s
sys     0m12.163s

Last edited by Joffer (2007-11-28 18:45:29)

kensai

Member

From: Puerto Rico

Registered: 2005-06-03

Posts: 2,484

Website

Re: 500GB for storage (iso/video) - ext3 or another fs?

<quote>My favorite is jfs, try a google search for "jfs deadline scheduler".</quote>

i think jfs is good mainly when you want to save cpu.. as it barely taxes it.

All hail the king of file systems JFS. You want to be kind to your CPU and warrant it a better longer life?, jfs + deadline scheduler.

You want to use deadline scheduler? Easy, add "elevator=deadline" to the end of the kernel line of grub/menu.lst, without the quotations of course.

DISCLAIMER: I'm hugely biased towards JFS.

---

Follow me in: Identi.ca, Twitter, Google+

byte

Member

From: Düsseldorf (DE)

Registered: 2006-05-01

Posts: 2,046

Re: 500GB for storage (iso/video) - ext3 or another fs?

I'm using JFS ever since I started with Linux (only three years or so). I don't have practical experience with others, except for Ext3, but so far I find it very unspectacular... it's fast enough and doesn't have dozens of tools and options to tweak -> KISS.

---

1000

wuischke

Member

From: Suisse Romande

Registered: 2007-01-06

Posts: 630

Re: 500GB for storage (iso/video) - ext3 or another fs?

You want to use deadline scheduler? Easy, add "elevator=deadline" to the end of the kernel line of grub/menu.lst, without the quotations of course.

Thanks a lot for this tip, I didn't know about this trick.

VikM

Member

Registered: 2007-11-10

Posts: 50

Re: 500GB for storage (iso/video) - ext3 or another fs?

You can enable deadline scheduler only for the device you want:

echo deadline > /sys/block/sda/queue/scheduler