A Extremely Fast Read Speed File System

leokolln · 2009-04-30 16:31:38

Other day i was playing a game and get annoyed with the long loading times between levels, then i raise a question to myself...

There is no File System optimized for Read Speed?
I did not want one "bloated" with journaling, snapshots and all those nice all use optimizations.

I just want a plain Bare, Very simple, File System that is indisputably fast at Read Speed!
It can also be processor demanding, it is not a problem for this use case...

Someone know a File System Like that?
What is the viability(and complexity) to make one?

Sorry by the bad english!

leokolln · 2009-04-30 16:59:41

"noatime,nodiratime" didn´t count!

The question is about a File System by itself!

lucke · 2009-04-30 17:16:39

ext4 can be used without journal, should speed things up.

reiser4 with lzo compression should be very fast at reading.

You can use tmpfs if you have enough RAM.

Last edited by lucke (2009-04-30 17:17:28)

Misfit138 · 2009-04-30 18:35:01

ext2

dagle · 2009-05-01 10:46:34

Get a lot of ram and load it all into memory.

psych · 2009-05-01 11:13:35

Ext4 without journal... or RAMDISK!
Or buy a SSD (Supertalent Ultradrive) with 250 MB/s read speed

app4des · 2009-05-01 11:56:33

journal or not doesn't matter I think because you are talking about "read" performance.

I always suggest ext3 with "noatime" for everything. It is not worth using ext4 as it is still buggy,

JFS/XFS are good and stable but sometimes have strange behavior, as most programs in linux seem to be writen/optimized for use with ext3.

If you want to experiment though, I would suggest XFS with "noatime". XFS imo is the best all-around file system available for linux.

Last edited by app4des (2009-05-01 11:59:29)

R00KIE · 2009-05-01 13:58:05

This is just a guess here but how much data do you need to load?
If it is a lot of MB not even the best filesystem in the world can speed that up, to make things worse I'm guessing that you aren't loading the data from contiguous files on the HD.

To add another variable to the way hard disks perform there is the possibility of having some data compressed and this will make it depend on how fast is your cpu.

As far as I know some compression formats although very efficient can have a very big speed penalty, also the speed and configuration of you memory subsystem can impact performance as well. Also if the data is compressed in a way that you can't use multiple cores to decompress it then it doesn't matter if you have a system with a single core cpu of one with 100 cpus.

Sometimes having as much ram as you can will be the best option, sometimes having the fastest ram possible is best.

To make things worse, newer systems seem to add another degree of complexity to this as for some loads one type of memory association is better and for other loads other type of association is better (I'm thinking of ganged / unganged configuration in the bios).

In my experience, when using ext3, if you add noatime and nodiratime to the mount option you will get a bit better performance, of course different filesystems will have different performance when reading but as far as I know no filesystem is the best of all for everything, so consider your overall experience.
Is it worth having a system that can have a speed penalty when used in everyday tasks just for the sake of having one game load the levels faster? Also how long will you be playing that particular game? Is it worth all the trouble just for one game you will stop playing in a couple of months?

On the use of SSD hard drives .... I've been more tempted to get one before but .... although read speed and access times are first class it seems there is more to this kind of drives than what meets the eye, some drives seem to behave terribly when writing under certain conditions (typically when there are multiple write requests), again the overall speed of the system, not just when using one program.

After considering all the benefits and downsides if you still really want faster load times for that game then the best way to find out the faster filesystem is to try them all and time the load times.
Happy benchmarking .

Obi-Lan · 2009-05-01 20:19:46

Get a raid controller with its own cpu and battery backuped ram and put some 15krpm disks into raid0.

R00KIE · 2009-05-02 14:52:58

.... Or mount a ramdisk, copy all game files to the ramdisk and run from there .... can't get any faster than that

spinifex · 2009-05-15 02:33:31

Misfit138 wrote:

ext2

Why Ext2?

I am basically looking for read only access to my audio visual (music) files.

Just trashed a drive and have gotten myself a 500G but was about to format XFS.

Only really need fast read access. Don't care about CPU cycles either. Got a dual core.

Suggestions appreciated please.

b9anders · 2009-05-15 07:02:21

If you don't care about journaling, ext2 is the way to go. real fast.

spinifex · 2009-05-15 09:48:32

ext2 is faster than XFS?

I was under the impression that XFS was designed for audio visual stuff on large drives, to provide good read speeds. I am under the impression that it handles lots of large files well, but has problems with lots of small files.

I was also under the impression that ext2 is only really good with small files.

Not flaming, but I was led to believe that ext2 is being 'phased out' of all the nix distros in favour of ext3 and ext4?

To be honest, I dont care about journaling because all my music/video is stored on a backup drive on a different system, so the journaling aspect is irrelevent to me. I can just restore from backup.

Is ext2 faster and better at handling lots of audio visual files (32,000 / 250 gig to be exact) than XFS?

karol · 2009-05-15 10:12:40

I played Starcraft on WinXP on an old box w/ 5400 rpm pata - whether the fs was ntfs or fat didn't matter, because the game is pretty small so it loaded instantly. If you like speed then drive a Ferrari not a truck/station wagon.

If the loading scheme (game levels or whatever) can use smp than a ramdisc + quadcore could help. Loading from raid to ram and writing back "in the background" may work, but is it really worth it? The drives are *really* big now, but their speed didn't improve as much as storage capacity.

I've read that it's faster to read an mp3 and decode it (w/ sufficient hardware support) than read an uncompressed/"unencoded" music file. Maybe you should try lzo compression?

spinifex · 2009-05-15 10:34:28

bit fuzzy on the izo compession your talking about.

I was just going to set up a raid 0, but couldnt be bothered stuffing around. I have a 3Ware 4LMPL raid card sitting in a WindowZzz box running 2TB in raid 5 and a 1TB 'archive' drive that spends most of its time switched off except for backing up -but they are in my backup (wifes) WindowZzz box so I was just wanting a 'quick' copy of my music to actually use and listen to. I had the drive in question formated to ext2 but I was getting glitches playing my music (an no it wasn't my permissions) - so I assumed it may have been the ext2 not coping with the size and number of my files.

Either way, have just done up my XFS drive now so will see how it goes.

Comments?

Arch Linux

#1 2009-04-30 16:31:38

A Extremely Fast Read Speed File System

#2 2009-04-30 16:59:41

Re: A Extremely Fast Read Speed File System

#3 2009-04-30 17:16:39

Re: A Extremely Fast Read Speed File System

#4 2009-04-30 18:35:01

Re: A Extremely Fast Read Speed File System

#5 2009-05-01 10:46:34

Re: A Extremely Fast Read Speed File System

#6 2009-05-01 11:13:35

Re: A Extremely Fast Read Speed File System

#7 2009-05-01 11:56:33

Re: A Extremely Fast Read Speed File System

#8 2009-05-01 13:58:05

Re: A Extremely Fast Read Speed File System

#9 2009-05-01 20:19:46

Re: A Extremely Fast Read Speed File System

#10 2009-05-02 14:52:58

Re: A Extremely Fast Read Speed File System

#11 2009-05-15 02:33:31

Re: A Extremely Fast Read Speed File System

#12 2009-05-15 07:02:21

Re: A Extremely Fast Read Speed File System

#13 2009-05-15 09:48:32

Re: A Extremely Fast Read Speed File System

#14 2009-05-15 10:12:40

Re: A Extremely Fast Read Speed File System

#15 2009-05-15 10:34:28

Re: A Extremely Fast Read Speed File System

Board footer