ext4 and fragmentation (not a question, just a compliment)

grndrush · 2009-04-02 05:57:54

I just wanted to pass on a bit of praise for a fs that's seems to get a lot of negative comments. And I hope this is the proper place for the comment (I know it'll be moved, otherwise).

I reinstalled* my x86_64 installation on 2008/12/21. I use LVM with 3 partitions: /, /home, and /var/cache/pacman (a tweak that seems to work well for me). / is ext4; the others, ReiserFS (moving / from ReiserFS to ext4 was the PURPOSE of the "reinstall", actually). / is 8GB; 48% used.

I originally installed LXDE, and, about a month ago, KDE4 (official version, not KDE4MOD).

I tend to install (and sometimes uninstall) a lot of packages - many just to see how they work (hence a good number of uninstalls). IOW, I bang on / pretty darned hard.

I rebooted a few days ago (3 months after the original install) and Linux decided to do it's periodic fsck on /. When it finished, it reported my level of fragmentation: *** 0.2% ***. After 3 months, 2 WM's, and a bunch of installs/uninstalls.

I was absolutely speechless. Astonished. Yes, a hard reset with an ext4 partition is still a risky proposition, it's not the fastest fs I've run, and (on my system, anyway) it uses 25% more disk space than ReiserFS (but is also quite "easy" on my single-core processor - an XP3500+, S939 ).

* The reason I can say with some authority that ext4 takes 25% more space than ext4, is that I didn't actually do a reinstall. I backed up / (which at that time was ReiserFS), reformatted the same partition as ext4, and copied the backup to it (which worked without a single glitch, much to my surprise, actually).

But ext4 appears to have taken the "disk fragmentation" issue from an art, to a SCIENCE. This, to me, is slicker than bat guano.

FYI.

Last edited by grndrush (2009-04-02 05:58:45)

broch · 2009-04-02 12:34:22

I believe that fragmentation does not matter anymore

But ext4 appears to have taken the "disk fragmentation" issue from an art, to a SCIENCE. This, to me, is slicker than bat guano.

I don't think that this is something special however:
after three years
#sudo xfs_db -c frag -r /dev/sda4 (home)
actual 110463, ideal 109161, fragmentation factor 1.18%
#sudo xfs_db -c frag -r /dev/sda2 (/)
actual 441705, ideal 439059, fragmentation factor 0.60%

I could cut down fragmentation further by increasing buffers (from my experience this would keep fragmentation of /home below 0.5% for years), but I don't find fragmentation as a performance factor. So ext4 is not doing anything extraordinary.

Last edited by broch (2009-04-02 12:43:13)

grndrush · 2009-04-02 16:45:34

I'm comparing ext4 to *ext3* performance, not xfs (sorry if that was somehow unclear). ext3 isn't BAD in this regard, but 3-7% was a more typical figure for me after a few months.

Doesn't xfs have an on-line defrag? I know one is supposedly coming for ext4 in 2.6.30, but I'm getting this performance WITHOUT an on-line defragger. And again, I'm HARD on /.

All my BIG files are on /home. A typical, clean Linux install (on my machine) has nearly 100K files. 4G/100K = 40K average file size. Further, it's typically all the 100-1K byte files in /etc one is constantly (or at least regularly) updating. I need something that's speedy with SMALL files, NOT big ones.

I was simply trying to pass along an observation on a fairly new product, HOPEFULLY to benefit others. I wasn't asking for ANYTHING. I converted / to ext4 the day the ISO containing it CAME OUT, and thought my experience (RELATIVE TO ext3) would be helpful.

Can't win for losing at this site 95% of the time...

Last edited by grndrush (2009-04-02 16:46:51)

broch · 2009-04-02 18:05:58

yes, xfs has on-line defrag, but I don't use it on my laptop. With 1.2% fragmentation, I don't see a reason to use it. Nice feature is allocsize. Specifying allocsize can effectively prevent fragmentation even if you have a lot of file activity.

I think that ext4 is optimized in such way that even 20% fragmentation should not affect fs performance.

Tera · 2009-04-02 20:30:18

broch wrote:

I think that ext4 is optimized in such way that even 20% fragmentation should not affect fs performance.

A filesystem, doesn't matter how good or bad it is, cannot affect the hard-drive's seek time when it cannot read the file(s) in a row and has to move it's read head from place A to place Y due fragmentation. That's just a mechanical fact.

broch · 2009-04-03 02:25:51

Tera wrote:

broch wrote:
I think that ext4 is optimized in such way that even 20% fragmentation should not affect fs performance.
A filesystem, doesn't matter how good or bad it is, cannot affect the hard-drive's seek time when it cannot read the file(s) in a row and has to move it's read head from place A to place Y due fragmentation. That's just a mechanical fact.

That is great news indeed (discovery that fragmentation will affect I/O) What I said is that ext4 is optimized is a such a way that even 20% should not affect ext4 performance.

Here is an older paper explaining why:
http://www.kernel.org/doc/ols/2006/ols2 … 93-208.pdf

grndrush · 2009-04-08 08:51:13

Just to make my original point...

ext3 just fsck'ed my * /var/cache/pacman/ * partition, which, since I *never* clean it out, should be 0.0% fragmented. It was 6.6%. 'Nuff said.

Arch Linux

#1 2009-04-02 05:57:54

ext4 and fragmentation (not a question, just a compliment)

#2 2009-04-02 12:34:22

Re: ext4 and fragmentation (not a question, just a compliment)

#3 2009-04-02 16:45:34

Re: ext4 and fragmentation (not a question, just a compliment)

#4 2009-04-02 18:05:58

Re: ext4 and fragmentation (not a question, just a compliment)

#5 2009-04-02 20:30:18

Re: ext4 and fragmentation (not a question, just a compliment)

#6 2009-04-03 02:25:51

Re: ext4 and fragmentation (not a question, just a compliment)

#7 2009-04-08 08:51:13

Re: ext4 and fragmentation (not a question, just a compliment)

Board footer