ext3 with dir_index - holy cow!

Gullible Jones · 2005-11-14 02:57:34

And to think I once thought this was all hype...

You see, I was getting grief from pacman on XFS. Not sure exactly what was going on there, but anyway... Just for kicks, I reinstalled on ext3 with directory indexing. You know, mke2fs -j -O dir_index, for both root and /home partitions.

It's quite freaky actually. Everything is visibly faster than on ReiserFS. :shock:

Why the heck isn't dir_index enabled by default?!

deficite · 2005-11-14 03:46:10

Yes, dir_index is very nice indeed. I have no clue why it isn't enabled by default, but if I remember right they're considering making the new installer use dir_index by default when formatting ext3 partitions.

Cam · 2005-11-14 04:02:48

Somebosy posted a short how-to a while ago on speeding up ext3 filesystems. I'm not 100% sure whether it was here or at LQ.org but I think it might have been here. Anyway, I did it and yeah, everything is running considerably faster. I can't compare it to Reiser though, never used anything but ext2 and more recently, ext3.

Cam · 2005-11-14 04:04:15

and the search feature indeed works

Gullible Jones · 2005-11-14 10:28:38

I think that's where I first found out about this... Dang. Thanks, Codergeek!

phrakture · 2005-11-14 16:00:32

Hmmm I've never noticed this before - are there any issues with doing this on a stable system? I don't want to screw things up by messing with the filesystem.

codergeek42 · 2005-11-16 23:33:38

Gullible Jones wrote:

I think that's where I first found out about this... Dang. Thanks, Codergeek!

Happy to be of help.

mmccaskill · 2005-11-17 00:24:47

So overall, is ext3 dir_index better than reiserfs? jfs?

Gullible Jones · 2005-11-17 01:20:26

I don't think it's quite as good as Reiser for small files - not quite sure there, there's not really much way for me to tell - but the read speeds seem better than Reiser, XFS, or JFS.

There is a disadvantage, that it is not as storage efficient as ReiserFS, since there's no tail packing. On current hardware, this is not really a problem.

mmccaskill · 2005-11-17 05:33:23

Well considering that I have a AMD64 3200 (still running 32bit until 64bit stable) with WD Raptor 74GB I'm not so sure I would notice much of a difference anyway.

nuopus · 2005-11-22 20:47:23

Is it possible to get this indexing feature on an existing root file system?

lucke · 2005-11-22 20:57:26

As the codergeek's howto points out, just boot into some live cd and execute

tune2fs -O dir_index /dev/hdXY
e2fsck -D /dev/hdXY

codergeek42 · 2005-11-23 00:01:57

Gullible Jones wrote:

There is a disadvantage, that it is not as storage efficient as ReiserFS, since there's no tail packing. On current hardware, this is not really a problem.

One must balance these things. The more storage efficient a filesystem becomes, the higher probability that files may get fragmented heavily. If you're really that worried about storage efficiency though, you could always create an ext2/ext3 FS with KJiB block sizes and 1 inode/block. That almost certainly will cut your performance though. *shrug*

shadowhand · 2005-11-23 00:36:24

lucke wrote:

As the codergeek's howto points out, just boot into some live cd and execute
tune2fs -O dir_index /dev/hdXY
e2fsck -D /dev/hdXY

Or just do it on your running system and reboot. 8)

codergeek42 · 2005-11-23 00:48:22

shadowhand wrote:

lucke wrote:
As the codergeek's howto points out, just boot into some live cd and execute
tune2fs -O dir_index /dev/hdXY
e2fsck -D /dev/hdXY
Or just do it on your running system and reboot. 8)

Erm. fsck'ing a live filesystem can cause massive data corruption! :!:

shadowhand · 2005-11-23 01:06:18

codergeek42 wrote:

shadowhand wrote:
lucke wrote:
As the codergeek's howto points out, just boot into some live cd and execute
tune2fs -O dir_index /dev/hdXY
e2fsck -D /dev/hdXY
Or just do it on your running system and reboot. 8)
Erm. fsck'ing a live filesystem can cause massive data corruption! :!:

Oh hey... I didn't notice the e2fsck line. Shutdown after using tune2fs with

shutdown -F -r

to force a fsck on reboot.

codergeek42 · 2005-11-23 01:24:03

That's the right way to do it.

Gullible Jones · 2005-11-23 01:42:52

You could also use magic sysrq to remount all filesystems read-only.

BTW, Codergeek, is it necessary to use '-O has_journal' when using tune2fs to enable full journalling?

codergeek42 · 2005-11-23 03:41:48

Gullible Jones wrote:

You could also use magic sysrq to remount all filesystems read-only.

I just boot to a LiveCD. It's simpler (imho).

BTW, Codergeek, is it necessary to use '-O has_journal' when using tune2fs to enable full journalling?

Not necessarily. I have that in there to add a journal to ext2 filesystems.

lilsirecho · 2005-11-23 05:56:39

Entered the dir_index code in two kernels, 2.6.13 and 2.6.14.

In each case, used the shutdown -F -r now ... procedure.

In each case, I had to reboot twice in order to complete the fsck of the partition.

Just a note on how it went for my system.

The system is up and running on either kernel.

nightfrost · 2005-11-23 11:03:47

Cam wrote:

and the search feature indeed works

I was actually looking for this one over at the wiki a while back, with no luck. How about moving it there? I could do some copy & pasting myself, if you guys feel like you got better things to do. What do you say?

lucke · 2005-11-23 11:16:36

Sure, it belongs to the WIKI.

Sounds disturbing...

Gullible Jones · 2005-11-23 11:43:08

Not really.

BTW, I tried full journalled mode... I don't know what's going on with your HDDs, Codergeek, but on this end full journalling causes write times to double as one might expect.

codergeek42 · 2005-11-23 16:16:16

@Gullible Jones: It does not seem to cause any significant reduction in terms of throughput or interactivity though.

Gullible Jones · 2005-11-23 17:32:29

No, but it's annoying as hell when you're copying stuff around. I'll stick with metadata journalling, thank you. May be slightly riskier if something bad happens, but my data is not incredibly important.

Arch Linux

#1 2005-11-14 02:57:34

ext3 with dir_index - holy cow!

#2 2005-11-14 03:46:10

Re: ext3 with dir_index - holy cow!

#3 2005-11-14 04:02:48

Re: ext3 with dir_index - holy cow!

#4 2005-11-14 04:04:15

Re: ext3 with dir_index - holy cow!

#5 2005-11-14 10:28:38

Re: ext3 with dir_index - holy cow!

#6 2005-11-14 16:00:32

Re: ext3 with dir_index - holy cow!

#7 2005-11-16 23:33:38

Re: ext3 with dir_index - holy cow!

#8 2005-11-17 00:24:47

Re: ext3 with dir_index - holy cow!

#9 2005-11-17 01:20:26

Re: ext3 with dir_index - holy cow!

#10 2005-11-17 05:33:23

Re: ext3 with dir_index - holy cow!

#11 2005-11-22 20:47:23

Re: ext3 with dir_index - holy cow!

#12 2005-11-22 20:57:26

Re: ext3 with dir_index - holy cow!

#13 2005-11-23 00:01:57

Re: ext3 with dir_index - holy cow!

#14 2005-11-23 00:36:24

Re: ext3 with dir_index - holy cow!

#15 2005-11-23 00:48:22

Re: ext3 with dir_index - holy cow!

#16 2005-11-23 01:06:18

Re: ext3 with dir_index - holy cow!

#17 2005-11-23 01:24:03

Re: ext3 with dir_index - holy cow!

#18 2005-11-23 01:42:52

Re: ext3 with dir_index - holy cow!

#19 2005-11-23 03:41:48

Re: ext3 with dir_index - holy cow!

#20 2005-11-23 05:56:39

Re: ext3 with dir_index - holy cow!

#21 2005-11-23 11:03:47

Re: ext3 with dir_index - holy cow!

#22 2005-11-23 11:16:36

Re: ext3 with dir_index - holy cow!

#23 2005-11-23 11:43:08

Re: ext3 with dir_index - holy cow!

#24 2005-11-23 16:16:16

Re: ext3 with dir_index - holy cow!

#25 2005-11-23 17:32:29

Re: ext3 with dir_index - holy cow!

Board footer