Getting ready for ext4

skottish · 2009-01-18 06:54:39

I'd like to thank everyone who contributed to this thread, dhave for getting rolling, and to pointone for the wiki entry. The conversion went well (aside my weirdness http://bbs.archlinux.org/viewtopic.php?id=63201 ). I successfully converted everything except for /boot, which I didn't even try. I did run into the kernel panic, but the instructions both here and in the wiki were simple and effective.

The CK defrag script takes a very long time to run, so I can't really report on that yet. Until it hit the directories owned by root in /home (.texmf*, which I forgot about), it seemed to have worked perfectly. Although, I'm not sure how I ended up with 68667 file in /home. I do know that the .texmf stuff exists around 38000 files into it.

I used the Ubuntu 9.04 Alpha 3 for the conversion. I never thought that I'd have a use for one of their discs until today. I'm glad I did.

Heema · 2009-01-18 08:36:54

skottish wrote:

I successfully converted everything except for /boot, which I didn't even try.

is there a problem with /boot ? as i am planning also to convert to ext4 but i have just one partition /

and did you notice any speed improvements ?

grndrush · 2009-01-18 10:00:56

I just converted my / fs to ext4 (but with a separate, untouched, ext2 /boot fs). I still got a kernel panic. Solution was the same: boot into fallback and run mkinitcpio.

tetonedge · 2009-01-18 14:52:01

looked at the wiki entry and it looks good, thanks for doing that.

skottish · 2009-01-18 16:42:02

Heema wrote:

skottish wrote:
I successfully converted everything except for /boot, which I didn't even try.
is there a problem with /boot ? as i am planning also to convert to ext4 but i have just one partition /
and did you notice any speed improvements ?

I don't know if there's a problem with /boot or not. I decided to get everything else up and running first. As mentioned earlier in this thread, /boot may not benefit from the new FS because of it's small size. Can we get some more opinions on this?

As far as performance goes, I'd say that I can see some improvement. We have 50MPH winds here right now and the power is sketchy. It seems like the file system checks are quicker. Other than that, I'll need a day or two. It took hours to run the defrag stuff and I'm not done yet.

pointone · 2009-01-18 17:07:21

Where'd you find the defrag goodness? I thought she wasn't released yet?

And there is no problem with converting /boot as long as you've installed the lastest version of GRUB with ext4 patch. I didn't split /boot from my root, so I converted everything. However, with a split /boot, there really is no need to convert. None of the benefits of ext4 would apply, unless you're storing documents in /boot... :S

skottish · 2009-01-18 17:13:20

pointone wrote:

Where'd you find the defrag goodness? I thought she wasn't released yet?

Post #82 of this thread of course!

http://ck.kolivas.org/apps/defrag/defrag-0.06/defrag

It's a brute force thing that takes a few hours if you have a lot of files. Permissions are a factor, and it will do one partition at a time.

Thanks for the opinion on /boot. Mine is split. Arch is my only OS (hurray), so it made sense.

dhave · 2009-01-18 17:20:14

@pointone: Thanks for doing the wiki entry. It looks good. Nice and compact, too.

@skottish: I've played around a little with Con K.'s defrag script, but I haven't run it on more than a few isolated directories. It does seem to take a while, but, then, so do all the Windows defrag utilities I've used in the past. You say "permissions are a factor". I'm guessing this means I need to run the defrag script as root if I want it to defrag any file that it encounters, right?

Any other caveats to using the defrag script?

pointone · 2009-01-18 17:32:45

Ah! I thought you were talking of the 'official' e4defrag.

Also, I figured this would be a good place to ask:

Apparently the 'flex_bg' option is enabled by default on new ext4 partitions, but the instructions posted for converting from ext3 do not use this option. Any idea what it does? Are we missing something big here? Anyone willing to 'test' enabling 'flex_bg' on a converted ext3 partition?

From some quick Googling, all I know is that it has something to do with preventing fragmentation...

skottish · 2009-01-18 17:41:10

dhave wrote:

@skottish: I've played around a little with Con K.'s defrag script, but I haven't run it on more than a few isolated directories. It does seem to take a while, but, then, so do all the Windows defrag utilities I've used in the past. You say "permissions are a factor". I'm guessing this means I need to run the defrag script as root if I want it to defrag any file that it encounters, right?
Any other caveats to using the defrag script?

As I mentioned above, I had some files owned by root in my home directory, and the script failed when it got to them. Of course I was running as a regular user then. I'm currently doing my /backup directory as root. Since everything there is replaceable, I thought that it would be a good place to start.

I can't see anything wrong with the script. The only strangeness that I've come across so far is a couple of my Firefox extensions decided that they weren't compatible with Firefox anymore. This may (probably?) have nothing to do with the other. Using your web browser while defragging the hard drive that holds it's configuration files may not be the best idea anyway.

skottish · 2009-01-18 19:11:55

Bump to update.

Running the script as root in a directory with files owned by multiple users works fine.

--EDIT--

I now finished /{etc,lib,usr,var,opt} with no known problems. I'm calling this a success.

LTSmash · 2009-01-18 19:29:56

Guys, any idea on this?

http://bbs.archlinux.org/viewtopic.php?id=63291

My system was rendered unusable because of that... and I can't even use to Arch liveCD to recover since it doesn't yet supports ext4 :s

jelly · 2009-01-19 15:38:27

skottish wrote:

I'd like to thank everyone who contributed to this thread, dhave for getting rolling, and to pointone for the wiki entry. The conversion went well (aside my weirdness http://bbs.archlinux.org/viewtopic.php?id=63201 ). I successfully converted everything except for /boot, which I didn't even try. I did run into the kernel panic, but the instructions both here and in the wiki were simple and effective.
The CK defrag script takes a very long time to run, so I can't really report on that yet. Until it hit the directories owned by root in /home (.texmf*, which I forgot about), it seemed to have worked perfectly. Although, I'm not sure how I ended up with 68667 file in /home. I do know that the .texmf stuff exists around 38000 files into it.
I used the Ubuntu 9.04 Alpha 3 for the conversion. I never thought that I'd have a use for one of their discs until today. I'm glad I did.

I also used the ubuntu live cd, but i think it failed big time, got a nice kernel panic while mounting my / . But then i thought reinstalling is better because then you have all your files created in ext4, so i am waiting for the new iso

skottish · 2009-01-19 16:43:20

jelly wrote:

I also used the ubuntu live cd, but i think it failed big time, got a nice kernel panic while mounting my / . But then i thought reinstalling is better because then you have all your files created in ext4, so i am waiting for the new iso

A possible solution for the kernel panic is listed both in this thread as well as in the wiki article that pointone made. Did you try it before you reinstalled?

dhave · 2009-01-19 17:04:34

I was about to ask how I could easily check to see if extents was turned on for a particular file, but then I tried "lsattr filename", and I see that a cute little "e" shows up for defragged ext4 files previously converted from ext3.

EDIT: ... as toofishes had already pointed out here, it turns out.

I was wanting to make sure that my time investment in defragging my ext4 partitions has paid off.

Last edited by dhave (2009-01-19 17:13:41)

zhuqin · 2009-01-19 17:24:30

Wrong post

Last edited by zhuqin (2009-01-19 17:25:19)

lucke · 2009-01-19 18:05:51

I love the flexibility of Linux, I managed to convert the root partition from ext3 to ext4 on one of my PCs without using a live medium \o/

Last edited by lucke (2009-01-19 18:07:20)

skottish · 2009-01-19 18:18:01

lucke wrote:

I love the flexibility of Linux, I managed to convert the root partition from ext3 to ext4 on one of my PCs without using a live medium \o/

I was too scared.

tinhtruong · 2009-02-01 05:50:22

After converting all my ext3 partition to ext4, my boot time increased 2 seconds! My /boot partition is still ext2.
What a huge performance gain!

Dieter@be · 2009-02-01 09:46:10

tinhtruong wrote:

After converting all my ext3 partition to ext4, my boot time increased 2 seconds! My /boot partition is still ext2.
What a huge performance gain!

do you mean decreased?

tinhtruong · 2009-02-02 06:55:13

Dieter@be wrote:

tinhtruong wrote:
After converting all my ext3 partition to ext4, my boot time increased 2 seconds! My /boot partition is still ext2.
What a huge performance gain!
do you mean decreased?

No, my boot time with / ext3 is 22 seconds, with / ext4 is 24 seconds.
Do I have to defrag to increase the performance of ext4?

TheGrudge · 2009-02-09 11:41:42

I also get weird access times wit ext4.
I'm one of the digiKam devs and right now I'm trying to optimize speed of moving / copying files inside of digiKam. But it is incredibly slow with my ext4 media partition.
Moving only 4 images takes 10 seconds. Moving my images and database onto a ext3 or reiserfs partition makes digiKam fast again. Moving those 4 images takes like 1 second.
Do you know a filesystem option that could slow down the performance so much? I also reformatted my media partition with reiserfs (to make sure the partition itself is not broken), it is fast. Putting ext4 on it makes it incredibly slow again.
Any suggestions?

Bogart · 2009-02-13 23:24:50

About Ext4's performance:

Ext3 had support for barriers, but they were off by default. Barriers give an extra safety to the filesystem, but they are performance hogs.

Ext4 has barriers on by default.

I did some sysbench random write runs* to see the impact of barriers:

JFS: 958 Kb/s
Ext3: 898 Kb/s
Ext4: 195 Kb/s
Ext4 (without barriers): 925 Kb/s

You get the idea, right?

Now, the problem I have is that I can't boot my / filesystem with barriers disabled. I have on my /etc/fstab the following line:

UUID=96acef42-d165-45a5-a034-e6450f4b9476 / ext4 noatime,barrier=0 0 1

But my dmesg says:

EXT4-fs: barriers enabled

If I mount a partition with something like:

mount -t ext4 -o barrier=0 /dev/sda3 /mnt/sda3

dmesg says:

EXT4-fs: barriers disabled

But using barrier=0 in /etc/fstab does not seem to have any effect.

Any suggestions?

*The commands used for testing in case anyone wants to try are:

sysbench --test=fileio --file-block-size=4K --file-total-size=100M --file-test-mode=rndwr prepare
sysbench --test=fileio --file-block-size=4K --file-total-size=100M --file-test-mode=rndwr run
sysbench --test=fileio --file-block-size=4K --file-total-size=100M --file-test-mode=rndwr cleanup

TheGrudge · 2009-02-14 00:26:58

You are right, turning off barriers fixed the speed issue in digiKam... thanks for the tip...

Xi0N · 2009-02-14 02:42:20

What, more or less, is this barriers thing about? Is it safe to disable it?
Thanks

Arch Linux

#101 2009-01-18 06:54:39

Re: Getting ready for ext4

#102 2009-01-18 08:36:54

Re: Getting ready for ext4

#103 2009-01-18 10:00:56

Re: Getting ready for ext4

#104 2009-01-18 14:52:01

Re: Getting ready for ext4

#105 2009-01-18 16:42:02

Re: Getting ready for ext4

#106 2009-01-18 17:07:21

Re: Getting ready for ext4

#107 2009-01-18 17:13:20

Re: Getting ready for ext4

#108 2009-01-18 17:20:14

Re: Getting ready for ext4

#109 2009-01-18 17:32:45

Re: Getting ready for ext4

#110 2009-01-18 17:41:10

Re: Getting ready for ext4

#111 2009-01-18 19:11:55

Re: Getting ready for ext4

#112 2009-01-18 19:29:56

Re: Getting ready for ext4

#113 2009-01-19 15:38:27

Re: Getting ready for ext4

#114 2009-01-19 16:43:20

Re: Getting ready for ext4

#115 2009-01-19 17:04:34

Re: Getting ready for ext4

#116 2009-01-19 17:24:30

Re: Getting ready for ext4

#117 2009-01-19 18:05:51

Re: Getting ready for ext4

#118 2009-01-19 18:18:01

Re: Getting ready for ext4

#119 2009-02-01 05:50:22

Re: Getting ready for ext4

#120 2009-02-01 09:46:10

Re: Getting ready for ext4

#121 2009-02-02 06:55:13

Re: Getting ready for ext4

#122 2009-02-09 11:41:42

Re: Getting ready for ext4

#123 2009-02-13 23:24:50

Re: Getting ready for ext4

#124 2009-02-14 00:26:58

Re: Getting ready for ext4

#125 2009-02-14 02:42:20

Re: Getting ready for ext4

Board footer