You are not logged in.

#1 2008-12-24 13:14:28

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Getting ready for ext4

EDIT (18jan09): Since this thread was begun, pointone has added a wiki section on converting ext3 partitions to ext4. Find it here.

---------------------

I guess this means I don't have much of a life, but I'm getting pretty excited about ext4. I haven't tried a new filesystem in ages.

There are good pages out there on ext4, such as this how-to and this overview. Still, I'm a little wobbly on just how the conversion/migration is going to go.

I won't be creating any ext4 partitions from scratch right away, but I'll be converting two partitions from ext3 to ext4. It looks as if this is all I'll have to do:

(1) Use kernel 2.6.28 or later.
(2) Make sure ext4 is enabled in my kernel config.
(3) In /etc/fstab, change the file type of my two ext3 partitions to ext4.

Is this right? Is there anything else necessary?

Now, here's an additional question:

If I want to apply extents to my existing ext3 partitions as I convert to ext4, do I have to do anything? I realize that in doing so, I will no long be able to access those formerly ext3 partitions as ext3. Or at least that's what I understand.

Last edited by dhave (2009-01-18 22:44:45)


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#2 2008-12-24 15:45:50

zukka
Member
From: Bologna Italy
Registered: 2006-06-27
Posts: 26

Re: Getting ready for ext4

I'm already using it (ext4dev) in a VirtualBox vm.
To do so I had to do the following things:
-tune2fs -E test_fs <partition> (This should not be needed once ext4dev becomes ext4)
-tune2fs -O extents <partition> (I think to enable extents support on an existing ext3 filesystem)
-mount it with -o extents .

Be aware that grub still can't boot from an ext4 partition !

I'm really lookiing forward for ext4 too tongue

Offline

#3 2008-12-24 16:07:23

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

@zukka: Thanks for the info. It sounds like it'll be pretty painless, assuming ext4 is stable and relatively bug-free. It should be, from what I've read.

I'm keeping my boot partition on ext2, where it's always been. I've had my other non-swap partitions on reiserfs, but when recently moved to a new hard disk, I moved the data that had been on reiserfs to ext3. I did this in anticipation of ext4.

I know I could go ahead and update to ext4dev or use ext4 with a release candidate of kernel 2.6.28, but I think I'll be patient a little while longer.


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#4 2008-12-25 21:12:21

Xi0N
Member
From: Bilbao - Spain
Registered: 2007-11-29
Posts: 832
Website

Re: Getting ready for ext4

Im a bit afraid about the new ext4 filesystem, in the same time as excited because the new filesystem thing.... the last new filesystem i tryed was ntfs!!!! (Years ago)

What i am planning to do, is wait until a new arch install cd is released with the ext4 capabilityes. Back all my data up, and wipe my disk completely.
Then, reformat all with ext4, and copy all my data back to where it should be in my disks smile

This is how i would feel the safest to do the migration smile

Last edited by Xi0N (2008-12-25 21:16:09)

Offline

#5 2008-12-25 21:46:12

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

Xi0N wrote:

Back all my data up, and wipe my disk completely.
Then, reformat all with ext4, and copy all my data back to where it should be in my disks smile

This is how i would feel the safest to do the migration smile

I'm going to go as soon as 2.6.28 makes it to /core, but you can be sure that I'll back up both my ext3 partitions before converting.

BTW, I see on www.kernel.org that 2.6.28 is officially released as of 24 December.


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#6 2008-12-25 22:41:30

jelly
Administrator
From: /dev/null
Registered: 2008-06-10
Posts: 714

Re: Getting ready for ext4

It's released today! I looking forward in upgrading to ext4, i should be a lot faster than ext3.  How long will it take till it will be in core, that takes a ~month ?

Offline

#7 2008-12-25 22:45:51

wonder
Developer
From: Bucharest, Romania
Registered: 2006-07-05
Posts: 5,941
Website

Re: Getting ready for ext4

jelly wrote:

It's released today! I looking forward in upgrading to ext4, i should be a lot faster than ext3.  How long will it take till it will be in core, that takes a ~month ?

I think it is a good estimate considering the period in which we are now - holidays


Give what you have. To someone, it may be better than you dare to think.

Offline

#8 2008-12-25 23:20:16

schuay
Package Maintainer (PM)
From: Austria
Registered: 2008-08-19
Posts: 564

Re: Getting ready for ext4

from http://kernelnewbies.org/Ext4:

How to use Ext4

This is the first stable version of Ext4, so even if the whole development and release of this filesystems has been slowed down and delayed a lot to guarantee the same level of stability that you'd expect from the current Ext3 implementation, the usual rules of any ".0" software apply.

One very important thing to keep in mind is that there is NOT Ext4 GRUB support. Well, that wasn't exactly true: There is grub support, but the grub versions used by your current distro don't support it. There's support in the GRUB2 development branch, but only from this commit and ahead. There're available grub2 packages in Ubuntu and debian-derived distros as the grub-pc package. In the 0.9x branch, there's not official support, but there's a Google SoC project that developed support for it, and Google finds patches. So choose yourself. The next release of distros based in Linux 2.6.28 will probably have support in one way or another. The safe option is to keep your /boot directory in a partition formatted with Ext3.

You also need an updated e2fsprogrs tool, of course, the latest stable version -1.41.3- is recommended.

Switching to Ext4 it's very easy. There're three different ways you can use to switch:

Creating a new Ext4 filesystem from the scratch

    * The easiest one, recommended for new installations. Just update your e2fsprogs package to Ext4, and create the filesystem with mkfs.ext4.

Migrate existing Ext3 filesystems to Ext4

You need to use the tune2fs and fsck tools in the filesystem, and that filesystem needs to be unmounted. Run:

    *

      tune2fs -O extents,uninit_bg,dir_index /dev/yourfilesystem

After running this command you MUST run fsck. If you don't do it, Ext4 WILL NOT MOUNT your filesystem. This fsck run is needed to return the filesystem to a consistent state. It WILL tell you that it finds checksum errors in the group descriptors - it's expected, and it's exactly what it needs to be rebuilt to be able to mount it as Ext4, so don't get surprised by them. Since each time it finds one of those errors it asks you what to do, always say YES. If you don't want to be asked, add the "-p" parameter to the fsck command, it means "automatic repair":

    *

      fsck -pf /dev/yourfilesystem

There's another thing that must be mentioned. All your existing files will continue using the old indirect mapping to map all the blocks of data. The online defrag tool will be able to migrate each one of those files to a extent format (using a ioctl that tells the filesystem to rewrite the file with the extent format; you can use it safely while you're using the filesystem normally)

Mount an existing Ext3 filesystem with Ext4 without changing the format

You can mount an existing Ext3 filesystem with Ext4 but without using features that change the disk format. This means you will be able to mount your filesystem with Ext3 again. You can mount an existing Ext3 filesystem with "mount -t ext4 /dev/yourpartition /mnt". Doing this without having done the conversion process described in the previous point will force Ext4 to not use the features that change the disk format, such as extents, it will use only the features that don't change the file format, such as mballoc or delayed allocation. You'll be able to mount your filesystem as Ext3 again. But obviously you'll be losing the advanges of the Ext4 features that don't get used...

by the way, they also have an awesome readable changelog at http://kernelnewbies.org/LinuxChanges

Offline

#9 2008-12-25 23:22:39

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

Thanks for that link, schuay. Very helpful info.


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#10 2008-12-26 09:37:23

shazeal
Member
From: New Zealand
Registered: 2007-06-05
Posts: 341

Re: Getting ready for ext4

My advice is to start from ext4 from scratch, otherwise there will be no extents for any of the existing files until they are re-written to the disk. If you do start with ext3 you will need to move all the files off the disk then back again or copy/delete/copy back to actually reap the benefits of extents (for data anyway). The system files are a bit easier you can just reinstall all your packages, still far easier just to start with ext4.

Offline

#11 2008-12-26 10:32:35

jelly
Administrator
From: /dev/null
Registered: 2008-06-10
Posts: 714

Re: Getting ready for ext4

Converting ext3 to ext4 is not recommended? Because hmm maybe i will reinstall my desktop but maybe first buy i bigger harddisk tongue My /home is still on one disk and i have 2 disks so that can be improved

BTW the holidays are the time that i work on my system smile

Offline

#12 2008-12-26 11:04:46

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

shazeal wrote:

My advice is to start from ext4 from scratch, otherwise there will be no extents for any of the existing files until they are re-written to the disk. If you do start with ext3 you will need to move all the files off the disk then back again or copy/delete/copy back to actually reap the benefits of extents (for data anyway). The system files are a bit easier you can just reinstall all your packages, still far easier just to start with ext4.

O.K., let me get this straight. I reread the kernelnewbies info, and it says this:

Migrate existing Ext3 filesystems to Ext4

You need to use the tune2fs and fsck tools in the filesystem, and that filesystem needs to be unmounted. Run:

      tune2fs -O extents,uninit_bg,dir_index /dev/yourfilesystem

After running this command you MUST run fsck. [...]

There's another thing that must be mentioned. All your existing files will continue using the old indirect mapping to map all the blocks of data. The online defrag tool will be able to migrate each one of those files to a extent format (using a ioctl that tells the filesystem to rewrite the file with the extent format; you can use it safely while you're using the filesystem normally) italics mine

So, if I convert rather than start from scratch, I'll have to use the new ext4 defrag tool to activate extents? Didn't I read that the defrag tool is not released yet?


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#13 2008-12-26 12:04:30

jelly
Administrator
From: /dev/null
Registered: 2008-06-10
Posts: 714

Re: Getting ready for ext4

I am going to reinstall Arch for ext4, i'm still using ext2 neutral. It's time for a upgrade & i can partion my arch install better, i have deleted my dualboot so now there is more space for Arch:D. Can't wait till the new kernel hit's the Arch ftp install cd.

For my laptop i am going to convert ext3 to ext4, it cost too much work to reinstall arch + configure my dualboot

Last edited by jelly (2008-12-26 12:05:43)

Offline

#14 2008-12-27 01:15:35

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

I see that 2.6.28 is in /testing now (26 Dec 08).


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#15 2008-12-27 01:30:06

smartboyathome
Member
From: $HOME
Registered: 2007-12-23
Posts: 334
Website

Re: Getting ready for ext4

I compiled my own kernel with the arch stock config and ext4 enabled and built into the kernel. Then, I copied my current installation with cp -a over to the other drive (excluding files like /media, /proc, and /sys, of course). I have Grub2 installed and working on it, so it won't be a problem when I delete my old setup and transfer over to the new one. I changed the type in fstab from ext3 to ext4 as well as the UUID, and of course added it to grub.cfg. Anyway, I got it installed, and try to boot up to the 2.6.28 kernel on the EXT4 partition, and init seems to detect its root partition as EXT3, and when it sees it has extents goes into a kernel panic. How would I correct this?

Offline

#16 2008-12-27 03:01:55

zachstruck
Member
Registered: 2008-12-18
Posts: 1

Re: Getting ready for ext4

smartboyathome:
I had the same problem when migrating to ext4.  Using grub2, what I had to do was add the parameter rootfstype=ext4 to the kernel line in grub.cfg and boot from my fallback kernel kernel26-fallback.img.

Offline

#17 2008-12-27 03:36:30

smartboyathome
Member
From: $HOME
Registered: 2007-12-23
Posts: 334
Website

Re: Getting ready for ext4

Thanks, and now it gets more complicated. I have tried doing as you say (booting with the rootfstype=ext4 in my kernel line in grub.cfg), and now I get that my partition is being detected like a normal ext3 drive by udev. Here's the output right before the rescue shell:

/dev/sda1: UUID="8dced6b0-2e15-4239-a9c3-88e457836e4d" SEC_TYPE="ext2" TYPE="ext3"
/dev/sda3: UUID="257a7749-e34b-4da8-a8e5-0124d2ddc0fa" SEC_TYPE="ext2" TYPE="ext3" 
/dev/sda5: TYPE="swap" UUID="0d8e1f10-2290-46c6-ae93-2ebc827e5e4a"

Here is the output of blkid after I boot into my still working arch install on ext3 with the 2.6.28 kernel:

/dev/sda1: UUID="8dced6b0-2e15-4239-a9c3-88e457836e4d" TYPE="ext4" 
/dev/sda3: UUID="257a7749-e34b-4da8-a8e5-0124d2ddc0fa" SEC_TYPE="ext2" TYPE="ext3" 
/dev/sda5: TYPE="swap" UUID="0d8e1f10-2290-46c6-ae93-2ebc827e5e4a"

How could it be detected differently on two installs which are almost identical? :S

Offline

#18 2008-12-27 11:54:57

jelly
Administrator
From: /dev/null
Registered: 2008-06-10
Posts: 714

Re: Getting ready for ext4

Well now i am interested in how the new Arch install cd is going to be, is Arch going to switch to grub2 for support of ext4 or still with the normal grub? Btw. isn't grub2 in testing?

Offline

#19 2008-12-27 12:05:08

wonder
Developer
From: Bucharest, Romania
Registered: 2006-07-05
Posts: 5,941
Website

Re: Getting ready for ext4

jelly wrote:

Btw. isn't grub2 in testing?

is in extra.
for grub there is already a feature request. http://bugs.archlinux.org/task/12576

Last edited by wonder (2008-12-27 12:06:38)


Give what you have. To someone, it may be better than you dare to think.

Offline

#20 2008-12-27 13:00:00

andrek
Member
From: Poland
Registered: 2007-12-27
Posts: 29

Re: Getting ready for ext4

I'd like to wipe out my current arch setup and install it again, but this time i'd like to have my / and /home partitions on ext4. How can I set them up during the installation? Do i have to download a newer installation cd?

Offline

#21 2008-12-27 13:05:26

daf666
Member
Registered: 2007-04-08
Posts: 470
Website

Re: Getting ready for ext4

Any compelling reasons one (me) should migrate his xfs to ext4?

Offline

#22 2008-12-27 14:55:48

denisius_maximus
Member
Registered: 2008-12-10
Posts: 12

Re: Getting ready for ext4

My problem is that, grub2 hangs during grub-install on separate ext4 partition, is there any thought what it can be? Somebody are using grub2 on separate /boot partition on ext4?:(

Offline

#23 2008-12-27 15:16:44

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

O.K., after finally getting through this process of converting two partitions from ext3 to ext4, including my root (but *not* my boot, which I left as ext2) partition, here's the outline of what I had to do. If others could scan this for errors or missteps, or make other improvements, please do so.

Note that this is for converting an existing ext3 partition to ext4, not for creating a new ext4 partition from scratch (which is arguably easier and probably more efficient, since you'll have all ext4 features enabled from the beginning).

Here's the procedure I followed:

(1) Back up any ext3 partitions that you want to convert to ext4. I didn't have to use my backups, but I came very close to needing to, and it made the process much more pleasant knowing that I had the backups if I needed them.

(2) Edit /etc/mkinitcpio.conf, adding "ext4" to the "MODULES=" line. NOTE: If you're using mkinitcpio-0.5.20-1 or later and klibc-1.5.14-2 or later (both still in [Testing] repo as of 31dec08), this step may no longer be necessary. Edited on 31dec08.

(3) Install kernel 2.6.28 or later. You can use Arch kernel26 from testing (as of 26dec08); just verify that the kernel version is at least 2.6.28. Ext4 is enabled by default in the kernel config. In the Arch kernel 2.6.28, ext4 is built by default as a module, which seems to be fine.

(4) In /etc/fstab, change the file type of any ext3 partitions that you want to change to ext4. Don't change /boot to ext4 unless you've upgraded to grub2.

(5) Now do the actual filesystem conversion using the tune2fs and fsck tools. These need to be run when your target filesystem is unmounted. So, take a recent LiveCD of some sort. NOTE: It needs to have a current version of e2fsprogs; I like SystemRescue 1.1.3, which has e2fsprogs 1.41.3. A current Arch install CD should serve, too, as long as it's recent. (I'd download and burn a fresh one just to make sure.)

(a) Boot from the LiveCD, then run this on the filesystems you want to convert from ext3 to ext4:

     

tune2fs -O extents,uninit_bg,dir_index /dev/yourfilesystem

(b) After running this command you MUST run fsck. If you don't do it, Ext4 WILL NOT MOUNT your filesystem. This fsck run is needed to return the filesystem to a consistent state. It WILL tell you that it finds checksum errors in the group descriptors - it's expected, and it's exactly what it needs to be rebuilt to be able to mount it as Ext4, so don't get surprised by them. Since each time it finds one of those errors it asks you what to do, always say YES. If you don't want to be asked, add the "-p" parameter to the fsck command, it means "automatic repair":

     

fsck -pf /dev/yourfilesystem

[This part is taken from the kernelnewbies page that shuay referred to above: http://kernelnewbies.org/Ext4 .]

On my 200Gb partition, I think fsck needed about 15 minutes to complete this, using the "-pf" switch.

(6) Edit the grub /boot/grub/menu.lst file if you migrated your /root partition. Add this to the kernel line:

rootfstype=ext4

NOTE: If you're using mkinitcpio-0.5.20-1 or later and klibc-1.5.14-2 or later (both still in [Testing] repo as of 31dec08), the step just above (#6) may no longer be necessary. Edited on 31dec08.

Note also that I'm using grub legacy (not grub2), so I didn't migrate my /boot partition and I was able to make grub changes in /boot/grub/menu.lst rather than the newer grub.cfg approach. I don't know what to do for Lilo, as I no longer use it.

(7) If your main kernel doesn't boot, try the fallback.

That should do it. Please, if I've omitted something or goofed up otherwise, say so soon! I don't want to mess anyone else up.

Last edited by dhave (2008-12-31 19:59:27)


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#24 2008-12-27 15:24:12

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,112

Re: Getting ready for ext4

denisius_maximus wrote:

My problem is that, grub2 hangs during grub-install on separate ext4 partition, is there any thought what it can be? Somebody are using grub2 on separate /boot partition on ext4?:(

Does zachstruck's suggestion above help? He wrote:

Using grub2, what I had to do was add the parameter rootfstype=ext4 to the kernel line in grub.cfg and boot from my fallback kernel kernel26-fallback.img.


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#25 2008-12-27 15:24:46

andre.ramaciotti
Member
From: Brazil
Registered: 2007-04-06
Posts: 649

Re: Getting ready for ext4

@dhave
On (1) you say you almost needed your backups. On which step did this happen?


(lambda ())

Offline

Board footer

Powered by FluxBB