Ext4 barriers -- pros and cons?

dhave · 2009-02-13 23:55:31

I was just reading Bogart's comments in this thread, which mention the effect of having barriers enabled on ext4 filesystems.

It's clear that it's a question of drive performance vs. data security, but I'm wondering what people's thoughts are. I usually would lean toward the performance side, at least on my desktop system, and am considering disabling barriers. First, thought, I'd like to hear some opinions either way, please.

Thanks.

Ranguvar · 2009-02-14 05:24:07

I just did a lot of reading up on this.

Personally, I'm going to have to benchmark (with real-world apps) the difference. In ext3 and such, the performance hit was massive, but as I understand it in ext4 it's gotten much better.

If it's over 10% or so, I think I'll drop them, but data reliability is always > speed (OTOH, no corruption problems yet on FSs without barriers).

Bogart · 2009-02-14 10:45:26

Ranguvar wrote:

I just did a lot of reading up on this.
Personally, I'm going to have to benchmark (with real-world apps) the difference. In ext3 and such, the performance hit was massive, but as I understand it in ext4 it's gotten much better.
If it's over 10% or so, I think I'll drop them, but data reliability is always > speed (OTOH, no corruption problems yet on FSs without barriers).

As I understand, barriers will have a big penalty on write intensive tasks. If you want to test in in a real world scenario and not a synthetic benchmark, I recommend something like:

# time tar -xzf mytarball.tar.gz

If you test a big tarball, like the kernel sources, you might be surprised by the difference.

About safety, well, Ext3 level of safety is good enough for me and most users, I guess.

But still, anyone knows how to turn them off by at boot time?

#EDIT#
In fact, adding barrier=0 in /etc/fstab is disabling barriers in spite of dmesg saying barriers enabled. Go figure why, but I benchmarked / filesystem and it behaves as without barriers.

Last edited by Bogart (2009-02-14 11:24:11)

dhave · 2009-02-14 14:32:20

This link mentioned by TheGrudge in another thread has a pretty good treatment of this issue, though it's basically left unresolved. Still, it's worth reading.

dhave · 2009-02-14 14:37:54

Bogart wrote:

#EDIT#
In fact, adding barrier=0 in /etc/fstab is disabling barriers in spite of dmesg saying barriers enabled. Go figure why, but I benchmarked / filesystem and it behaves as without barriers.

So, is this the best way to disable barriers in ext4, other than the fact that the status is incorrectly reported?

Bogart · 2009-02-14 15:51:11

dhave wrote:

Bogart wrote:
#EDIT#
In fact, adding barrier=0 in /etc/fstab is disabling barriers in spite of dmesg saying barriers enabled. Go figure why, but I benchmarked / filesystem and it behaves as without barriers.
So, is this the best way to disable barriers in ext4, other than the fact that the status is incorrectly reported?

Yes, if you want to mount your filesystem without barriers the correct way is to put the option in /etc/fstab. Otherwise you would have to "mount -o remount,barrier=0 /" after booting, which is not very nice.

Questions one could ask himself for making a decision could be:

- Have you been using Ext3 for a long time? Did you ever have problems with losing data?
- Do you ever do any write intensive task (copying lots of files or big files, extracting big archives, etc...) and care about performance?

I'd say (it's my own opinion) that for most desktop users barriers are a bit overkill. The decision is more difficult for those running a server with a big (and busy) database where both performance and data safety are very important.

And of course, test by yourself. For example, time the extraction of a 100 MB picture archive both with barriers on and off, and decide how much you care about the difference.

dhave · 2009-02-14 16:45:14

Thank you, Bogart, for the guidance.

TheGrudge · 2009-02-16 08:40:01

I will activate barriers again, my home partition was damaged today, although Linux went down normally and everything was unmounted clean.
At startup, I got the very well hated "Your filesystem is damaged.... manually repair..." message, never seen this in the last 8 years with ext3!
As I started the manual repair, I got one error about a "resize node" and two "block count" errors.
The repair went well I guess, at least the machine is running again and all my files seem to be here.
I really liked ext4 for its speed (although my media partition is always 68% fragmented, even on a fresh install) but if I get this all the time now, I guess I will go back to ext3 or reiserfs ;-(

dhave · 2009-02-16 11:56:13

TheGrudge wrote:

I will activate barriers again, my home partition was damaged today, although Linux went down normally and everything was unmounted clean.
At startup, I got the very well hated "Your filesystem is damaged.... manually repair..." message, never seen this in the last 8 years with ext3!
As I started the manual repair, I got one error about a "resize node" and two "block count" errors.
The repair went well I guess, at least the machine is running again and all my files seem to be here.
I really liked ext4 for its speed (although my media partition is always 68% fragmented, even on a fresh install) but if I get this all the time now, I guess I will go back to ext3 or reiserfs ;-(

But isn't ext4 w/ barriers better in many (all?) respects than ext3 w/out barriers?

iBertus · 2009-02-16 11:59:35

TheGrudge wrote:

I will activate barriers again, my home partition was damaged today, although Linux went down normally and everything was unmounted clean.
At startup, I got the very well hated "Your filesystem is damaged.... manually repair..." message, never seen this in the last 8 years with ext3!
As I started the manual repair, I got one error about a "resize node" and two "block count" errors.
The repair went well I guess, at least the machine is running again and all my files seem to be here.
I really liked ext4 for its speed (although my media partition is always 68% fragmented, even on a fresh install) but if I get this all the time now, I guess I will go back to ext3 or reiserfs ;-(

Wow, I've had many crashes over the years with JFS and never any data loss or anything such as you describe.

TheGrudge · 2009-02-16 12:04:27

dhave wrote:

But isn't ext4 w/ barriers better in many (all?) respects than ext3 w/out barriers?

For me ext4 with barriers is simply too slow. I develop digiKam and right now I'm doing speed optimizations. Copying 5 files in digiKam takes 10 seconds with ext4 and barriers enabled, without barriers (or reiserfs for example) it is done in 1 second.
digikam uses sqlite and during file operations a lot of tables are modified. With barriers and ext4 the performance loss is really annoying. Imagine copying / moving 200 files...

Bogart · 2009-02-16 13:04:13

Well, I guess ext4 isn't as stable as it should. I've seen several reports on LKML of rather important bugs and every week there are critical fixes being pushed into 2.6.29 (and some then backported to .28.x).

So for most users, the best strategy would be to wait at least until .29 is out to switch to ext4. And then decide about barriers (but if someone does want barriers, they should have them enabled in ext3 too).

moz · 2009-02-17 01:06:15

Hm strange, what kernel are you using? After i put barrier=0 in fstab according to dmesg barriers have been disabled on all of my ext4 partitions except for the root partition. Running your tests gives me these values:
/home (1.4766Mb/sec)
/ (1.5975Mb/sec)

So obviously barriers have been disabled on both partitions, but its unclear as to why dmesg doesnt say the same thing for both partitions.
Btw im using kernel .29-rc5

Bogart · 2009-02-17 10:56:09

moz wrote:

Hm strange, what kernel are you using? After i put barrier=0 in fstab according to dmesg barriers have been disabled on all of my ext4 partitions except for the root partition. Running your tests gives me these values:
/home (1.4766Mb/sec)
/ (1.5975Mb/sec)
So obviously barriers have been disabled on both partitions, but its unclear as to why dmesg doesnt say the same thing for both partitions.
Btw im using kernel .29-rc5

Yes, same here, dmesg says that my root partition has barriers enabled, but they are really disabled. No idea why dmesg is wrong. I'm using 2.6.28.5.

1LordAnubis · 2009-02-17 21:27:15

TheGrudge wrote:

dhave wrote:
But isn't ext4 w/ barriers better in many (all?) respects than ext3 w/out barriers?
For me ext4 with barriers is simply too slow. I develop digiKam and right now I'm doing speed optimizations. Copying 5 files in digiKam takes 10 seconds with ext4 and barriers enabled, without barriers (or reiserfs for example) it is done in 1 second.
digikam uses sqlite and during file operations a lot of tables are modified. With barriers and ext4 the performance loss is really annoying. Imagine copying / moving 200 files...

How fast is it on ext3?.. You're saying that ext4 with barriers is slower than ext3 without them?

TheGrudge · 2009-02-17 21:34:41

Yes, it is much faster with ext3, reiserfs, jfs, xfs and ext4 w/o barriers... tested it yesterday with a new partition that has been setup with all those filesystems.
Sqlite3 creates a journal.db file all the time, for every transaction. Maybe this fast create / delete cycle is not optimal with barriers.

Ranguvar · 2009-02-18 17:44:06

More info... it appears that write barriers don't work on top of device mapper (dm-loop, dm-crypt (LUKS), LVM). So I'll be going without them. (If you really need both encryption and write barriers, I *think* loop-AES, eCryptfs, and TrueCrypt work with them)

I'm going to get a backup drive anyways.

EDIT: 2.6.29-rc1 and up have barrier support for device-mapper on top of simple devices. I don't fully understand what that means yet, but oh well.
http://git.kernel.org/?p=linux/kernel/g … dd355712fa

Last edited by Ranguvar (2009-02-18 17:59:10)

kringel · 2009-03-09 03:04:10

Hi there,

I've posted here something about barriers and how to enable them at boot time with dmesg saying successfully that they are disabled:
http://bbs.archlinux.org/viewtopic.php? … 63#p512263

zyghom · 2009-03-09 11:26:09

Ranguvar wrote:

EDIT: 2.6.29-rc1 and up have barrier support for device-mapper on top of simple devices. I don't fully understand what that means yet, but oh well.
http://git.kernel.org/?p=linux/kernel/g … dd355712fa

I have some doubts:

13:23:37 papio@baboon:~$ uname -a
Linux baboon 2.6.29-rc7-ARCH #1 SMP PREEMPT Mon Mar 9 12:44:37 EET 2009 i686 Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz GenuineIntel GNU/Linux
13:23:41 papio@baboon:~$ dmesg | grep barr
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
EXT4-fs: barriers enabled
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
JBD: barrier-based sync failed on dm-3:8 - disabling barriers
13:24:07 papio@baboon:~$

Lars Stokholm · 2009-04-10 17:25:20

zyghom wrote:

Ranguvar wrote:
EDIT: 2.6.29-rc1 and up have barrier support for device-mapper on top of simple devices. I don't fully understand what that means yet, but oh well.
http://git.kernel.org/?p=linux/kernel/g … dd355712fa
I have some doubts:
13:23:37 papio@baboon:~$ uname -a
Linux baboon 2.6.29-rc7-ARCH #1 SMP PREEMPT Mon Mar 9 12:44:37 EET 2009 i686 Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz GenuineIntel GNU/Linux
13:23:41 papio@baboon:~$ dmesg | grep barr
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
EXT4-fs: barriers enabled
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
JBD: barrier-based sync failed on dm-3:8 - disabling barriers
13:24:07 papio@baboon:~$

I get the same thing with 2.6.29. Is this supposed to be supported device mapper now or should I just ignore it?

Last edited by Lars Stokholm (2009-04-10 17:26:45)

mcover · 2009-04-13 23:33:44

Lars Stokholm wrote:

zyghom wrote:
Ranguvar wrote:
EDIT: 2.6.29-rc1 and up have barrier support for device-mapper on top of simple devices. I don't fully understand what that means yet, but oh well.
http://git.kernel.org/?p=linux/kernel/g … dd355712fa
I have some doubts:
13:23:37 papio@baboon:~$ uname -a
Linux baboon 2.6.29-rc7-ARCH #1 SMP PREEMPT Mon Mar 9 12:44:37 EET 2009 i686 Intel(R) Core(TM)2 Duo CPU T7500 @ 2.20GHz GenuineIntel GNU/Linux
13:23:41 papio@baboon:~$ dmesg | grep barr
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
EXT4-fs: barriers enabled
EXT4-fs: barriers enabled
JBD: barrier-based sync failed on dm-0:8 - disabling barriers
JBD: barrier-based sync failed on dm-3:8 - disabling barriers
13:24:07 papio@baboon:~$
I get the same thing with 2.6.29. Is this supposed to be supported device mapper now or should I just ignore it?

From what i understand it keeps enabling and disabling barriers. I really would like to know if 2.6.29 supports dm+barriers, because that would save me repartitioning my hd.

dm on top of simple devices just means no dm on top of dm (e.g. LUKS on top of LVM).

Maybe its going to be better in 2.6.30: http://www.linux-archive.org/device-map … 0-rc2.html

I've purposely not been using things like LVM because of this issue. Well, i think i'll just try it, thou i'll keep my home a simple device.

TheGrudge · 2009-04-20 21:26:33

I just removed ext4 from my /var partition again.
Ext4 f****d up my pacman database three times now... enough is enough. I don't know why /var is constantly corrupt here, I checked the harddisk, everything is fine. Maybe because /var is accessed a lot?
10 minutes ago I restarted my laptop because of a kernel update, and again the ext4 /var partition claimed that it is unclean. But then the fsck could not repair it and a lot of pacman files (or parts of it) landed in "lost+found".
Pacman is broken now, I'm glad I had made a backup just 2 hours ago. I use reiserfs on /var again, and I'm really thinking about removing ext4 from the other partitions, too.
Btw I had barriers on, so I don't know what will happen if they are turned off... I thought barriers save the user from corrupting fs?

TheGrudge · 2009-04-20 21:34:03

One additional note: I suspend my laptop a lot, so a real reboot mostly happens on a kernel update. Maybe this is an issue, too...

stabele · 2009-04-20 21:34:10

There is good chance that option nodelalloc in fstab may have prevented your troubles.

Last edited by stabele (2009-04-20 22:56:02)

TheGrudge · 2009-04-20 21:44:20

Hmm maybe, but to be honest: I don't want to try this anymore ;-)
Losing the pacman DB isn't funny...

Arch Linux

#1 2009-02-13 23:55:31

Ext4 barriers -- pros and cons?

#2 2009-02-14 05:24:07

Re: Ext4 barriers -- pros and cons?

#3 2009-02-14 10:45:26

Re: Ext4 barriers -- pros and cons?

#4 2009-02-14 14:32:20

Re: Ext4 barriers -- pros and cons?

#5 2009-02-14 14:37:54

Re: Ext4 barriers -- pros and cons?

#6 2009-02-14 15:51:11

Re: Ext4 barriers -- pros and cons?

#7 2009-02-14 16:45:14

Re: Ext4 barriers -- pros and cons?

#8 2009-02-16 08:40:01

Re: Ext4 barriers -- pros and cons?

#9 2009-02-16 11:56:13

Re: Ext4 barriers -- pros and cons?

#10 2009-02-16 11:59:35

Re: Ext4 barriers -- pros and cons?

#11 2009-02-16 12:04:27

Re: Ext4 barriers -- pros and cons?

#12 2009-02-16 13:04:13

Re: Ext4 barriers -- pros and cons?

#13 2009-02-17 01:06:15

Re: Ext4 barriers -- pros and cons?

#14 2009-02-17 10:56:09

Re: Ext4 barriers -- pros and cons?

#15 2009-02-17 21:27:15

Re: Ext4 barriers -- pros and cons?

#16 2009-02-17 21:34:41

Re: Ext4 barriers -- pros and cons?

#17 2009-02-18 17:44:06

Re: Ext4 barriers -- pros and cons?

#18 2009-03-09 03:04:10

Re: Ext4 barriers -- pros and cons?

#19 2009-03-09 11:26:09

Re: Ext4 barriers -- pros and cons?

#20 2009-04-10 17:25:20

Re: Ext4 barriers -- pros and cons?

#21 2009-04-13 23:33:44

Re: Ext4 barriers -- pros and cons?

#22 2009-04-20 21:26:33

Re: Ext4 barriers -- pros and cons?

#23 2009-04-20 21:34:03

Re: Ext4 barriers -- pros and cons?

#24 2009-04-20 21:34:10

Re: Ext4 barriers -- pros and cons?

#25 2009-04-20 21:44:20

Re: Ext4 barriers -- pros and cons?

Board footer