[SOLVED] Ext4 file system corruption on root and home after powercuts.

LinuxLover471 · 2026-01-21 14:29:40

Hello! I hope that you are doing well.

This issue has been bugging me for quite some time. Almost every time a power cut happens, I have to boot into a live usb and run fsck on both of them. I have used archboot's 300MB .iso to make the loading time less painful, but doing all of that for 2 fsck lines is dumb.

I think the "performance optimizations" I have done from the wiki are causing this?
See:
Improving_performance/Boot_process#Initramfs
Improving_performance/Boot_process#Filesystem_mounts

My Configuration:

/etc/fstab:

# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/sda1
#UUID=6a25f563-dfec-4c54-a428-f269f558e632	/         	ext4      	rw,relatime	0 1

# /dev/sda5
UUID=8a4b323b-4947-4564-a790-b011b28e2d41	/home     	ext4		rw,defaults,commit=20,relatime,noauto,x-systemd.automount	0 2

# /dev/sda7
UUID=5005ee07-3397-4ec4-a0ed-23fb94fbe047	none      	swap		defaults,discard  	0 0

Relevant part of /etc/mkinitcpio.conf:

HOOKS=(systemd autodetect microcode keyboard sd-vconsole block filesystems fsck)

Relevant part of /boot/syslinux/syslinux.cfg:

APPEND root=/dev/sda1 rw nvidia_drm.modeset=1 zswap.enabled=1 cfg80211.ieee80211_regdom=IN rootflags=rw,defaults,commit=20,noatime usbcore.autosuspend=-1 zswap.max_pool_percent=45 zswap.accept_threshold_percent=95 zswap.compressor=lz4 tsc=reliable clocksource=tsc loglevel=3 quiet

Output of "# systemctl status systemd-fsck-root.service":

○ systemd-fsck-root.service - File System Check on Root Device
     Loaded: loaded (/usr/lib/systemd/system/systemd-fsck-root.service; static)
     Active: inactive (dead) since Wed 2026-01-21 18:14:31 IST; 1h 36min ago
   Duration: 4.849s
 Invocation: 50332eea3b9f4c7b81e0b8cb61301911
       Docs: man:systemd-fsck-root.service(8)
   Main PID: 202 (code=exited, status=0/SUCCESS)
   Mem peak: 182.9M
        CPU: 1.206s

Jan 21 18:14:24 archlinux systemd[1]: Starting File System Check on /dev/sda1...
Jan 21 18:14:24 archlinux systemd-fsck[204]: /dev/sda1: recovering journal
Jan 21 18:14:24 archlinux systemd-fsck[204]: fsck.ext4: Invalid argument while recovering journal of /dev/sda1
Jan 21 18:14:24 archlinux systemd-fsck[204]: /dev/sda1: Clearing orphaned inode 2523449 (uid=0, gid=0, mode=0100755, size=152576)
Jan 21 18:14:24 archlinux systemd-fsck[204]: /dev/sda1: Clearing orphaned inode 2548844 (uid=0, gid=0, mode=0100755, size=47064)
Jan 21 18:14:24 archlinux systemd-fsck[204]: /dev/sda1: Clearing orphaned inode 2548838 (uid=0, gid=0, mode=0100755, size=305136)
Jan 21 18:14:24 archlinux systemd-fsck[204]: /dev/sda1 was not cleanly unmounted, check forced.
Jan 21 18:14:25 archlinux systemd-fsck[204]: /dev/sda1: Inode 1704642 extent tree (at level 1) could be narrower.  IGNORED.
Jan 21 18:14:26 archlinux systemd-fsck[204]: /dev/sda1: 311811/2621440 files (0.2% non-contiguous), 3711999/10485760 blocks
Jan 21 18:14:26 archlinux systemd[1]: Finished File System Check on /dev/sda1.

That output was when a power cut happened and when I rebooted, I luckily didn't have to fire up a live USB. But when I used fsck -n on the root and home partitions, they showed some errors, means the check didn't fix the drive.

What behaviour I expect is, that when the system boots, it checks the filesystems, then if any inconsistencies are found, it repairs them, then continues.

Relevant system info:

Motherboard: GA-G31M-ES2L Rev 2.0 (FI latest Bios)
Storage:
Device 1: /dev/sda (Adata SU800 512GB SSD)
 /dev/sda1 (root) ext4
/dev/sda5 (home) ext4

Conclusion/Solution:
Turn off Ext4#Enabling_fast_commit, it's unsafe for power cuts.
Also, if that doesn't help, turn off Write caching on your drive by using "# hdparm -W 0 /dev/driveblock".
You should also use noatime unless you need it: Ext4#Disabling_access_time_update
Don't turn off barriers, they can cause severe file system damage and data loss: Ext4#Turning_barriers_off

Please ask for any other information if needed.
Thanks!
LL

Last edited by LinuxLover471 (2026-01-24 06:14:13)

seth · 2026-01-21 14:55:27

Jan 21 18:14:24 archlinux systemd-fsck[204]: fsck.ext4: Invalid argument while recovering journal of /dev>

Don't copy and paste out of the pager - the trun>

…cated line make that useless.

I expect is, that when the system boots, it checks the filesystems, then if any inconsistencies are found, it repairs them

what will generally happen, but if the filesystem or initramfs is sufficiently corrupted might simply not be possible because of all the data garbage.
Edit: what kind of errors do you get when the boot fails after such incident?

every time a power cut happens

https://www.amazon.com/s?k=ups

Last edited by seth (2026-01-21 14:58:18)

topcat01 · 2026-01-21 17:49:03

A UPS is a good idea. I have multiple Cyber Power CP1500 PFCLCD pure sine wave (while on battery) line-interactive UPSs on my servers. Apart from that, I recommend noatime unless you need atimes. Also, it seems your disk does not have proper write barriers, so it might help to test turning off the disk cache and see if that improves things. ext4 is quite robust, so needing fsck is unusual.

LinuxLover471 · 2026-01-22 04:04:13

seth wrote:

Jan 21 18:14:24 archlinux systemd-fsck[204]: fsck.ext4: Invalid argument while recovering journal of /dev>
Don't copy and paste out of the pager - the trun>

…cated line make that useless.

Updated the output!

seth wrote:

I expect is, that when the system boots, it checks the filesystems, then if any inconsistencies are found, it repairs them
what will generally happen, but if the filesystem or initramfs is sufficiently corrupted might simply not be possible because of all the data garbage.
Edit: what kind of errors do you get when the boot fails after such incident?

I will have to reproduce the error, will report when I am able to. Though I remember it said something like this:
"Failed to start file system check on /dev/sda1"

seth wrote:

every time a power cut happens
https://www.amazon.com/s?k=ups

I will need to ask my parents to buy a UPS (I am a teen), but I need a solution for the time being.

topcat01 wrote:

Apart from that, I recommend noatime unless you need atimes.

Thanks, I enabled noatime in my /etc/fstab, for the home partition (root partition was already using noatime).

topcat01 wrote:

Also, it seems your disk does not have proper write barriers

I assume you are talking about Ext4#Turning_barriers_off? If so, it's not set on both partitions,i.e., I am not using barriers=0.

topcat01 wrote:

so it might help to test turning off the disk cache and see if that improves things.

I think you are talking about write caching? If so, I have disabled it via "# hdparm -W 0 /dev/sda". Though I am wondering if there is any alternative...

I will list some specs about my storage and motherboard on the original post.

PS I am using Ext4#Enabling_fast_commit. Though I don't think it causes such issues?

Thanks!

Last edited by LinuxLover471 (2026-01-23 04:43:30)

seth · 2026-01-22 09:26:12

PS I am using Ext4#Enabling_fast_commit. Though I don't think it causes such issues?

The wiki wrote:

The ext4 "fast commits" feature introduces a new, lighter-weight journaling method.

e2fsck wrote:

Jan 21 18:14:24 archlinux systemd-fsck[204]: fsck.ext4: Invalid argument while recovering journal of /dev/sda1

"Invalid argument while recovering journal" seems incredibly rare and your experience is somewhat atypical, so: "likely"

LinuxLover471 · 2026-01-22 10:52:15

I managed to recreate the issue even after I had used noatime, turned off write caching.
Here is the exact error text:

[FAILED] Failed to start File system Check on /dev/sda1.
[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Mountpoints Configured in the Real Root.
You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, or "exit" to continue bootup.

Cannot open access to console, the root account is locked.
See sulogin(8) man page for more details.

Press Enter to continue.

I typed out the whole message, though I doubt an image is necessary. Also pressing Enter doesn't let me go anywhere, just repeats the message. (NOT the errors!).
Also, I would try disabling the fast_commit method, then try to reproduce the issue.

PS I tried reproducing with fast_commit off, cut off the power 3 times while compiling, streaming a video, etc. It did not corrupt, though I am not sure whether these results are exhaustive. So I would mark this issue as [SOLVED] for now, but if it reoccurs, please don't mind me reopening it.

Thanks!

Last edited by LinuxLover471 (2026-01-22 11:23:26)

topcat01 · 2026-01-22 18:49:00

I wouldn't advise barrier=0 in fstab for ext4. The idea is to disable write caching for the drive, which you have done (check with smartctl -x), and keep barriers enabled in ext4 (keep everything default), for safe commits of metadata.

Last edited by topcat01 (2026-01-22 18:50:06)

LinuxLover471 · 2026-01-23 04:33:44

topcat01 wrote:

I wouldn't advise barrier=0 in fstab for ext4. The idea is to disable write caching for the drive, which you have done (check with smartctl -x), and keep barriers enabled in ext4 (keep everything default), for safe commits of metadata.

I meant that I am not using barrier=0, I will update the post to clarify that.
Also with write caching, I still haven't experienced corruption even with it on.
It seems like fast_commit was the culprit, I am removing the "?" from the original post's subject.

Sorry for the confusing wording, and Thanks for the reply!

Arch Linux

#1 2026-01-21 14:29:40

[SOLVED] Ext4 file system corruption on root and home after powercuts.

#2 2026-01-21 14:55:27

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#3 2026-01-21 17:49:03

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#4 2026-01-22 04:04:13

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#5 2026-01-22 09:26:12

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#6 2026-01-22 10:52:15

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#7 2026-01-22 18:49:00

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

#8 2026-01-23 04:33:44

Re: [SOLVED] Ext4 file system corruption on root and home after powercuts.

Board footer