You are not logged in.

#1 2012-03-24 04:18:04

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

[SOLVED] "Filesystem check failed" during boot process

I was using XFCE on my netbook recently and clicked "shutdown".  It logged out to the command prompt and nothing happened.  Assuming that I'd clicked "logout" by mistake, I typed "pacman -Syu" to run an update...  And moments later it suddenly started the shutdown process.

Now, whenever I boot up, I see the message below.  I tried booting from a GParted CD to check the filesystems for errors, but none were detected.

Can anyone suggest what I might need to do to fix the problem?

rootfs: clean, 11026/246512 files, 477083/984576 blocks
home: clean, 4386/527280 files, 237183/2105344 blocks
/dev/sdb2 is mounted.  e2fsck: Cannot continue, aborting.

************* FILESYSTEM CHECK FAILED ************
*
* Please repair manually and reboot. Note that the root
* file system is currently mounted read-only. To remount
* it read-write type: mount -n -o remount,rw /
* when you exit the maintenance shell the system will
* reboot automatically.
*
*************************************************

Give root password for maintenance
(or type Control-D to continue):

Last edited by esuhl (2012-03-27 01:20:07)

Offline

#2 2012-03-24 06:54:03

hadrons123
Member
From: chennai
Registered: 2011-10-07
Posts: 1,249

Re: [SOLVED] "Filesystem check failed" during boot process

What happens when you do try the suggestion given by it?


LENOVO Y 580 IVYBRIDGE 660M NVIDIA
Unix is user-friendly. It just isn't promiscuous about which users it's friendly with. - Steven King

Offline

#3 2012-03-24 07:16:10

lijpbasin
Member
Registered: 2011-08-09
Posts: 28

Re: [SOLVED] "Filesystem check failed" during boot process

You can boot the system using a archlinux live cd, and run fsck manually on every linux partition in the old system with options.
DON'T mount any of the filesystems before running fsck, or your data will probably be lost. If you want to check the partition information first, run fdisk -l with root privileges.

Offline

#4 2012-03-24 07:19:40

hadrons123
Member
From: chennai
Registered: 2011-10-07
Posts: 1,249

Re: [SOLVED] "Filesystem check failed" during boot process

esuhl wrote:

.  I tried booting from a GParted CD to check the filesystems for errors, but none were detected.

@lijpbasin
Out of curiousity, how Is your suggestion better than Gparted?


LENOVO Y 580 IVYBRIDGE 660M NVIDIA
Unix is user-friendly. It just isn't promiscuous about which users it's friendly with. - Steven King

Offline

#5 2012-03-24 09:47:42

Gcool
Member
Registered: 2011-08-16
Posts: 1,456

Re: [SOLVED] "Filesystem check failed" during boot process

/dev/sdb2 is mounted.  e2fsck: Cannot continue, aborting.

As mentioned already, the reason it's failing is because the /dev/sdb2 partition is mounted before it can be fsck'd. What is this partition exactly (which filesystem, what's on it, mountpoint,...)?


Burninate!

Offline

#6 2012-03-24 09:52:42

Shark
Member
From: /dev/zero
Registered: 2011-02-28
Posts: 684

Re: [SOLVED] "Filesystem check failed" during boot process

Write umount /dev/sdb2 in konsole or unmount partition in gparted by right click on problematic partition and click unmount. Than proced.

Last edited by Shark (2012-03-24 10:03:24)


If you have built castles in the air, your work need not be lost; that is where they should be. Now put foundations under them.
Henry David Thoreau

Registered Linux User: #559057

Offline

#7 2012-03-24 10:19:19

skunktrader
Member
From: Brisbane, Australia
Registered: 2010-02-14
Posts: 1,538

Re: [SOLVED] "Filesystem check failed" during boot process

Did you try adding the fsck hook to /etc/mkinitcpio.conf ?

Offline

#8 2012-03-24 11:44:40

DSpider
Member
From: Romania
Registered: 2009-08-23
Posts: 2,273

Re: [SOLVED] "Filesystem check failed" during boot process

Are you mounting the root drive with "ro" (read only) in fstab?


"How to Succeed with Linux"

I have made a personal commitment not to reply in topics that start with a lowercase letter. Proper grammar and punctuation is a sign of respect, and if you do not show any, you will NOT receive any help (at least not from me).

Offline

#9 2012-03-24 15:44:41

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

Thank you all for your replies :-)

hadrons123 wrote:

What happens when you do try the suggestion given by it?

I don't exactly know how I should "repair manually"...  I thought a filesystem check in GParted would have been enough...

lijpbasin wrote:

You can boot the system using a archlinux live cd, and run fsck manually on every linux partition in the old system with options.
DON'T mount any of the filesystems before running fsck, or your data will probably be lost. If you want to check the partition information first, run fdisk -l with root privileges.

Thanks.  I tried booting from an ArchBang live CD last night and ran "e2fsck -pcv" on each partition.  The check completed, but the problem persists...  I just tried again using "fsck" as you suggested and got the following output (but the machine still fails to boot with the same error as before):

fsck from util-linux 2.20.1
e2fsck 1.41.14 (22-Dec-2010)
usr was not cleanly unmounted, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
usr: 115811/458752 files (1.7% non-contiguous), 783161/1834496 blocks
Gcool wrote:
/dev/sdb2 is mounted.  e2fsck: Cannot continue, aborting.

As mentioned already, the reason it's failing is because the /dev/sdb2 partition is mounted before it can be fsck'd. What is this partition exactly (which filesystem, what's on it, mountpoint,...)?

I'm not overly familiar with the Linux boot process, but why would it suddenly be mounting this partition before fsck-ing it?  Even if it was mounted when the netbook shutdown, surely the boot process starts with all partitions unmounted...?  Anyway, the partition details from /etc/fstab are below.

<file system>  <dir>  <type> <options>                    <dump> <pass>
tmpfs          /tmp   tmpfs  nodev,nosuid                 0      0
/dev/sda1      /      ext2   defaults,noatime             0      1
/dev/sdb1      /home  ext2   defaults,noatime,user_xattr  0      1
/dev/sdb2      /usr   ext2   defaults,noatime             0      1

I'm (now) aware that mounting /usr as a separate partition is a bad idea and (having read the "Error when booting with the new initscripts" thread), I have told pacman to ignore updates to the initscripts package (until I figure out how best to work round or fix that issue).

Shark wrote:

Write umount /dev/sdb2 in konsole or unmount partition in gparted by right click on problematic partition and click unmount. Than proced.

If I type "umount /dev/sdb2" at the prompt, I get the following error:

umount: /usr: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))

I tried "umount -l /dev/sdb2" to unmount when the device becomes free... but it never does.

I also booted into GParted, but there is no option to "unmount" /dev/sdb2.  There is a "mount" option in the menu, but it is greyed out.  If I open a terminal window from the GParted Live environment and type "sudo umount /dev/sdb2" I get the following message:

umount: /dev/sdb2: not mounted
DSpider wrote:

Are you mounting the root drive with "ro" (read only) in fstab?

No (as shown above).  I don't *think* I've messed anything up with fstab or any other config files -- everything was working perfectly till the netbook shutdown whilst pacman was running.

Offline

#10 2012-03-24 15:53:42

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

skunktrader wrote:

Did you try adding the fsck hook to /etc/mkinitcpio.conf ?

That looks promising.  I have to go out now, but I'll look into that later.  Judging by this post about mounting /usr on a separate partition (as I do), such hooks may be required.

Offline

#11 2012-03-24 16:19:46

DSpider
Member
From: Romania
Registered: 2009-08-23
Posts: 2,273

Re: [SOLVED] "Filesystem check failed" during boot process

By looking at your fstab, you have 2 drives in your netbook ? (sda and sdb)

And is there any particular reason why you chose ext2 ?

Last edited by DSpider (2012-03-24 16:20:19)


"How to Succeed with Linux"

I have made a personal commitment not to reply in topics that start with a lowercase letter. Proper grammar and punctuation is a sign of respect, and if you do not show any, you will NOT receive any help (at least not from me).

Offline

#12 2012-03-24 18:53:43

Gcool
Member
Registered: 2011-08-16
Posts: 1,456

Re: [SOLVED] "Filesystem check failed" during boot process

esuhl wrote:
skunktrader wrote:

Did you try adding the fsck hook to /etc/mkinitcpio.conf ?

That looks promising.  I have to go out now, but I'll look into that later.  Judging by this post about mounting /usr on a separate partition (as I do), such hooks may be required.

That's it indeed. If you're using /usr on a separate partition; there's a few additional requirements in the initramfs to make it work correctly. It's also documented on the wiki, here.


Burninate!

Offline

#13 2012-03-24 23:37:26

skunktrader
Member
From: Brisbane, Australia
Registered: 2010-02-14
Posts: 1,538

Re: [SOLVED] "Filesystem check failed" during boot process

esuhl wrote:
skunktrader wrote:

Did you try adding the fsck hook to /etc/mkinitcpio.conf ?

That looks promising.  I have to go out now, but I'll look into that later.  Judging by this post about mounting /usr on a separate partition (as I do), such hooks may be required.

Actually the comments in /etc/mkinitcpio.conf contain this

##   NOTE: If you have /usr on a separate partition, you MUST include the
#    fsck and shutdown hooks.

Offline

#14 2012-03-24 23:44:43

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

DSpider wrote:

By looking at your fstab, you have 2 drives in your netbook ? (sda and sdb)

That's right - it's an Asus EEE 901.  It has two separate SSDs -- a ~3.7GiB one and (a slightly lower-performing) ~14.9GiB one.  The space limitations are the reason that I have /usr mounted separately, whilst keeping the rest of / on the faster drive.

DSpider wrote:

And is there any particular reason why you chose ext2 ?

I *think* I read something about it being a good choice because... the lack of journaling reduces the wear of the SSDs... or it uses less disk space...  Maybe I should reconsider using something better...

skunktrader wrote:

Did you try adding the fsck hook to /etc/mkinitcpio.conf ?

Gcool wrote:

That's it indeed. If you're using /usr on a separate partition; there's a few additional requirements in the initramfs to make it work correctly. It's also documented on the wiki, here.

Hmmm...  I added the shutdown and fsck hooks to mkinitcpio.conf as suggested in the wiki, but it doesn't seem to have made any difference -- I get the same error every time I boot.

Offline

#15 2012-03-24 23:47:13

skunktrader
Member
From: Brisbane, Australia
Registered: 2010-02-14
Posts: 1,538

Re: [SOLVED] "Filesystem check failed" during boot process

esuhl wrote:

Hmmm...  I added the shutdown and fsck hooks to mkinitcpio.conf as suggested in the wiki, but it doesn't seem to have made any difference -- I get the same error every time I boot.

And you remembered to run to run

# mkinitcpio -p linux

after editing?

Offline

#16 2012-03-25 01:22:28

lijpbasin
Member
Registered: 2011-08-09
Posts: 28

Re: [SOLVED] "Filesystem check failed" during boot process

hadrons123 wrote:
esuhl wrote:

.  I tried booting from a GParted CD to check the filesystems for errors, but none were detected.

@lijpbasin
Out of curiousity, how Is your suggestion better than Gparted?

Sorry but I'm not familiar with Gparted, so I just posted what I would do if the same error occured, as a suggest

Offline

#17 2012-03-25 01:45:45

lijpbasin
Member
Registered: 2011-08-09
Posts: 28

Re: [SOLVED] "Filesystem check failed" during boot process

esuhl wrote:

No (as shown above).  I don't *think* I've messed anything up with fstab or any other config files -- everything was working perfectly till the netbook shutdown whilst pacman was running.

It seems the problem was closely related to the pacman process, assuming you had finished upgrading your system, rebooted, and the same problem appeared, then the upgrading must have brought something new into your system, a fix on mkinitcpio might be needed (as suggested by skunktrader).
If the problem was caused by the UNFINISHED pacman process, then finish (or undo) the upgrading process would fix it, since an uncleanly unmounted filesystem would have been repaired by fsck. Can you finish or undo the pacman process without booting the failed system? I don't know how to do it, but it's possible (like the arch installation process, and in my system pacman logs what's done in /var/log/pacman.log) and it's also risky!! I don't encourage you to do this, but if the problem happened to me, I will just backup my data and try.

Last edited by lijpbasin (2012-03-25 01:47:09)

Offline

#18 2012-03-25 01:51:48

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

skunktrader wrote:

And you remembered to run to run

# mkinitcpio -p linux

after editing?

Ahh... good point!  I've booted from the ArchBang live CD, mounted /dev/sda1 (which contains all of /, except /usr and /home) to /mnt/myroot and run the command below:

[live@archbang ~]$ sudo mkinitcpio -c /mnt/myroot/etc/mkinitcpio.conf -g /mnt/myroot/boot/initramfs-linux.img
==> Starting build: 3.0-ARCH
  -> Parsing hook: [base]
  -> Parsing hook: [udev]
  -> Parsing hook: [autodetect]
  -> Parsing hook: [pata]
  -> Parsing hook: [scsi]
  -> Parsing hook: [sata]
  -> Parsing hook: [filesystems]
  -> Parsing hook: [usbinput]
==> ERROR: Hook 'shutdown' can not be found.
==> ERROR: Hook 'fsck' can not be found.
==> Generating module dependencies
==> Creating gzip initcpio image: /mnt/myroot/boot/initramfs-linux.img
==> WARNING: errors were encountered during the build.  The image may not be complete.
==> Image generation successful

Oh dear...  I rebooted and now get this message:

Waiting 10 seconds for device /dev/sda1 ...
Root device '/dev/sda1' doesn't exist. Attempting to create it.
ERROR: Unable to determine major/minor number of root device '/dev/sda1'.
You are being dropped to a recovery shell
    Type 'exit' to try to continue booting
sh: can't access tty: job control turned off
[ramfs /]#

Am I doing something really stupid?!

Last edited by esuhl (2012-03-25 01:52:26)

Offline

#19 2012-03-25 02:03:22

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: [SOLVED] "Filesystem check failed" during boot process

The hooks are in /usr/lib/initcpio, so maybe you should mount /usr too:

[karol@black ~]$ pacman -Ql mkinitcpio | grep -e shutdown -e fsck
mkinitcpio /usr/lib/initcpio/hooks/shutdown
mkinitcpio /usr/lib/initcpio/install/fsck
mkinitcpio /usr/lib/initcpio/install/shutdown
mkinitcpio /usr/lib/initcpio/shutdown

Last edited by karol (2012-03-25 02:04:42)

Offline

#20 2012-03-25 02:16:43

lijpbasin
Member
Registered: 2011-08-09
Posts: 28

Re: [SOLVED] "Filesystem check failed" during boot process

Perhaps you should run mkinitcpio from the target system, like in the aif scripts:

run_controlled mkinitcpio "chroot $var_TARGET_DIR /sbin/mkinitcpio -p linux" $TMP_MKINITCPIO_LOG "Rebuilding initcpio images ..."

The actual command is

chroot $var_TARGET_DIR /sbin/mkinitcpio -p linux

The target root filesystem / and /usr needs to be mounted first (eg. to /mnt and /mnt/usr, as a result, var_TARGET_DIR will be /mnt)

Last edited by lijpbasin (2012-03-25 02:22:53)

Offline

#21 2012-03-25 02:22:57

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: [SOLVED] "Filesystem check failed" during boot process

Argh, of course he should have chrooted ...

Offline

#22 2012-03-26 03:49:17

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

lijpbasin wrote:

Perhaps you should run mkinitcpio from the target system, like in the aif scripts...

karol wrote:

Argh, of course he should have chrooted ...

Thanks -- I booted into "maintenance mode" mounted the three drives (/, /home and /usr) as described in the chroot wiki page, and ran ran chroot /mnt/arch /sbin/mkinitcpio -p linux.  The image was successfully created, but unfortunately it still fails to boot (with the same error as in my original post).

I've rebooted a few times now and noticed that, just before the error message appears, /dev/sdb2 is successfully fsck-ed:

:: Running Hook [shutdown]
:: performing fsck on '/dev/sda1'
rootfs: clean, 11027/246512 files, 477964/984576 blocks
:: performing fsck on '/dev/sdb2'
usr was not cleanly unmounted, check forced.
usr: 115811/458752 files (1.7% non-contiguous), 783161/1834496 blocks
INIT: version 2.88 booting

Yet later on, e2fsck fails causing the subsequent "FILESYSTEM CHECK FAILED" message:

/dev/sdb2 is mounted.  e2fsck: Cannot continue, aborting.

Thanks for all your help so far!  Any more ideas...?

Last edited by esuhl (2012-03-26 04:03:39)

Offline

#23 2012-03-27 01:19:51

esuhl
Member
From: UK
Registered: 2009-09-16
Posts: 140

Re: [SOLVED] "Filesystem check failed" during boot process

Aha!  I realised that my version of initscripts was probably outdated, so I chrooted again to do a pacman -Syu, and after the update my system boots normally!

Thanks again :-)

Offline

Board footer

Powered by FluxBB