You are not logged in.

#1 2013-06-29 10:54:13

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

[SOLVED] Maintenance mode because fsck takes too long?

I run Arch on what is mostly used as a MythTV box and it has several large >1TB (ext4) partitions. Occasionally when booting the fsck takes too long and eventually I just get kicked to maintenance mode, which is getting really annoying, and occasionally causing me to miss recordings*. I don't upgrade this box very often, but this problem started when I did my last big upgrade in February and was forced into using systemd. I've just done another upgrade (which helped clean out a bit of cruft with the filesystem changes) and I hoped that systemd would handle this better with this upgrade, but if anything it's more frequent (could just be coincidence). How can I fix this?

Last edited by retsaw (2013-07-08 19:31:55)

Offline

#2 2013-06-29 12:38:32

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 27,786
Website

Re: [SOLVED] Maintenance mode because fsck takes too long?

You can easily disable fsck in fstab for these drives - but then you may want to do regular post-boot checks of them: periodically unmount one and fsck it.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3 2013-06-29 13:26:27

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

Thanks. I know I can disable it, but I was hoping for a solution, not a workaround. And don't I want to keep the fsck for the case of unclean shutdowns?

Systemd is supposed to be better than SysVInit, in some ways it is, but does seem a lot more complicated to learn and the added complexity has for me been more trouble than it is worth.  Perhaps because I don't yet fully understand how systemd works I am misdiagnosing the problem, but any suggestion that help me confirm (or invalidate) my idea of what I think the cause of the problem is would be welcome.

Offline

#4 2013-06-29 23:48:02

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,127

Re: [SOLVED] Maintenance mode because fsck takes too long?

I know that systemd waits 5 minutes by default for a disk to appear. Do you think fsck takes longer than 5 minutes and so systemd gives up? (This wouldn't usually cause it to dump you to an emergency shell unless something like / is on the relevant partition, though.) Is root or something on one of those large partitions?


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#5 2013-06-30 13:42:02

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: [SOLVED] Maintenance mode because fsck takes too long?

You could probably disable the checks on ext4, for newly formatted partitions, since a while ago the default has been not to periodically check the partitions.

However I don't know if some change along the way made this default safe/sane when compared with partitions formatted longer ago, maybe someone more familiar with this can chime in and provide more info.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#6 2013-06-30 14:03:22

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,127

Re: [SOLVED] Maintenance mode because fsck takes too long?

Really? genfstab still produces output for me which includes checks for ext4 partitions (lines end with a 2 rather than a 0). Unless the script somehow knows my partitions were formatted earlier which seems extremely unlikely to me. (However, although I skimmed the contents of the script, I didn't read it at all carefully.)

Last edited by cfr (2013-06-30 14:04:30)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#7 2013-06-30 18:31:41

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

I did check after I last posted and the periodic check were disabled on two of my three large partitions, (I recreate them and copy the data when I swap in a new hard drive). It happened again this morning and instead of lazily rebooting as I had got in the habit of doing (to save me entering my password) I had a look and the partition responsible (which I should mention is on a LVM2 VG) had not been activated, I manually activated it just fine, there were no errors in dmesg and the short SMART self-test came back without errors, the other partition on that LVM VG had been activated. Yesterday a partition on a different disk was the problem.

I think it is 5 minutes before I get dumped to the emergency shell, but I don't time it, so it might be. And no, my / partition is on a small 10G partition. If it is at all relevant I don't use LVM for my / or my swap, but I do for the rest of them.

Offline

#8 2013-06-30 19:19:11

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,127

Re: [SOLVED] Maintenance mode because fsck takes too long?

Whether the file system is periodically checked depends on the relevant line in fstab. Depending on how you are referring to them there, recreating them wouldn't require you to change that. (But I'm not terribly clear what you are recreating. You say the partition but do you mean the logical volume? That is, you add in the new drive, partition etc., add to the volume group, recreate the logical volume? Just guessing here...)

Is the relevant service(s) enabled for handling the lvm partitions? Do you get any clues from the status of that when things fail?

PS Do you need the large volumes enabled during the boot sequence? Or could you arrange things so that those are mounted a little later? e.g. on login. Just wondering if systemd might like that better...

Last edited by cfr (2013-06-30 19:20:41)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#9 2013-06-30 19:40:25

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

Whether a fsck is run on boot depends on the line in the fstab. I was referring to the periodic full checks that used to be default on ext partitions which used to be set every x days or y mounts (whichever came first), you can check what is set using dumpe2fs. When I said recreate, I actually meant create a new replacement partition on my new drive and move the data rather than copy the actual partition across to the new drive, sorry for not being clear.

I don't necessarily need them enabled during boot, but I do need them enabled before the MythTV backend starts, it might cause a problem if MythTV tries to record something before they are mounted, so I can't really delay them much. It is actually set to autologin and bring up the MythTV frontend by default, since that is its primary use and I only occasionally use it as a desktop computer.

Last edited by retsaw (2013-06-30 22:00:51)

Offline

#10 2013-06-30 20:01:05

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: [SOLVED] Maintenance mode because fsck takes too long?

I don't think this is a fsck problem at all.  I think you are being bit by the Arch lvm2 (lvmetad) bug.  It typically bites those that use lvm for their rootfs, but I have seen a number of people indicating that their systems without rootfs on lvm were not coming up as well.  Things are now handled by udev instead of the old way, which was a simple shell script that ran a few commands to sacn for and activate the volume groups. 

Can you try using the old packages before the lvm2 stuff changed to lvmetad?  I think this occurred around October of last year... maybe a bit later.  It is in the commit history of the package.  You will ahve to downgrade both the lvm2 pacakge as well as device-mapper.  If you are going to try this though, you should be aware that the binaries for the old package are in /sbin.  This will install just fine as pacman is at the moment.  But you should edit the list of tracked files in /var/lib/pacman/local/{lvm2,device-mapper}*/files to reflect a path along /ust/bin instead of /sbin.  (Pacman 4.2 will remove the abity to install files along a path of symlinks, though I think 4.2 is a little ways away still).

Offline

#11 2013-06-30 21:45:55

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

Thanks, I'll try that and see how it goes.

Offline

#12 2013-06-30 22:27:40

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,127

Re: [SOLVED] Maintenance mode because fsck takes too long?

retsaw wrote:

Whether a fsck is run on boot depends on the line in the fstab. I was referring to the periodic full checks that used to be default on ext partitions which used to be set every x days or y mounts (whichever came first), you can check what is set using dumpe2fs.

Interesting. Thanks. I looked up the man pages for this and found info in tune2fs. However, I'm not really clear what difference it makes i.e. setting it on the filesystem triggers periodic checks on boot; setting it in fstab triggers periodic checks on boot. Clearly missing something...


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#13 2013-06-30 23:40:23

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

The settings in the filesystem don't trigger checks by itself, it works in conjunction with the fsck done on boot. A full check takes a while to complete, so you don't want to run it every time you boot, but the creators of the ext2 filesystem clearly thought running a full check periodically was necessary to check for any possible corruption, so if you ran fsck on it and it was due a periodic check (or wasn't cleanly unmounted) it would run a full check, otherwise it skips it, with the advent of the journal (with ext3) if the drive wasn't cleanly unmounted it would just replay the journal, but the periodic checks were still deemed necessary (until recently, I guess).

Offline

#14 2013-06-30 23:47:08

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,127

Re: [SOLVED] Maintenance mode because fsck takes too long?

It still checks periodically for me with ext4 and it warns me if a file system is not subject to checking (e.g. on a USB key). Not every time, though. Just periodically for each system and sometimes it says that it will check in x number of mounts.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#15 2013-07-08 19:36:50

retsaw
Member
From: London, UK
Registered: 2005-03-22
Posts: 132

Re: [SOLVED] Maintenance mode because fsck takes too long?

On WonderWoofy's advice I rolled back lvm2 and device-mapper to 2.01.98-1, and it has been over a week without issue so I am tentatively marking this thread as solved. Thanks for the help.

Offline

#16 2013-07-08 20:21:33

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: [SOLVED] Maintenance mode because fsck takes too long?

So, I still have one machine that is on LVM2 that I had been holding back.  But recently, I decided to see if I could get it working.  So I used the official build sources from ABS, and modified it to use the git repo instead (rather sloppily I might add).  So now the machine is using lvmetad, and I have not had that problem since.  So something between the last stable release and now has been changed that has rendered the problem fixed.  But I'm not willing to take the time to bisect it to figure out what it is.  Maybe someone else with the same issue can give it a whirl. 

I put a diff of the PKGBUILD into a pastebin if you are interested. https://bbs.archlinux.org/viewtopic.php … 3#p1297303

Offline

Board footer

Powered by FluxBB