[solved] power outage on SSD - can't boot anymore

OlaffTheGreat · 2013-09-18 19:46:22

Hi,
I just had a power outage, battery was unplugged. Boot process stops on the rescue prompt.
The last line of journalctl -xb says :

Failed at step EXEC spawning /bin/plymouth: no such file or directory, error 2

My system is installed on a SSD, Crucial M4 250GB.

Last edited by OlaffTheGreat (2013-09-21 15:41:41)

cfr · 2013-09-18 21:24:04

Unless you use plymouth, that error is harmless. Something else is the problem. systemd always tries to execute plymouth unless you've masked the service and, for most people around here, it always fails because very few of us seem to use plymouth.

karol · 2013-09-18 21:31:27

cfr wrote:

Unless you use plymouth, that error is harmless. Something else is the problem. systemd always tries to execute plymouth unless you've masked the service and, for most people around here, it always fails because very few of us seem to use plymouth.

Link to the thread about such errors and masking: https://bbs.archlinux.org/viewtopic.php … 2#p1302082

There's another user with this error: https://bbs.archlinux.org/viewtopic.php?id=169931 and the rescue prompt.

Edit:
Maybe it's not related to plymouth after all: https://bbs.archlinux.org/viewtopic.php?id=157146
http://lists.freedesktop.org/archives/s … 12377.html

Edit 2:
There's also a bug report https://bugs.archlinux.org/task/34345

Last edited by karol (2013-09-18 21:41:39)

WonderWoofy · 2013-09-18 22:00:09

So the system is actually booting, but it is just not booting to where you want it to. I mean, you're able to get to the emergency.target (or the rescue.target I can never remember which is which) and you can access the journal from there.

Many times, journalctl -xn can give nice friendly sane output with a general idea of what is going wrong. But other times it hides the nitty gritty details that can lead you to the actual issue. So maybe try just running journalctl -b --full to see where the boot is actually making it to and what kinds of big red errors are being thrown at you. I too get the plymouth error, as I see no reason for a splash screen on a <3 sec boot. But what you are looking for is likely a ways back from where you are seeing the errors.

The last time this happened to me, I switched to a kernel I had not yet used. It threw a bunch of errors at me, the most significant of which were the ones that said that it did not recognize the vfat filesystem (my ESP). It was at that point that I knew that my kernel and initramfs had not been properly copied to where they belonged in the ESP.

OlaffTheGreat · 2013-09-18 22:29:24

From the links gave by karol, I understand plymouth has some issues, but it may be not my own real problem.
As WonderWoofy suggests, I gonna through journalctl options and search where it went wrong. I'll come back then with more informations.

fukawi2 · 2013-09-19 00:30:59

Based on my experience with this issue.... I was still passing an init run-level on the kernel line in grub/syslinux/whatever I was using at the time.

Systemd doesn't have run-levels, so it didn't know how to handle the extra '5' on the end of the kernel line.

cfr · 2013-09-19 00:42:27

systemd is supposed to convert run-levels for backwards compatibility so it should work even though it isn't systemd's "native" way of specifying things. That is, according to the man page systemd.special. However, I can't quite figure out how this works or what these compatibility targets consist of.

EDIT: https://wiki.archlinux.org/index.php/Systemd#Targets

EDIT: I gather from your tweet (mentioned in another thread) that runlevels no longer exist? So is the manual page wrong and the wiki updated? Also, maybe this should get mentioned somewhere prominent since I am guessing there may be lots of users still relying on runlevels to boot. I don't remember ever getting clear instructions that this was going to be deprecated, at any rate. (But that could just be the fact that I use a sieve for a memory.)

Last edited by cfr (2013-09-19 01:00:18)

fukawi2 · 2013-09-19 02:06:14

I recall seeing something about run-levels being depreciated, but didn't make the connection in remembering that I had options in my bootloader for booting to '3' or '5'. I think the 'compatibility' extends to commands like 'init 1' or 'telinit 1' etc... Not the kernel command line.

WonderWoofy · 2013-09-19 02:09:02

They did extend to the kernel command line at one point I know for sure. But I too tried using one of these not too long ago for something or another, only to have it fail. Though, like cfr, I don't remember seeing a deprecation announcement either.

karol · 2013-09-19 02:10:52

cfr wrote:

EDIT: I gather from your tweet (mentioned in another thread) that runlevels no longer exist? So is the manual page wrong and the wiki updated? Also, maybe this should get mentioned somewhere prominent since I am guessing there may be lots of users still relying on runlevels to boot. I don't remember ever getting clear instructions that this was going to be deprecated, at any rate. (But that could just be the fact that I use a sieve for a memory.)

Archers should have at least a vague idea how their systems work and if they're using systemd, they should know that runlevels went the way of the dodo / initscripts &
SysVinit: https://wiki.archlinux.org/index.php/Runlevels

cfr · 2013-09-19 02:20:28

I am pretty sure it did used to work for runlevels on the kernel command line, though, even though I agree it does not seem to do so now.

cfr · 2013-09-19 02:25:53

karol wrote:

cfr wrote:
EDIT: I gather from your tweet (mentioned in another thread) that runlevels no longer exist? So is the manual page wrong and the wiki updated? Also, maybe this should get mentioned somewhere prominent since I am guessing there may be lots of users still relying on runlevels to boot. I don't remember ever getting clear instructions that this was going to be deprecated, at any rate. (But that could just be the fact that I use a sieve for a memory.)
Archers should have at least a vague idea how their systems work and if they're using systemd, they should know that runlevels went the way of the dodo / initscripts &
SysVinit: https://wiki.archlinux.org/index.php/Runlevels

But they did work with systemd. You could say the same about commands like reboot and shutdown. Those are equally deprecated from systemd's perspective.

I just don't remember there ever being an announcement that users needed to migrate to targets as part of the systemd move or any later update. I agree that users should know about targets etc. but it would also be nice to have a heads-up when compatibility features are going to disappear. Given that systemd did support this and now doesn't, that is a case of compatibility code being removed. But I don't remember anything in pacman's output on in the news about this happening.

EDIT: Oops. Sorry. I didn't realise I was posting below my own post. I'm rather confused about what I did to this thread. Need sleep.

Last edited by cfr (2013-09-19 02:27:31)

karol · 2013-09-19 02:33:13

When prompted https://www.archlinux.org/news/end-of-i … s-support/ I simply read the wiki https://wiki.archlinux.org/index.php?ti … to_systemd and kept reading on.
https://wiki.archlinux.org/index.php/Systemd#Targets helped me get my head around the lack of /etc/inittab.

OlaffTheGreat · 2013-09-19 10:11:51

Well, I see a lot is beeing said about run levels and systemd. Like many of you, if I had the use of init, I have to learn more about systemd before to be at ease. For the moment I'm still quiet buffled.

For now, I can give you relevant red or bold lines I find in journalctl -b --full.

Sep 19 11:09:43 obsidian kernel: Notice: NX (Execute Disable) protection cannot be enabled: non-PAE kernel!

Sep 19 11:09:43 obsidian kernel: scsi 0:0:0:0: Direct-Access     ATA      M4-CT256M4SSD2   040H PQ: 0 ANSI: 5
Sep 19 11:09:43 obsidian kernel: sd 0:0:0:0: [sda] 500118192 512-byte logical blocks: (256 GB/238 GiB)
Sep 19 11:09:43 obsidian kernel: sd 0:0:0:0: [sda] Write Protect is off
Sep 19 11:09:43 obsidian kernel: sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
Sep 19 11:09:43 obsidian kernel: sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Sep 19 11:09:43 obsidian kernel:  sda: sda1 sda2
Sep 19 11:09:43 obsidian kernel: sd 0:0:0:0: [sda] Attached SCSI disk

Sep 19 11:09:43 obsidian kernel: scsi 2:0:0:0: Direct-Access     Single   Flash Reader     1.00 PQ: 0 ANSI: 0
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] 7744512 512-byte logical blocks: (3.96 GB/3.69 GiB)
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Write Protect is off
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Mode Sense: 03 00 00 00
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] No Caching mode page present
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Assuming drive cache: write through
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] No Caching mode page present
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Assuming drive cache: write through
Sep 19 11:09:43 obsidian kernel:  sdb: sdb1
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] No Caching mode page present
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Assuming drive cache: write through
Sep 19 11:09:43 obsidian kernel: sd 2:0:0:0: [sdb] Attached SCSI removable disk

Sep 19 11:09:43 obsidian kernel: EXT4-fs (dm-2): mounting with "discard" option, but the device does not support discard

Sep 19 11:09:43 obsidian systemd-vconsole-setup[200]: /usr/bin/loadkeys failed with error code 1.

Sep 19 11:09:44 obsidian kernel: intel_rng: FWH not detected

Sep 19 11:09:44 obsidian kernel: ACPI Warning: 0x00000828-0x0000082f SystemIO conflicts with Region \PMIO 1 (20130328/utaddress-251)

Sep 19 11:09:44 obsidian kernel: composite sync not supported

Sep 19 11:09:46 obsidian systemd[1]: var.mount: Directory /var to mount over is not empty, mounting anyway.

Sep 19 11:09:50 obsidian systemd[1]: Unit fancontrol.service entered failed state.

Sep 19 11:24:33 obsidian systemd-tmpfiles[2944]: stat(/run/user/1000/gvfs) failed: Permission denied

For the "discard" option, I will open a thread about SSD if I don't find further informations.
Does the last line show it can't turn on the user ?

WonderWoofy · 2013-09-19 15:12:33

You are probably going to have to give more than just the bold and red lines. Post a large chunk (maybe like from half way the current boot to the end) to a pastebin and then link to it from here. This should be pretty easy if you can get internet access from the emergency.target. Just get that going, and then use something like wgetpaste, curlpaste, ix, or any other pastebin client. Then you can pipe the journalctl output to one of those to have it sent to the pastebin service f your choice. You will probably either have to use the --no-pager journalctl swtich, or even redirect it to a file first and then use that to post it.

OlaffTheGreat · 2013-09-19 15:41:18

I could have give a pastebin in the first place. It would have save me the waste of time selecting and copying each lines !
So here a pastebin.

I discovered I could launch a lxde session from another tty -while the boot process is still stuck on tty1. Even if some user's access are limited though, I could connect to the internet and write this post.

cfr · 2013-09-19 21:55:02

karol wrote:

When prompted https://www.archlinux.org/news/end-of-i … s-support/ I simply read the wiki https://wiki.archlinux.org/index.php?ti … to_systemd and kept reading on.
https://wiki.archlinux.org/index.php/Systemd#Targets helped me get my head around the lack of /etc/inittab.

It's more the lack of any announcement regarding the disappearance of the compatibility code which bothers me. I'm already using targets in my own boot files. (To be honest, I hadn't been using Linux long enough to have a particularly deep understanding of runlevels/inittab. darwin is a little different.)

@OP,
Note line 753. Did you have /var unmounted at some point? Stuff seems to have ended up under the /var directory on the root partition. Also, there seems to be something odd about your console configuration - is there a "en" keymap? And some of your mount options are being ignored as they don't apply to the devices or filesystems you are mounting. Maybe you are trying to use "discard" on an HDD, for example?

Last edited by cfr (2013-09-19 22:08:04)

OlaffTheGreat · 2013-09-20 15:12:39

About /var, I don't see how it would be unmounted... when the power outage occured, the system was running, so /var was mounted...

For keymap, I use a script I launch on login

setxkbmap -option grp:switch,grp:shifts_toggle,grp_led:scroll us,fr-latin9

so it is an english/us keymap by default.

The discard option, I don't know yet. When I read about SSD settings, I learnt it serve to enable TRIM, so I don't understand why it doesn't work (it's a Crucial M4 250GB)

I still don't get how to fix back the boot process...

OlaffTheGreat · 2013-09-21 15:41:06

Problem solved by a full upgrade of the system.
Thank you for your help

cfr · 2013-09-21 20:21:27

You should investigate what is under your mount of /var though, even though the immediate problem is solved.

OlaffTheGreat · 2013-09-22 21:04:07

/var issue as well the discard option issue, perhaps others. Maybe I could open a thread to deal with those issues.
Thank you for the reminder anyway.

Arch Linux

#1 2013-09-18 19:46:22

[solved] power outage on SSD - can't boot anymore

#2 2013-09-18 21:24:04

Re: [solved] power outage on SSD - can't boot anymore

#3 2013-09-18 21:31:27

Re: [solved] power outage on SSD - can't boot anymore

#4 2013-09-18 22:00:09

Re: [solved] power outage on SSD - can't boot anymore

#5 2013-09-18 22:29:24

Re: [solved] power outage on SSD - can't boot anymore

#6 2013-09-19 00:30:59

Re: [solved] power outage on SSD - can't boot anymore

#7 2013-09-19 00:42:27

Re: [solved] power outage on SSD - can't boot anymore

#8 2013-09-19 02:06:14

Re: [solved] power outage on SSD - can't boot anymore

#9 2013-09-19 02:09:02

Re: [solved] power outage on SSD - can't boot anymore

#10 2013-09-19 02:10:52

Re: [solved] power outage on SSD - can't boot anymore

#11 2013-09-19 02:20:28

Re: [solved] power outage on SSD - can't boot anymore

#12 2013-09-19 02:25:53

Re: [solved] power outage on SSD - can't boot anymore

#13 2013-09-19 02:33:13

Re: [solved] power outage on SSD - can't boot anymore

#14 2013-09-19 10:11:51

Re: [solved] power outage on SSD - can't boot anymore

#15 2013-09-19 15:12:33

Re: [solved] power outage on SSD - can't boot anymore

#16 2013-09-19 15:41:18

Re: [solved] power outage on SSD - can't boot anymore

#17 2013-09-19 21:55:02

Re: [solved] power outage on SSD - can't boot anymore

#18 2013-09-20 15:12:39

Re: [solved] power outage on SSD - can't boot anymore

#19 2013-09-21 15:41:06

Re: [solved] power outage on SSD - can't boot anymore

#20 2013-09-21 20:21:27

Re: [solved] power outage on SSD - can't boot anymore

#21 2013-09-22 21:04:07

Re: [solved] power outage on SSD - can't boot anymore

Board footer