How can I recover from emergency mode?

jones · 2013-06-14 20:57:57

Hi,

a while ago I ran into quite a lot of trouble with getting Arch Linux to work on my desktop PC with an UEFI mb but it ultimately worked with GRUB.

Which was for like 5 minutes, because right after I booted into my fresh install, I ran pacman -Syu and rebooted and that was it. My kernel hadn't updated ... some UEFI thing I suppose... properly. I did not anticipate this as I am a somewhat proficient user on DE level, but I am not even safe at root/linux level.

So I am at the point where I am in emergency mode and unfortunately I neither have an Arch Linux CD nor live USB stick at the moment. The problem is nearly the same as in this topic, except for some partitioning details.

My question: suppose I have a CD, what should I do?

Last edited by jones (2013-06-14 21:02:14)

Trilby · 2013-06-14 21:05:51

First, if your description of the cause of the problem is correct, you wont need a live CD or usb.

I've never used UEFI, so you'll have to check the wiki for details, but as I understand it, after a new kernel is installed, you need to move/copy the kernel and the initramfs from /boot/ to some other folder/partition (/boot/efi ??), right?

If so, all you need to do is mount the relevant partitions:

mount /dev/sdXN /mnt/
mount /dev/sdXn /mnt/efi

then move the files:

cp /mnt/vmlinuz-linux /mnt/efi/
cp /mnt/initramfs-linux.img /mnt/efi/

Then unmount the partitions

umount /dev/sdXn
umount /dev/sdXN

Note, these are just my guesses of what the actual folders will be, but this info should be in the UEFI wiki.

cfr · 2013-06-14 23:57:53

This is not true for grub. Grub expects the kernel and initramfs to be under /boot where Arch installs them. No update is typically required for grub if the kernel is updated. If grub itself was updated, that would be different.

This is assuming that your kernel and initramfs are named the default Arch way and that you have a standard grub setup e.g. you generated grub.cfg using grub-mkconfig -o /boot/grub/grub.cfg.

Note that following Trilby's instructions won't do any harm. I just wouldn't expect it to do any good.

In emergency mode, you can mount your partitions in the way Trilby suggested. For example:

mount /sdXn /mnt
mount /sdXm /mnt/boot/efi

Check /mnt/var/log/pacman.log for errors and to see exactly what was updated.

Also post the output of ls -R /boot/efi and the contents of fstab.

Note: I do use UEFI and grub is set up on my machine.

Last edited by cfr (2013-06-15 00:00:01)

jones · 2013-06-15 08:42:35

Thank you both so far.

On my laptop now, my desktop is right now in emergency mode. First of all, there is something buggy with the emergency mode as there was this

Give root password for maintenance
(or type Control-D to continue):

So I entered the root password and it kind of freezes except for the blinking cursor. Had to hard-reset via the button on the tower.
I had to type Control-D once and then could enter the password.

to cfr: Yes, kernel and initramfs should all be configured the default way, at least to the best of my knowledge. I followed the BG 2 months ago and did not perform any customization. For getting grub to work, I did this:

#!/bin/bash
# Arch Linux Installation Script
# Created by s1ln7m4s7r - Portugal

pacman -S grub-efi-x86_64 os-prober
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=arch_grub --recheck --debug
mkdir -p /boot/grub/locale
cp /usr/share/locale/en\@quot/LC_MESSAGES/grub.mo /boot/grub/locale/en.mo
grub-mkconfig -o /boot/grub/grub.cfg

which is obviously not by me as s1n... helped me out a bit via e-mail. So I'd say I have a standard grub setup.

Now that I am within "bare" console/linux environment, I don't know how to post the output text for the commands you posted.

Last edited by jones (2013-06-15 08:56:06)

Trilby · 2013-06-15 11:06:39

Lets go back to the beginning. You assumed there was a problem booting the kernel - but you wouldn't get to emergency mode without the kernel. It looks more like init could not be found.

What are the last messages before it asks you for the root password?

jones · 2013-06-15 12:11:43

Trilby wrote:

What are the last messages before it asks you for the root password?

Alright, I just rebooted from Win 7 into arch_grub and it says (please excuse if I make any typos):

/dev/sdb2: clean, 325999/"big number" files, "big number"/"big number" blocks
systemd-fsck[167]: dosfsck 3.0.16, 01 Mar 2013, FAT32 LFN
systemd-fsck[167]: /dev/sdb3: 70 files, "big number"/"big number" clusters
Welcome to emergency mode! After logging in, type "journalctl -xb" to view 
system log, "systemctl reboot" to reboot, "systemctl default" to try again
to boot into default mode.
Give root password for maintenance
(or type Control-D to continue):

Last edited by jones (2013-06-15 12:12:15)

Trilby · 2013-06-15 12:23:18

Eh ... did you put the emergency target on your kernel line in the boot loader?

There are no errors there, it looks like the emergency target has been selected. Can you type `systemctl default` from that shell?

jones · 2013-06-15 12:45:48

Trilby wrote:

Eh ... did you put the emergency target on your kernel line in the boot loader?

Not that I recall to the best of my knowledge. I wouldn't even know how to do that so I guess if I did, it was unconsciously somehow.

Can you type `systemctl default` from that shell?

Yes, and here is what happened:

systemd-fsck[302]: dosfsck 3.0.16, 01 Mar 2013, FAT32, LFN
systemd-fsck[302]: /dev/sdb3 70, "big number"/"big number" cluster
systemd-fsck[357]: dosfsck 3.0.16, 01 Mar 2013, FAT32, LFN
systemd-fsck[357]: /dev/sdb3 70, "big number"/"big number" cluster
[    31.04numbers] systemd[1]: Failed to mount /boot.
[    31.04numbers] systemd[1]: Dependency failed for Local File Systems.
Welcome to emergency mode! After logging in, type "journalctl -xb" to view 
system log, "systemctl reboot" to reboot, "systemctl default" to try again
to boot into default mode.
Give root password for maintenance
(or type Control-D to continue):

Last edited by jones (2013-06-15 12:46:07)

Trilby · 2013-06-15 14:18:20

There's the problem, your boot partition can't be remounted. Let's start back at the beginning: what's your partitioning scheme? What are the partitions, and what file systems do they use?

I assume you used a fat filesystem for your efi partition, right? Were the fat fs tools added to your mkinitcpio.conf?

jones · 2013-06-15 14:28:39

Like I said earlier, if I knew how to to post the output of 'blkid' (and fstab) to paste.ubuntu.com (or a similar site), I would have done it already. But I don't know how that works, so is that even possible? Or should I do that manually, is that indeed what everybody else does?

Yup, the efi partition is a fat fs. I have no idea if fat fs tools were added to the mkinitcpio.conf, sorry.

WonderWoofy · 2013-06-15 14:30:11

Often times, when people have FAT filesystems in their fstab, they make the mistake of calling it FAT32 (or fat32), when what is needed is actually 'vfat'. Having the wrong fstype can certainly stop your boot dead in its tracks.

jones · 2013-06-15 15:56:23

Ok, here are the results which I photographed: http://imgur.com/a/GxOmq

WonderWoofy · 2013-06-15 16:12:10

You shoud use something like wgetpaste to redirect the output to a pastebin like site. Trying to view low quality photos of your screen is not the greatest.

jones · 2013-06-15 16:33:54

Oh, that is what I was looking for all the time!
Well then, a new status quo: I can't use wgetpaste because I don't have it installed and I can't install it becaue I can't get the internet connection to work. It says

dhpcd: command not known

hokasch · 2013-06-15 16:35:08

Try dhcpcd.

WonderWoofy · 2013-06-15 16:38:34

dhcpd is the daemon for dhcp, which is what serves ip addresses. You are thinking of dhcpcd, which is the dhcp client daemon. They both end in 'd' because they are both daemons, but they work together in a server/client setup.

jones · 2013-06-15 16:54:58

# dhcpcd
dhcpcd[265]: version 5.6.7 starting
dhcpcd[265]: no valid interfaces ound
dhcpcd[265]: no interfaces have a carrier
dhcpcd[265]: forked into background, child pid 266
# ping -c 3 google.de
ping: inknown host google.de
# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
# iwconfig
bash: iwconfig: command not found

Last edited by jones (2013-06-15 16:57:59)

hokasch · 2013-06-15 17:04:01

Please put in some effort yourself.. you have to give dhcpcd an interface.
https://wiki.archlinux.org/index.php/Ne … ent_Daemon

jones · 2013-06-15 20:12:57

Ok, so the mainboard has the Realtek RTL8111/8168 module. I does not get loaded at boot.
lscpi -v does not list any loaded/used kernel driver. ip link only shows lo. Out of cluelessness, I tried dhcpcd with eth0 and enp2s0. Both interfaces weren't found. So I downloaded the Realtek driver for Linux from the official website and put it on a USB stick but I can't mount it, mount does not show any name/data related to the USB stick.

cfr · 2013-06-15 21:38:36

You could try this:
- boot an Arch install USB into live mode
- get a network connection (as if you were following the installation procedure)
- install wgetpaste (pacman -S wgetpaste)
- mount your partitions under /mnt
- now you can access the contents of the file system e.g. /mnt/etc/fstab and use lsblk -f etc.
- use wgetpaste to post the output somewhere

Note that this will only install wgetpaste temporarily because you are "installing" it in the live environment.

Roken · 2013-06-16 07:25:07

Check your fstab to make sure that there are no incorrect UUIDs/disknames/disk identifiers in there. In the pre-systemd days the system would get past it, but with systemd any error will stop the boot short and drop you to the emergency terminal.

jones · 2013-06-16 10:04:16

Ok, I ran the live CD, chrooted into my install and got wgetpaste to work:
blkid:

/dev/sda1: LABEL="SYSTEM" UUID="9801-80D0" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="4355cabf-5f6c-4b3e-adf0-565013e3f4e1" 
/dev/sda2: PARTLABEL="Microsoft reserved partition" PARTUUID="d8cceea5-bd3c-4058-8c87-01fd1f75def2" 
/dev/sda3: UUID="FE8CE0E48CE09903" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="8fed909a-2a50-4ce0-9d16-b76c30941dcb" 
/dev/sdb1: LABEL="Data" UUID="F23039C930399617" TYPE="ntfs" PARTLABEL="Microsoft basic data" PARTUUID="ca387156-0076-4f23-a5f4-99b716c3af5b" 
/dev/sdb2: UUID="1f226f11-94ad-4791-8a45-b2c6353dd9ab" TYPE="ext4" PARTLABEL="arch" PARTUUID="2d6e3a80-115c-4296-a292-f319657bba83" 
/dev/sdb3: UUID="5684-47B2" TYPE="vfat" PARTLABEL="gpt-uefi-ding" PARTUUID="36bf32c2-85f1-43f5-a289-27e1b9e69c77" 
/dev/sdb4: UUID="7026941b-5bf1-4033-a715-d14049f49d20" TYPE="swap" PARTLABEL="swap" PARTUUID="a2f0fa78-2c23-4adc-94d2-abacb9158979" 
/dev/sdb5: UUID="d9a14da4-de58-483f-a5f9-d76056706d6a" TYPE="ext4" PARTLABEL="home" PARTUUID="1d395e23-29c4-4f03-b980-9ec729a7ef4a" 
/dev/sr0: UUID="2013-05-31-23-04-19-00" LABEL="ARCH_201306" TYPE="iso9660" PTTYPE="dos" 
/dev/loop0: TYPE="squashfs" 
/dev/loop1: UUID="f5fa8d02-203c-4081-945b-0998427d363c" TYPE="ext4" 
/dev/loop2: UUID="f5fa8d02-203c-4081-945b-0998427d363c" TYPE="ext4" 
/dev/mapper/arch_root-image: UUID="f5fa8d02-203c-4081-945b-0998427d363c" TYPE="ext4"

lsblk:

NAME                     MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                        8:0    0 232,9G  0 disk 
├─sda1                     8:1    0   100M  0 part 
├─sda2                     8:2    0   128M  0 part 
└─sda3                     8:3    0 232,7G  0 part 
sdb                        8:16   0 931,5G  0 disk 
├─sdb1                     8:17   0 736,2G  0 part 
├─sdb2                     8:18   0    30G  0 part /
├─sdb3                     8:19   0   512M  0 part /boot/efi
├─sdb4                     8:20   0     8G  0 part 
└─sdb5                     8:21   0 156,8G  0 part /home
sr0                       11:0    1 523,3M  0 rom  
loop0                      7:0    0 226,9M  1 loop 
loop1                      7:1    0   1,5G  1 loop 
└─arch_root-image (dm-0) 254:0    0   1,5G  0 dm   /etc/resolv.conf
loop2                      7:2    0   1,5G  0 loop 
└─arch_root-image (dm-0) 254:0    0   1,5G  0 dm   /etc/resolv.conf

fstab:

# 
# /etc/fstab: static file system information
#
# <file system>	<dir>	<type>	<options>	<dump>	<pass>
# /dev/sdb2
UUID=1f226f11-94ad-4791-8a45-b2c6353dd9ab	/         	ext4      	rw,relatime,data=ordered	0 1

# /dev/sdb5
UUID=d9a14da4-de58-483f-a5f9-d76056706d6a	/home     	ext4      	rw,relatime,data=ordered	0 2

# /dev/sdb3
UUID=5684-47B2      	/boot   	vfat      	noatime	          0 2

# /dev/sdb4
UUID=7026941b-5bf1-4033-a715-d14049f49d20	none      	swap      	defaults  	0 0

hokasch · 2013-06-16 11:07:40

If you are using grub, you should mount the UEFI partition as /boot/efi:

#/etc/fstab
UUID=64595f37-fbcf-4c83-b48c-9c5b76dfa4e1 /boot ext4 defaults 0 1
/dev/sda1 /boot/efi vfat defaults 0 1

See the grub wiki for more details - you can run through these steps from your chroot, but only if you have booted in UEFI mode. I would suggest to start from the beginning and clean out the UEFI partition first.
Systemd was likely failing to mount it because vfat is not in your mkinitcpio.conf, so the module needed to mount fat32 is not present in the boot image. There are other bootloaders for UEFI (which require the UEFI partition mounted as /boot)), I would just stick with the grub method though as it seems to be the most "simple" solution.

Last edited by hokasch (2013-06-16 11:10:04)

jones · 2013-06-16 12:50:13

Well I booted with a live CD and although modprobe efivars does not yield "no such device found" (click here), /sys/firmware/ only has acpi and memmap folders qhich is quite confusing.

I might give it another try at some point.
I suppose there is no actual merit to continue with this thread if there was any in the first place.
So... as this thread is not solved, I say forget this thread because I clearly am at a totally different point from what I originally posted. Please do not reply here.
A big thanks to those who replied and tried to help.

Last edited by jones (2013-06-16 12:53:43)

Arch Linux

#1 2013-06-14 20:57:57

How can I recover from emergency mode?

#2 2013-06-14 21:05:51

Re: How can I recover from emergency mode?

#3 2013-06-14 23:57:53

Re: How can I recover from emergency mode?

#4 2013-06-15 08:42:35

Re: How can I recover from emergency mode?

#5 2013-06-15 11:06:39

Re: How can I recover from emergency mode?

#6 2013-06-15 12:11:43

Re: How can I recover from emergency mode?

#7 2013-06-15 12:23:18

Re: How can I recover from emergency mode?

#8 2013-06-15 12:45:48

Re: How can I recover from emergency mode?

#9 2013-06-15 14:18:20

Re: How can I recover from emergency mode?

#10 2013-06-15 14:28:39

Re: How can I recover from emergency mode?

#11 2013-06-15 14:30:11

Re: How can I recover from emergency mode?

#12 2013-06-15 15:56:23

Re: How can I recover from emergency mode?

#13 2013-06-15 16:12:10

Re: How can I recover from emergency mode?

#14 2013-06-15 16:33:54

Re: How can I recover from emergency mode?

#15 2013-06-15 16:35:08

Re: How can I recover from emergency mode?

#16 2013-06-15 16:38:34

Re: How can I recover from emergency mode?

#17 2013-06-15 16:54:58

Re: How can I recover from emergency mode?

#18 2013-06-15 17:04:01

Re: How can I recover from emergency mode?

#19 2013-06-15 20:12:57

Re: How can I recover from emergency mode?

#20 2013-06-15 21:38:36

Re: How can I recover from emergency mode?

#21 2013-06-16 07:25:07

Re: How can I recover from emergency mode?

#22 2013-06-16 10:04:16

Re: How can I recover from emergency mode?

#23 2013-06-16 11:07:40

Re: How can I recover from emergency mode?

#24 2013-06-16 12:50:13

Re: How can I recover from emergency mode?

Board footer