You are not logged in.
For the last several of my approximately monthly Arch Linux upgrades on my 2 Dell PowerEdge R430 servers, I have had to manually reinstall the Linux kernel to get the machine to boot after the upgrade. Just after the upgrade, when the machines do not come up, I see this message on the screen which seems to be from something Dell put on the machine:
UEFI0002: The system was reset due to a timeout horn the watchdog timer.
Check the System Event Log (SEL) or crash dumps from Operating System to
identify the source that triggered the watchdog timer reset.
Update the firmware or driver for the identified device.
Auailable Actions:
F1 to Continue and Retry Boot Order
F2 for System Setup (BIOS)
F10 for LifeCycle Controller
- Enable/Configure iDRAC
- Update or Backup/Restore Server Firmware
- Help Install an Operating System
- F11 for Boot Manager
If I hit `F1` and try to boot, it gives this error from the Arch boot process:
:: running early hook [udev]
kmod: error while loading shared libraries: liblzma.so.5: cannot open shared obj
ect file: No such file or directory
systemd-tmpfiles: error while loading shared libraries: liblzma.so.5: cannot ope
n shared object file: No such file or directory
/usr/lib/systemd/systemd-udevd: error while loading shared libraries: liblzma.so
.5: cannot open shared object file: No such file or directory
:: running hook [udev]
:: Triggering uevents...
udevadm: error while loading shared libraries: liblzma.so.5: cannot open shared
object file: No such file or directory
udevadm: error while loading shared libraries: liblzma.so.5: cannot open shared
object file: No such file or directory
udevadm: error while loading shared libraries: liblzma.so.5: cannot open shared
object file: No such file or directory
Waiting 10 seconds for device /dev/disk/by-uuid/43e3cff2-80e-4887-afdd-897m12d
6dc4 ...
ERROR: device 'UUID=43e3cff2-8e7e-4007-afaa-897acl2d6dc41 not found. Skipping fs
ck.
:: mounting 'OUID=43e3cff2-80e-4887-afaa-897ac12d6dc41 on real root
mount: /new_root: can't find UUID=43e3cff2-80e-4887-afaa-897ac12d6dc4.
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off
[rootfs ]# _
To get the machine running again, I execute steps like this after booting off of a rescue USB drive:
fdisk -l # Find the boot drive
mkdir /mnt/root/
mount /dev/sdb2 /mnt/root # Mount the boot drive
arch-chroot /mnt/root
cd /var/cache/pacman/pkg
pacman -U linux-4.14.5-1-x86_64.pkg.tar.xz linux-firmware-20171206.fdee922-1-any.pkg.tar.xz linux-api-headers-4.12.7-1-any.pkg.tar.xz # Reinstall packages
Does anyone have ideas of what might be going on here? I would like to get back to being able to upgrade these servers without having to make a physical appearance afterwards.
Offline
Compare your fstab (or systemd generators, whatever you use) with the entries in your bootloader. It tries to mount something that isn't there. Weird missing libraries during the boot process could be old libraries expected from the old kernel image. I suspect your /boot isn't mounted properly when you do the updates.
Offline
I don't have "/boot" on a separate partition, it's just on "/". Heres' the snippet from "/boot/syslinux/syslinux.cfg":
LABEL arch
MENU LABEL Arch Linux
LINUX ../vmlinuz-linux
INITRD ../intel-ucode.img,../initramfs-linux.img
APPEND root=UUID=43e3cff2-8e7e-4887-afaa-897ac12d6dc4
Which matches the fstab:
# Static information about the filesystems.
# See fstab(5) for details.
#
# <file system> <dir> <type> <options> <dump> <pass>
# /dev/sdb2
UUID=43e3cff2-8e7e-4887-afaa-897ac12d6dc4 / ext4 rw,relatime,data=ordered 0 1
# /dev/sdb1
/dev/mapper/swap none swap defaults 0 0
# /dev/sda1
UUID=d2605736-5fb8-4dbb-b0dd-c67a41f54ccf /array ext4 rw,relatime,data=ordered 0 2
Those look OK to me. It also doesn't line up that a kernel reinstall would fix an issue with the boot (not that anything does to my mind).
Offline
You have a UEFI system. The EFI partition must be VFAT. You do not have a separate boot partition. You do not appear to have a separate EFI partition mounted at /boot/efi.
But, / is ext4.
Um?
Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way
Offline
I will look into how to configure UEFI, it sounds like I might be doing something wrong there.
I thought I disabled UEFI in the BIOS so that I wouldn't have to deal with it, but maybe I didn't or maybe it's not possible.
Offline
So, you think it is configured as MBR? Could be that you are in compatibility mode. I based by assertion on
UEFI0002: The system was reset due to a timeout horn the watchdog timer.
Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way
Offline
I thought I disabled UEFI in the BIOS...
Read this: https://www.happyassassin.net/2014/01/2 … work-then/
Offline
OK, I had my terminology wrong. I should have said that I configured the machine through the firmware UI to use BIOS compatibility mode for booting.
I do seem to have combined BIOS+GPT which that article recommends against. Perhaps I'll try switching to UEFI-native booting.
21:20:01 bpotter@resiliency syslinux sudo parted /dev/sdb
GNU Parted 3.2
Using /dev/sdb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: DELL PERC H730 Mini (scsi)
Disk /dev/sdb: 199GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 1075MB 1074MB
2 1075MB 199GB 198GB ext4 legacy_boot
Offline
UEFI0002: The system was reset due to a timeout horn the watchdog timer.
could be coming from the firmware itself and the system is booting arch using CSM
Offline
You have a UEFI system. The EFI partition must be VFAT. You do not have a separate boot partition. You do not appear to have a separate EFI partition mounted at /boot/efi.
But, / is ext4.Um?
Added: OP uses syslinux, which unlike refind and grub do not support UEFI filesystem drivers for ext4 which allow booting without a separate boot partition.
Managing AUR repos The Right Way -- aurpublish (now a standalone tool)
Offline