You are not logged in.

#1 2021-02-15 12:01:28

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Botched update left system unbootable/unchrootable, how I recovered

The point of this post is to share how I fixed an arch-chroot problem caused by an incomplete update and get some feedback before I attempt to add the recovery suggestion to a wiki page.
For example, perhaps there was a way to run pacman from the live environment and target the broken partition to avoid the hacky call to tar I used (see below).  I am out of time now to check the pacman man page.

EDIT:

man pacman wrote:

--sysroot <dir>
Specify an alternative system root. Pacman will chroot and chdir into the system root prior to running. This allows mounted guest systems to be properly operated on. Any other paths given will be interpreted as relative to the system root. Requires root privileges.

EDIT2: Thinking most of this is here: https://wiki.archlinux.org/index.php/Pa … an_upgrade

But seems like arch-chroot can simplify that procedure a bit.  Might be worth noting the specific challenge I faced on that wiki subsection (ie inability to chroot at all).

I experienced a locked up shortly after I ran `pacman -Syu` ... I recall seeing both glibc and libuv (perhaps a few others) were what pacman wanted to update.  I hit Enter after seeing it and  opened my newsreader shortly thereafter which the packages were downloading.  The newsreader covered up the terminal in which pacman was running so I do not know at what point during the transaction the lock-up occurred.  I could not ALT+Fx or connect via ssh from another box. 

Upon forcefully rebooting, I got (from memory) some error around /bin/init and upon trying again a kernel panic.  Further, I was unable to use `arch-chroot` from a live environment seeing only:

arch-chroot /mnt
chroot: failed to run command */bin/bash*: Input/output error

After googling a bit, the fix to the chroot problem was to manually untar glibc that just came down.  After doing that, I was able to run arch-chroot into the partition and repair things. 

### target partition was mounted to /mnt
cd /mnt
tar --wildcards -avxf var/cache/pacman/pkg/glibc-2.33-4-x86_64.pkg.tar.zst usr/lib/\*
cd /
arch-chroot /mnt /bin/bash

Interestingly, there is no record of the initial transaction in /var/log/pacman.log... not even that I ran `pacman -Syu` with the correct time stamp.  I suspect some data loss occurred.  I see nothing in the journalctl output for the date/time stamp aside from the initial boot.

For others experiencing this, once in the chroot, I ran pacman to manually replace those two packages.

pacman -S glibc libuv

That threw orphaned file errors so I had to throw in some --overwrite globs to get it to finish.

pacman -S glibc --overwrite '/usr/*' --overwrite /etc/rpc --overwrite /var/db/Makefile
pacman -S libuv --overwrite '/usr/*'

Last edited by graysky (2021-02-15 15:48:46)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#2 2021-02-15 12:40:35

Scimmia
Fellow
Registered: 2012-09-01
Posts: 11,461

Re: Botched update left system unbootable/unchrootable, how I recovered

Come on, graysky, you know better than to post how-tos here.

Offline

#3 2021-02-15 12:43:14

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Botched update left system unbootable/unchrootable, how I recovered

graysky wrote:

The point of this post is to share how I fixed an arch-chroot problem caused by an incomplete update and get some feedback before I attempt to add the recovery suggestion to a wiki page.  For example, perhaps there was a way to run pacman from the live environment and target the broken partition to avoid the hacky call to tar I used (see below).  I am out of time now to check the pacman man page.

Wasn't meant as a howto...


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

Board footer

Powered by FluxBB