Arch Linux

mgunt5

Member

Registered: 2024-06-17

Posts: 17

Advice on Manual Intervention

So four days ago, November 16, 2024, it finally happened to me. An update “broke my system” and brought me to a TTY instead of the login manager. Yep, it is a rite of passage here, I guess. Anywho, I attempted to use Timeshift to restore to a last known good system snapshot, only for the TTY error (it said to check `journalctl `for some logs or something) to persist upon reboot. I ultimately had to reinstall my Arch Linux system, restore my personal files from a known backup from Déjà Dup Backups, and reinstall all of my previously installed packages from a `pacman` package log list of past packages.

My guess is that due to the rolling release nature of Arch Linux, updates that break a system are bound to eventually happen, and  “manual intervention is required” for those cases. So I do believe from my research that an Arch Linux user is expected to read from the Arch Linux Newsletter of any updates that require manual intervention and do the necessary steps to resolve the issue. Was there anything I did wrong that led to my system being broken at that point? What can I do to better create snapshots of my system files besides Timeshift so that if this happens again, I do not have to reinstall my system and restore my personal file backups over again?

I hope this post does not rub anyone the wrong way as I know this community can be rather rough on new users. Please be merciful if I made any slip-ups. Thank you.

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,107

Re: Advice on Manual Intervention

Since you've already wiped the system in question, there's no way for anyone to know what happened. It could have been something you did wrong, it could have been something trivial, it could have been a broken package (although that's unlikely at this point), etc. There's really nothing we can do here.

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 23,289

Re: Advice on Manual Intervention

One main thing people need to be aware of when relying on timeshift/btrfs snapshots is that if you use your ESP as /boot and are booting kernels from there, the btrfs snapshots will not include the kernel. So if you timeshift to an older state that had an older kernel, you will be booting the new kernel from the ESP without any modules for that kernel (since those got rolled back) and you will have to at least downgrade the kernel package from a live disk at the worst case.

Generally speaking... it's almost impossible for an update to completely break your system. If you're unlucky a kernel update breaks your boot, keep an Arch ISO handy so you can chroot and downgrade the kernel. If "bricked my system" is actually just "my GUI didn't come up", check whether you can still switch to a TTY with Ctrl+Alt+F2... and friends.

IMO your first reaction should be trying to understand what's even going wrong so you might even be able to get a solution going if it's something systemic -- if you can't do that for some reason, then timeshift back to a known good state (noting the caveat noted above) -- reinstall should really be very very far from a reaction to the boot failing unless you have visible evidence that data got irrevocably hosed.

mgunt5

Member

Registered: 2024-06-17

Posts: 17

Re: Advice on Manual Intervention

So Timeshift would not be a good idea given that my `/boot` and `/efi` partitions are not included in my Timeshift BTRFS snapshots?  What can I do to ensure that my `/efi` and `/boot` partitions are included? My EFI system partition seems to seperate from my `/` partition.

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 23,289

Re: Advice on Manual Intervention

You "can't" really, unless you have a way to configure timeshift to keep periodic backups of filesystems that don't support snapshots on it's own that are then restored in the same "transaction". since you mention a distinct /efi, what bootloader are you using and what filesystem is /boot ? E.g. systemd-boot can't trivially boot into a specific snapshot as it doesn't support btrfs by default. If you use a bootloader that can actually read btrfs then snapshot restoration should work without a hitch, but then we are back at the fact we have no information on how your system was set up originally.