A/B booting to last known good state

jernst · 2022-01-28 23:00:02

A bunch of Linux-based systems implement A/B booting: where they keep two boot configurations around, and if the system fails to boot or shows errors after an upgrade, it reboots to the previous working configuration. Usually it involves two /boot partitions, where one is being booted from, and the other being upgraded, so the working one is still around even after the update.

I've seen this mostly done for embedded (GUI-less, unattended, or mobile) systems, but IMHO it would be nice to have that for a full desktop or server. Here is some documentation how they do it at Balena: https://www.balena.io/docs/reference/OS … rollbacks/ -- a bit convoluted!

Has anybody implemented such a thing based on Arch, or good ideas how to go about it?

I'm particularly intrigued by potentially combining this with btrfs (or such) filesystem snapshots. You could basically upgrade a working system into a new file system snapshot, leaving the current system unchanged, and then mount /boot and / from the new snapshot upon reboot. If that works, that becomes the baseline and the old snapshots can be deleted. If it does not work, the system reboots with the old snapshot and the new one gets deleted.

Thoughts?

Head_on_a_Stick · 2022-01-28 23:13:17

https://archlinux.org/packages/communit … 4/snapper/

https://archlinux.org/packages/communit … rub-btrfs/

Slithery · 2022-01-29 00:06:11

jernst wrote:

Has anybody implemented such a thing based on Arch

Yes. Lots of people have already done this. Just search the wiki for a solution that is compatible with your filesystem.

I'm particularly intrigued by potentially combining this with btrfs (or such) filesystem snapshots. You could basically upgrade a working system into a new file system snapshot, leaving the current system unchanged, and then mount /boot and / from the new snapshot upon reboot. If that works, that becomes the baseline and the old snapshots can be deleted. If it does not work, the system reboots with the old snapshot and the new one gets deleted.
Thoughts?

Yes, you can. This is already implemented in various solutions in the repos...

jernst · 2022-01-29 00:22:04

Slithery wrote:

Just search the wiki for a solution that is compatible with your filesystem.

I did, but didn't find anything. Searching for "A/B" or "boot recover" or "last good state" and variations ... What's the magic incantation that escapes me?

kermit63 · 2022-01-29 02:36:18

jernst wrote:

What's the magic incantation that escapes me?.

jernst wrote:

I'm particularly intrigued by potentially combining this with btrfs (or such) filesystem snapshots.

btrfs? then drill down to the snapshots section?

jasonwryan · 2022-01-29 03:32:23

I have two kernels installed on my Arch boxes: on my laptop - custom + vanilla, and on my desktop - vanilla + LTS. Additionally, I have two initramfs on both: mkinitcpio + dracut. That gives me multiple options to boot if something screws up, as was the case with the move to compressing modules, where I had not enabled it in my custom kernel, so I had to boot into vanilla to fix it.

That was the first time in at least 7 or 8 years that I haven't been able to boot an Arch box. Allan has got sloppy.

jernst · 2022-01-29 04:02:55

kermit63 wrote:

jernst wrote:
What's the magic incantation that escapes me?.
jernst wrote:
I'm particularly intrigued by potentially combining this with btrfs (or such) filesystem snapshots.
btrfs? then drill down to the snapshots section?

I'm aware of how btrfs snapshots work. What I don't know, and might take a while to find out, is how to use them with an A/B bootloader configuration. The link in my original post to Balena shows a structure for a non-btrfs setup. Between all of those systemd.services etc this is something I'd much rather reuse from somebody who figured it out than me trying from scratch. That's why I asked.

GrabbenD · 2023-07-27 14:11:13

Highly interested in this too!
Any updates @jernst?

Arch Linux

#1 2022-01-28 23:00:02

A/B booting to last known good state

#2 2022-01-28 23:13:17

Re: A/B booting to last known good state

#3 2022-01-29 00:06:11

Re: A/B booting to last known good state

#4 2022-01-29 00:22:04

Re: A/B booting to last known good state

#5 2022-01-29 02:36:18

Re: A/B booting to last known good state

#6 2022-01-29 03:32:23

Re: A/B booting to last known good state

#7 2022-01-29 04:02:55

Re: A/B booting to last known good state

#8 2023-07-27 14:11:13

Re: A/B booting to last known good state

Board footer