Arch Linux

Akim

Member

Registered: 2021-08-04

Posts: 42

Shrinking LVM-on-LUKS maths

I am following the arch wiki guide (https://wiki.archlinux.org/title/Resizi … UKS_volume) on how to shrink LVM-on-LUKS and I am really puzzled by the "Resize LUKS volume" step because it uses the following formula:

NEW_LUKS_SECTOR_COUNT = PV_EXTENT_COUNT * PV_EXTENT_SIZE IN BYTES / LUKS_SECTOR_SIZE

PV Size               454.31 GiB / not usable 3.00 MiB
PE Size               4.00 MiB
Total PE              116303
sector size:  512

(116303 extents + 1 unusable extent) * 4194304 B/extent / 512 B/sector = 952762368 sectors

Everything is fine until the mysterious 4194304 B/extent number. I simply don't understand where it comes from. Supposedly it should be (454.31×1024^3)/116303 but it isn't. The numerator is fine since it is just 454.31 Gib in bytes, so solving for x:

116304 × (454.31×1024^3)/x × 1/512 = 952762368 => x = 116303.36

Where does this .36 come from? I don't know how significant it is but I'm hesitant to proceed unless I fully understand what is hapenning.

Upd:
Using 116303 instead of 116303.36 gives 952765317.1 sectors, i.e a 2949 sectors difference, so seems pretty significant.

Last edited by Akim (2023-01-28 19:41:19)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,417

Re: Shrinking LVM-on-LUKS maths

For LVM, you have a metadata offset (1st PE offset, usually 1MiB), and a PE size that everything else aligns to. LVM can only use whole PE, so the size available to LVM is the block device size minus 1st PE offset divided by PE size (and any partial PE is unusable).

If your LUKS device size is a multiple of 4 MiB, the "not usable 3 MiB" is due to the 1st PE 1MiB offset. You would not have any unusable space if you used multiple of 4 MiB plus 1 MiB for the LUKS device size. Or simply by changing the PE size to 1 MiB, then you don't have unusable space as long as everything is MiB aligned.

The formula in the wiki ignores the LVM metadata offset. It simply happens to "work out" by adding 1 more extent and that extent happens to be larger than space required for metadata. However the LVM metadata offset could also be larger than 1 extent, then it doesn't work at all.

You always have to consider data offsets.

For LUKS, you have a header, this header may be 2 MiB (for LUKS 1), or 16 MiB (for LUKS 2), or anything else. The size of the crypt mapping is the size of the LUKS block device minus that header size. Check with cryptsetup luksDump or compare `blockdev --getsize64` /dev/lukspartition vs /dev/mapper/luksmapping.

So the total device size is size of LUKS header + size of LVM metadata (1st PE offset) + PE*PE_SIZE. That's if you somehow start this from LVM point of view which mainly tells you the PV size.

You shouldn't use fractions (454.31 GiB) for anything as that most likely already suffers from rounding errors.

Akim

Member

Registered: 2021-08-04

Posts: 42

Re: Shrinking LVM-on-LUKS maths

For LVM, you have a metadata offset (1st PE offset, usually 1MiB), and a PE size that everything else aligns to. LVM can only use whole PE, so the size available to LVM is the block device size minus 1st PE offset divided by PE size (and any partial PE is unusable).

If your LUKS device size is a multiple of 4 MiB, the "not usable 3 MiB" is due to the 1st PE 1MiB offset. You would not have any unusable space if you used multiple of 4 MiB plus 1 MiB for the LUKS device size. Or simply by changing the PE size to 1 MiB, then you don't have unusable space as long as everything is MiB aligned.

The formula in the wiki ignores the LVM metadata offset. It simply happens to "work out" by adding 1 more extent and that extent happens to be larger than space required for metadata. However the LVM metadata offset could also be larger than 1 extent, then it doesn't work at all.

You always have to consider data offsets.

For LUKS, you have a header, this header may be 2 MiB (for LUKS 1), or 16 MiB (for LUKS 2), or anything else. The size of the crypt mapping is the size of the LUKS block device minus that header size. Check with cryptsetup luksDump or compare `blockdev --getsize64` /dev/lukspartition vs /dev/mapper/luksmapping.

So the total device size is size of LUKS header + size of LVM metadata (1st PE offset) + PE*PE_SIZE. That's if you somehow start this from LVM point of view which mainly tells you the PV size.

You shouldn't use fractions (454.31 GiB) for anything as that most likely already suffers from rounding errors.

Thank you for a detailed response. I know I'm asking for a lot here but can you (or someone else) please show how to properly do the math for my system? Unfortunately, I'm still uncertain about how to do it right.

PV size: 1022849187840 B / not usable 713216 B
PE size: 4194304 B
Total PE: 243866
Allocated PE: 243866
Sector size: 512

# blockdev --getsize64 /dev/crypt/root
1005668270080

# blockdev --getsize64 /dev/mapper/crypt-root
1005668270080

# lsblk
nvme0n1 (type disk)
    —nvme0n1p1 (type part, boot partition)
    —nvme0n1p2 (type part)
          *cryptdisk (type crypt)
              •crypt-swap (type lvm)
              •crypt-root (type lvm)

Last edited by Akim (2023-01-28 23:59:52)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,417

Re: Shrinking LVM-on-LUKS maths

Well, what exactly is it that you want to do?

If you first shrink crypt/root filesystem by 100G (only possible if your filesystem supports shrinking), you can then shrink crypt/root LV by 100G (these two steps can be combined into one using `lvresize --resizefs`, so LVM will handle the filesystem resize for you, but it only works for supported filesystems), then shrink cryptdisk PV by 100G, then shrink nvme0n1p2 LUKS by 100G, and finally shrink nvme0n1p2 partition by 100G - these last two steps can also be combined by cryptsetup close/open or reboot - LUKS will use the new partition size. That's assuming you use simple LUKS w/o shenanigans like integrity or multiple data segments.

It's the same 100G in each step, it's just that the commands to do so (other than lvresize) usually don't understand -100G as a parameter, they expect you to give values relative to their device size and offsets. So you end up doing math for each step along the way.

Another option is to copy all files off, redo partitions from scratch, and copy files back. That's also the way to go if your filesystem doesn't support shrinking.

In general, growing is easier than shrinking, because almost everything supports growing, and almost all commands do "grow to max" by default so you don't have to specify sizes in each step. So growing does not involve maths or rather the commands do the math for you when they don't do it for shrinking…

Akim

Member

Registered: 2021-08-04

Posts: 42

Re: Shrinking LVM-on-LUKS maths

If you first shrink crypt/root filesystem by 100G (only possible if your filesystem supports shrinking), you can then shrink crypt/root LV by 100G (these two steps can be combined into one using `lvresize --resizefs`, so LVM will handle the filesystem resize for you, but it only works for supported filesystems), then shrink cryptdisk PV by 100G, then shrink nvme0n1p2 LUKS by 100G, and finally shrink nvme0n1p2 partition by 100G - these last two steps can also be combined by cryptsetup close/open or reboot - LUKS will use the new partition size. That's assuming you use simple LUKS w/o shenanigans like integrity or multiple data segments.

My end goal is to increase the boot partition by 1G (/dev/nvme0n1p1) by shrinking root by 1G. I already shrunk the filesystem and root LV with lvresize --resizefs however now I cannot figure out the new LUKS sector count for `cryptsetup -b $NEW_LUKS_SECTOR_COUNT resize cryptdisk`, as I'm still unsure about how to use the formula, or whether it's even correct or not.

NEW_LUKS_SECTOR_COUNT = PV_EXTENT_COUNT * PV_EXTENT_SIZE IN BYTES / LUKS_SECTOR_SIZE

So if you could help me out with this one calculation, it would have been wonderful.

frostschutz

Member

Registered: 2013-11-15

Posts: 1,417

Re: Shrinking LVM-on-LUKS maths

In order to grow partition1 you'd have to shift the entire contents of partition2. That's annoying to do and can go wrong - you need a backup and at this point, you can just use the file copy method instead. If it's far from 100% full, it's even more efficient since you don't have to re-write everything, just the files themselves.

Alternatively if p1 is very small, it might be a better choice to just abandon it and make a new partition at the end of disk (after shrinking p2). If you shrink by 1G you'd end up with a 1G /boot at the end of disk. If you want it to be larger you have to shrink it a little more.

Did you already shrink the PV with pvresize? According to your output above, your LVM is fully allocated (Total = Allocated, 0 free PE).

Can you show `blockdev --getsize64` for /dev/nvme0n1p2 and /dev/mapper/cryptdisk? Also `parted /dev/nvme0n1 unit s print free`?

Akim

Member

Registered: 2021-08-04

Posts: 42

Re: Shrinking LVM-on-LUKS maths

Did you already shrink the PV with pvresize? According to your output above, your LVM is fully allocated (Total = Allocated, 0 free PE).

Can you show `blockdev --getsize64` for /dev/nvme0n1p2 and /dev/mapper/cryptdisk? Also `parted /dev/nvme0n1 unit s print free`?

Yes, I already ran `pvresize --setphysicalvolumesize 1022849523200B /dev/mapper/cryptdisk` successfully.

# blockdev --getsize64 /dev/nvme0n1p2
1023940042240

# blockdev --getsize64 /dev/mapper/cryptdisk
1023923265024

#parted /dev/nvme0n1 unit s print free
...
Start      End         Size         File System
34s      2047s      2014s      Free space
...

#pvdisplay /dev/mapper/cryptdisk
...
Allocatable: yes (but full)
Free PE: 0
...

Yes, my boot partition is very small, 256M (other partition is 953.6G). In fact, it's so small that pacman was throwing a zstd error a couple of times saying there is no space left (which in hindsight I probably shouldn't have ignored) but everything was working fine overall. Well, yesterday it couldn't boot anymore so I have to increase the size of boot. I would not want to abandon it because then I would have to do more shenanigans to set up another boot partition.

A bit unrelated but just in case you know something about it. I did do a backup, cloned the entire SSD to an identical one. However I don't know how reliable this backup is (or how to check it's reliablity) because when I just received that SSD I accidentally peeled a tiny element off the circuit board, which looked like a transistor or something I did do backups with it in the past and when manually examining the files they seemed fine but of course I can't know for sure if it's the case for every piece of data. Any advice?

Last edited by Akim (2023-01-29 11:25:35)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,417

Re: Shrinking LVM-on-LUKS maths

OK, in that case it's fine.

For cryptsetup resize --size [or --device-size], you don't add the 16 MiB of the LUKS header. So it would be `cryptsetup resize --device-size=975465M cryptdisk`. And blockdev should report 1022849187840 bytes (same as pvdisplay) as the size afterwards. But if you don't mind rebooting, you also don't need cryptsetup resize at all.

The partition size must be the size reported by `pvdisplay --units b` ( 1022849187840 B ) plus 16777216 (16 MiB) of the LUKS header (which you can see in `cryptsetup luksDump` or by doing math like `blockdev --getsize64 /dev/nvme0n1p2` minus `blockdev --getsize64 /dev/mapper/cryptdisk` )

So the partition size should be 1022865965056 bytes (1997785088 sectors, or 249723136 4K-sectors).

parted 'resizepart' wants the end sector so you have to add the start sector to the size (as shown by parted, or /sys/block/nvme0n1/nvme0n1p2/start). Check with 'print free' that the partition size is correct before rebooting.

If LUKS / LVM complains after reboot (partition too small) you'll just have to repeat the parted resizepart step [from a live cd]. By itself, parted resizepart does not damage your data.

(You should then have 1G of free space at the end of disk, which you can then use for your new boot partition, without moving anything else. Creating this partition and formatting it would damage data, so you should verify that LUKS/LVM works with the new partition sizes first.)

Hope this helps,

Last edited by frostschutz (2023-01-29 12:53:17)

Akim

Member

Registered: 2021-08-04

Posts: 42

Re: Shrinking LVM-on-LUKS maths

(You should then have 1G of free space at the end of disk, which you can then use for your new boot partition, without moving anything else. Creating this partition and formatting it would damage data, so you should verify that LUKS/LVM works with the new partition sizes first.

I completed all the steps and I do have a 1G free space at the end of the disk now. You mentioned that creating the partition would damage the data. Do you mean that my files could potentially get damaged or is it inevitable? Also how do I verify that LUKS/LVM work with new partition sizes? I rebooted and ran `creyptsetup luksOpen /dev/nvme0n1p2 cryptdisk` without any errors, does it mean that everything is fine or should I check something else before creating the new partition?

By the way, thank you for your immense help so far

frostschutz

Member

Registered: 2013-11-15

Posts: 1,417

Re: Shrinking LVM-on-LUKS maths

If your filesystems are still there (LVM would complain if the device was too small) then you're fine.