Understanding Btrfs (or not so much)

zacariaz · 2022-09-10 14:22:06

I'm a reasonable intelligent guy, which has even been documented, though, I fear, not so much on this forum, but Btrfs, has given me brain cancer, I'm sure, trying understand how it works, so I really hope that someone will be kind enough to enlighten me, if at all possible.

I've been trying a lot of different approaches, using the install script, in order to try at do, what I want to do, but best I can do, is get some obscure python error. Something about 'dict' on 'dict', or rather. However, I'm pretty sure it's user error, thus I won't dwell on that.

I have 2 discs, an SSD and an HHD.
I'd very much like for everything to be encrypted, which is, apparently, problematic, in terms of autologin to my WM of choice, but that I can work with.

I am perfectly happy with the suggested layout for the SSD, with the EFI, root, home, snapshot, log, etc., and it works out of the box, that is, besides the autologin issues, of course.

Then there's the HDD, and that's where my brain starts to melt. All I really want to do, is have my user directories, or home directory, on that, plus perhaps snapshot, if at all possible, and shared data, between the users. There will likely be 2 users, thus, mount as home, with sub volumes for downloads, documents and so on. However I see fit. Perhaps it doesn't make a whole lot of sense, I really don't know anymore, but then again, I'm really just playing around, trying to get into the game again. What I do know, from some 8 hours of painful experience, is that I am apparently utterly incapable of understand the subject, Btrfs.
For example, every single piece of documentation, every guide and every tutorial, I've come across, relating to a multidisc setup, talks about raid. I don't want to do raid, of any kind. I just want ease of management, and the ability to decide what is stored on the fast SSD, and what is stored on the slow HDD, without worrying too much, if the various partitions are large enough.

And then there's the top level volume, or sub volume (?), always, with no exception, set as root. Is it even possible to use Btrfs, without it involving the root filesystem? Even that, would be helpful.

Alas, every single thing I've tried, has failed utterly, convincing me, that there's something really basic, that I just don't understand, and I've now given up, hoping to be able to set it up as I desire, without having to start over from scratch, should the miracle of comprehension, some day happen.

ANY help that brings me closer to enlightenment, will be greatly appreciated.
Suggestions to alternative approaches, are also welcome, but I am perfectly capable of doing it all, the "normal" way, but I do find Btrfs very interesting, so that is the subjected, how to best configure the system, involving that, and what is possible and not possible.

Best regards.

NB.
If you, yourself, have a multidisc setup, with Btrfs, perhaps just sharing the FSTAB, or other relevant config files will be useful. I suspect it may well be a sowing machine situation, where a tiny detail, makes all the difference.

tucuxi · 2022-09-10 18:19:23

Too much irrelevant text. I didn't read all of it. I assume you want to see someone's fstab:

LABEL=BTRFS  /            btrfs  subvol=@,compress=zstd,autodefrag     0 0
LABEL=BTRFS  /var         btrfs  subvol=@var,nodev,nodatacow           0 0    # nosuid breaks docker
LABEL=BTRFS  /home        btrfs  subvol=@home,nodev,nosuid,autodefrag  0 0
LABEL=BTRFS  /btrfs       btrfs  subvolid=5                            0 0
LABEL=BOOT   /boot        vfat   codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro,nodev,noexec,nosuid	0 2

zacariaz · 2022-09-10 18:43:16

tucuxi wrote:

Too much irrelevant text. I didn't read all of it. I assume you want to see someone's fstab:

LABEL=BTRFS  /            btrfs  subvol=@,compress=zstd,autodefrag     0 0
LABEL=BTRFS  /var         btrfs  subvol=@var,nodev,nodatacow           0 0    # nosuid breaks docker
LABEL=BTRFS  /home        btrfs  subvol=@home,nodev,nosuid,autodefrag  0 0
LABEL=BTRFS  /btrfs       btrfs  subvolid=5                            0 0
LABEL=BOOT   /boot        vfat   codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro,nodev,noexec,nosuid	0 2

To each his own.
Thank you for the effort.
Presumably this involves one disc?

Slithery · 2022-09-10 18:50:24

zacariaz wrote:

For example, every single piece of documentation, every guide and every tutorial, I've come across, relating to a multidisc setup, talks about raid. I don't want to do raid, of any kind. I just want ease of management, and the ability to decide what is stored on the fast SSD, and what is stored on the slow HDD, without worrying too much, if the various partitions are large enough.

If you don't want a RAID setup then simply don't create one.
Just format the drives separately and create the subvolumes that you want on each drive and mount them normally.

zacariaz wrote:

And then there's the top level volume, or sub volume (?), always, with no exception, set as root.

You can use either the top-level volume or a subvolume as root, it depends on your preference.

zacariaz wrote:

Is it even possible to use Btrfs, without it involving the root filesystem? Even that, would be helpful.

Of course it is, just the same as any other filesystem. Just create whatever subvolumes (if any) that you want and mount them as you wish.

You're just overthinking things.

tucuxi · 2022-09-10 19:01:40

zacariaz wrote:

Presumably this involves one disc?

Nope, the file system extends across one partition and one complete disk in a raid0 data/raid 1 metadata setup.

zacariaz · 2022-09-10 19:03:02

Slithery wrote:

If you don't want a RAID setup then simply don't create one.
Just format the drives separately and create the subvolumes that you want on each drive and mount them normally.
You're just overthinking things.

I've tried that, without success, but perhaps that's not my fault, after all.

What I haven't really been able to find examples of, are btrfs partitions (not sub volumes), not mounted as root, so that's the one place, I may be missing something.

Is the "top level" subvolume, or whatever it's called, actually necessary? (I think it is)
Are the remaining sub volumes mounted relative to that, or to the root file system?
Can you mount a btrfs filesystem in another btrfs filesystem?
Can you mount an encrypted btrfs filesystem on an unencrypted one?
I could go on.
Again I'm back at lagging the most fundamental understanding, it would seem, and for reasons beyond my comprehension, I am not able to find the examples/instructions I seek.

Well, I'm done for today, and tomorrow I'll experiment some more.

Thank.

zacariaz · 2022-09-10 19:07:05

tucuxi wrote:

zacariaz wrote:
Presumably this involves one disc?
Nope, the file system extends across one partition and one complete disk in a raid0 data/raid 1 metadata setup.

And you can do this with completely different size and types of discs?

tucuxi · 2022-09-10 19:45:34

zacariaz wrote:

tucuxi wrote:
zacariaz wrote:
Presumably this involves one disc?
Nope, the file system extends across one partition and one complete disk in a raid0 data/raid 1 metadata setup.
And you can do this with completely different size and types of discs?

Sure.

Cvlc · 2022-09-29 15:35:03

What I would do (I wouldn't use archinstall, although I haven't tried if it works for such a setup) :

NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
zram0       253:0    0   7,5G  0 disk  [SWAP]
nvme0n1     259:0    0   120G  0 disk  
├─nvme0n1p1 259:1    0   300M  0 part  /boot
└─nvme0n1p2 259:2    0 119,7G  0 part  
  └─root    254:0    0 119,7G  0 crypt /
                                       /.snapshots
sda         259:0    0 476,9G  0 disk 
├─sda1      259:1    0 476,9G  0 part   
  └─home    254:0    0 476,6G  0 crypt /home/bob
                                       /home/bob/.snapshots

nvme0n1 being the SSD, and sda the HDD.

Create a GPT partition table and partitions following https://wiki.archlinux.org/title/GPT_fd … ition_type

luks format all partitions except the ESP :

# cryptsetup luksFormat --label Root /dev/nvme0n1p2
# cryptsetup luksFormat --label Home /dev/sda1

Open the luks volumes

# cryptsetup open /dev/disk/by-label/Root root
# cryptsetup open /dev/disk/by-label/Home home

Format both partitions with btrfs separately

# mkfs.btrfs /dev/mapper/root
# mkfs.btrfs /dev/mapper/home

# mkfs.fat -F 32 -n ESP /dev/nvme0n1p1

Create the subvolumes that you mentioned above to avoid using the top level subvolume

# mount /dev/mapper/root /mnt

# btrfs subvolume create /mnt/@
# btrfs subvolume create /mnt/@snapshots
etc.....

# umount /mnt

# mount /dev/mapper/home /mnt
# btrfs subvolume create /mnt/@home

# umount /mnt

Mount the filesystems

# mount -o subvol=@ /dev/mapper/root /mnt
# mount --mkdir /dev/disk/by-label/ESP /mnt/boot
# mount -o subvol=@home /dev/mapper/home /mnt/home

Then pacstrap and finish installation. If you used correct partition types, you can comment out the ESP and Root partition from fstab. However, keep /home because it cannot be auto mounted as it is on a different drive.

edit /etc/crypttab as so:

# Configuration for encrypted block devices.
# See crypttab(5) for details.

# NOTE: Do not list your root (/) partition here, it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf).

# <name>       <device>                                     <password>              <options>
home		LABEL=Home	-	tpm2-device=auto

This is with TPM based decryption, but see the relevant wiki page for other ways. If the password is the same as the root partition, it will be reused for the home volume.

zacariaz · 2022-10-02 10:34:39

First of all, thank you so much for the effort. It really is very appreciated, and I'm sorry I only answer now, but haven't really had time.

Cvlc wrote:

What I would do (I wouldn't use archinstall, although I haven't tried if it works for such a setup) :
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
zram0       253:0    0   7,5G  0 disk  [SWAP]
nvme0n1     259:0    0   120G  0 disk  
├─nvme0n1p1 259:1    0   300M  0 part  /boot
└─nvme0n1p2 259:2    0 119,7G  0 part  
  └─root    254:0    0 119,7G  0 crypt /
                                       /.snapshots
sda         259:0    0 476,9G  0 disk 
├─sda1      259:1    0 476,9G  0 part   
  └─home    254:0    0 476,6G  0 crypt /home/bob
                                       /home/bob/.snapshots
nvme0n1 being the SSD, and sda the HDD.
Create a GPT partition table and partitions following https://wiki.archlinux.org/title/GPT_fd … ition_type

* I'm simply too lazy to do it manually, as I used to, but I do think it's the only solution, so I'll have to see if I can pull myself together.
* As for the suggested setup, it seems perfectly sensible, however, I am not sensible and always want to do things the hard way.
- zram: I've tried to read up on it, but I'm not really sure I understand. Is it just a compressed ramdisk? If so, it is of little use to me. What I really want, is the ability to hibernate, and speed isn't really all that important, but on the other hand, I don't want to use swap in general. I suppose I'm odd that way, but in general, if I don't need it, I don't want it. Then of course the big issue is figuring out what you actually need, but swap, at least, I don't.
- Boot/EFI: As discussed in another thread, I really think it is rather silly to EFI mount at all, unless it is actually needed, not least because there doesn't seem to be a general consensus, where it should be mounted. It should be possible to mount only when needed, and do so safely, though I haven't done much about reading up on it yet. I have a working system, and currently I need it to continue working. Anyway, the point being, that I don't see any reason to have a separate boot partition, if that is the setup, but perhaps I'm mistaken?
I don't really know how snapshots work. I get the general, but it seems to me, if possible, which I believe it is, that it would be more sensible, to have then all stored on the HDD. Again, I may well be mistaken.

Cvlc wrote:

luks format all partitions except the ESP :

# cryptsetup luksFormat --label Root /dev/nvme0n1p2
# cryptsetup luksFormat --label Home /dev/sda1

Open the luks volumes

# cryptsetup open /dev/disk/by-label/Root root
# cryptsetup open /dev/disk/by-label/Home home

Now this is where I started having some serious issues, again because I very much want to do things a certain way.
When I turn on my PC, I want to be presented with a password prompt, and not have to worry about it, except in special cases. I'd still like to be able to have multiple user account, with separately encrypted $USER. User will be prompted during boot, or preboot may be the right term, the user being determined depending on the password, automatically open $USER and login, post boot. On logout, $USER should be closed, and ideally also the root partition, provided not other users, real or otherwise are logged in, and when logging in, using the normal login prompt, be it graphical or not, all relevant encrypted devices, should again be opened automatically.
Some of this is possible, like having multiple different password for decryption of a device, and I heard that some solutions exists, when it comes to opening multiple encrypted devices, with a single password, though, from what I've read, it seems a bit dodgy.
It seems perfectly reasonable to assume that you can close a device on logout, but opening it again through the login prompt? I'm not so sure about that, nor do I know about closing root on logout, and opening it again, on login. As for separately encrypt $USER, as I understand it, that's not really and option at all, at this point in time, when using btrfs, but perhaps I am mistaken, or perhaps it has changed?
I suppose I would be willing to compromise, sort of, by effectively having to login twice, one for boot and one for actual login, but the user should not have any different experience on the second login, that you would on any system, without encryption, that is, the user should not have to worry about opening or closing their personal encrypted device, in any way, shape or form.
Well, that is the ideal, but I suspect there are good reasons... well, reasons anyway, that I haven't been ably to figure out how.

Cvlc wrote:

Format both partitions with btrfs separately
# mkfs.btrfs /dev/mapper/root
# mkfs.btrfs /dev/mapper/home

# mkfs.fat -F 32 -n ESP /dev/nvme0n1p1
Create the subvolumes that you mentioned above to avoid using the top level subvolume

Yes, that is one of the things I just can't get my head around. The purpose of different levels of subvolumes, and why the top level is apparently bad to use, or even how to avoid it.

btrfs subvolume create /mnt/@

I mean, isn't the above doing exactly the opposite?

Cvlc wrote:

# mount /dev/mapper/root /mnt

# btrfs subvolume create /mnt/@
# btrfs subvolume create /mnt/@snapshots
etc.....

# umount /mnt

# mount /dev/mapper/home /mnt
# btrfs subvolume create /mnt/@home

# umount /mnt
Mount the filesystems
# mount -o subvol=@ /dev/mapper/root /mnt
# mount --mkdir /dev/disk/by-label/ESP /mnt/boot
# mount -o subvol=@home /dev/mapper/home /mnt/home
Then pacstrap and finish installation. If you used correct partition types, you can comment out the ESP and Root partition from fstab. However, keep /home because it cannot be auto mounted as it is on a different drive.

Edit: I thought the last quote meant something else.

Cvlc wrote:

edit /etc/crypttab as so:
# Configuration for encrypted block devices.
# See crypttab(5) for details.

# NOTE: Do not list your root (/) partition here, it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf).

# <name>       <device>                                     <password>              <options>
home		LABEL=Home	-	tpm2-device=auto
This is with TPM based decryption, but see the relevant wiki page for other ways. If the password is the same as the root partition, it will be reused for the home volume.

The rest I pretty much know, except for that last part, which doesn't make sense at all, but then again, I've never heard about TPM, and on the whole, know very little about the different option. In my world it is either secure, or it is not, and I can't for the life om me comprehend why the type of encryption, would significantly affect the availability of, I think, rather essential features.

In any case, I've learned a bit, perhaps even significantly so, and am that much closer to the answer now.

Thanks a bunch and best regards.

Arch Linux

#1 2022-09-10 14:22:06

Understanding Btrfs (or not so much)

#2 2022-09-10 18:19:23

Re: Understanding Btrfs (or not so much)

#3 2022-09-10 18:43:16

Re: Understanding Btrfs (or not so much)

#4 2022-09-10 18:50:24

Re: Understanding Btrfs (or not so much)

#5 2022-09-10 19:01:40

Re: Understanding Btrfs (or not so much)

#6 2022-09-10 19:03:02

Re: Understanding Btrfs (or not so much)

#7 2022-09-10 19:07:05

Re: Understanding Btrfs (or not so much)

#8 2022-09-10 19:45:34

Re: Understanding Btrfs (or not so much)

#9 2022-09-29 15:35:03

Re: Understanding Btrfs (or not so much)

#10 2022-10-02 10:34:39

Re: Understanding Btrfs (or not so much)

Board footer