Future Proof Disk Configuration (with Btrfs)

Miblo · 2013-09-24 19:15:39

I'm expecting delivery of a 2TB internal hard drive any time soon and planning how best to install it. The system is a simple home desktop computer with the following configuration:

`lsblk`

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0  74.5G  0 disk 
├─sda1   8:1    0    99M  0 part /boot
├─sda2   8:2    0    16G  0 part /
├─sda3   8:3    0     8G  0 part /var
└─sda4   8:4    0  50.4G  0 part /home
sdb      8:16   0 149.1G  0 disk 
└─sdb1   8:17   0   149G  0 part /Storage
sdc      8:32   0 931.5G  0 disk 
└─sdc1   8:33   0 931.5G  0 part /media/Earth
sr0     11:0    1  1024M  0 rom

/proc/mounts

rootfs / rootfs rw 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sys /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
dev /dev devtmpfs rw,nosuid,relatime,size=1022324k,nr_inodes=255581,mode=755 0 0
run /run tmpfs rw,nosuid,nodev,relatime,mode=755 0 0
/dev/sda2 / ext4 rw,noatime,data=ordered 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /sys/fs/cgroup tmpfs rw,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/bfqio cgroup rw,nosuid,nodev,noexec,relatime,bfqio 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=32,pgrp=1,timeout=300,minproto=5,maxproto=5,direct 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
mqueue /dev/mqueue mqueue rw,relatime 0 0
configfs /sys/kernel/config configfs rw,relatime 0 0
tmpfs /tmp tmpfs rw 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0
/dev/sdb1 /Storage ext3 rw,noatime,data=ordered 0 0
/dev/sda3 /var ext4 rw,noatime,data=ordered 0 0
/dev/sda1 /boot ext4 rw,noatime,data=ordered 0 0
/dev/sda4 /home ext4 rw,noatime,data=ordered 0 0
/dev/sdc1 /media/Earth ext4 rw,nosuid,nodev,relatime,data=ordered 0 0

In the future I may get a small-ish SSD for /boot and the rootfs, and keep /home and /var on a big storage drive / array. Bearing this in mind – if indeed it sounds like a good idea at all – I'm wondering if any Btrfs guys could offer me some advice on what to do with this 2TB drive when it arrives. As I understand it, I think I have two options:

Format the whole 2TB partition as Btrfs, create a subvolume each for /home and /var, copy the existing stuff from /home and /var (or add /dev/sda4 and /dev/sda3 to their intended subvolumes) and point /etc/fstab to these Btrfs subvolumes, possibly also adding /dev/sdb1 – currently mounted at /Storage – to the /home subvolume, although this is pretty much an afterthought
Format the 2TB partition – possibly as ext4 unless advised that Btrfs is up to the task – and make fstab put it in any old place (anybody know the name of a celestial body 2× the mass of Earth?). Then, when everybody and their Grandma thinks that Btrfs is as solid as a rock and I've invested in the small-ish SSD, juggle my stuff around until I have the aforementioned configuration

What do you guys think? Have I properly grasped how Btrfs works? Is it really as straightforward as it seems? All help much appreciated.

Pse · 2013-09-24 21:52:00

The first options sounds about right, if I understood correctly. You can format /dev/sda4 and /dev/sda3 as btrfs and add them to the pool or you could just create subvolumes in your new drive, copy everything over, and then use those two partitions for something else entirely.

I used btrfs for a while and only had one problem (caused by using old versions of btrfs-progs, don't do that!). IMO it's safe for general use, but YMMV. Regardless of the FS you choose, you should consider backing up all important stuff.

Edit: this may help picking the name. It's 2x the diameter of Earth, not 2x the mass, but still...: http://www.futurity.org/diamond-planet- … -of-earth/

Last edited by Pse (2013-09-24 21:55:51)

graysky · 2013-09-24 22:07:47

Subvols seems needlessly complex to me. My setup: (sda is SSD and sdb is HDD):

% lsblk -fs                 
NAME   FSTYPE LABEL  UUID                                 MOUNTPOINT
sda                                                       
├─sda1 vfat          D214-5158                            
├─sda2 ext4   boot   13471a46-65e4-22dc-bed8-2aaa4c57f071 /boot
└─sda3 btrfs  arch64 17f87892-30b4-4d96-bfea-431cd90e770f /
sdb                                                       
├─sdb1 ext4   cache  ae7d2b9f-3f6f-4f30-998d-a4055b5b85f5 /var/cache
├─sdb2 ext4   arch32 1e6d1e34-ce5d-4263-bc6d-50fefec8d893 
├─sdb3 ext4   mini   1141bad3-d2ff-4a3f-898b-f939b6641e09 
├─sdb4 ext4   fchroot 1e3bdca0-e868-42a0-96bc-4b4bd2fe4514 
├─sdb5 ext4   uchroot 18f514e5-b696-4eb4-a7d6-d493550e3d94 
├─sdb6 ext4   data   f6644741-09c3-4606-8851-0e957016c51b /mnt/data
sr0

Last edited by graysky (2013-09-24 22:11:24)

donniezazen · 2013-09-25 04:33:31

I prefer single disk with subvolumes. Subvolumes are easy to create and delete and are not recursively snapshotted which makes them pretty convenient for system restoration. sda1 is empty partition for core.img, sda2 is swap and sda3 is my single btrfs partition.

lsblk -fs
NAME  FSTYPE LABEL UUID                                 MOUNTPOINT
sda1                                                    
└─sda                                                   
sda2  swap         70234555-5679-43f6-9690-8701b415be8a [SWAP]
└─sda                                                   
sda3  btrfs        b707de08-fa14-4ced-abbd-c204dd83307b /
└─sda                                                   
sr0                                                     
[donnie@archlinux ~]$ sudo btrfs sub list /
ID 256 gen 1229 top level 5 path root
ID 258 gen 1229 top level 256 path home
ID 259 gen 1037 top level 256 path opt
ID 260 gen 1037 top level 256 path var/cache/pacman/pkg

Miblo · 2013-09-25 19:23:33

Cheers for this, guys. None of you are warning me away from Btrfs, so I will go with it. Just intrigued that you used Btrfs for a while, Pse. What made you turn back?

The only question mark lies over whether to use subvolumes or discrete partitions. I think what you say about subvolumes, donniezazen, is pretty much why I see them as one of the stand-out features of Btrfs. How are they complex for you, graysky? (Cheers for the kernel, by the way.)

Also, do you mind if I ask why you guys are only keeping your /var/cache on its own partition / subvolume, rather than the whole of /var? I'd've thought the logs at least could benefit from being in there to decrease writes to the SSD.

And cheers for the Diamond Planet link, Pse. Strictly speaking I suppose I should be after something twice the density of Earth, but diameter will do just fine. My new 2TB will be labelled 55Cancri_e.

donniezazen · 2013-09-25 19:42:31

I am using pkg as subvolume and not whole var because of ignorance. I haven't done much research on its ins and outs. My philosophy is that you want to keep bare essential in your main root subvolume. You want other non-essential subvolumes inside your primary subvolume (root in my case). When you will take snapshot of your primary subvolume it is snapshot everything except subvolumes inside it, that is, subvolumes are not snapshotted recursively. I think this streamlines your backup and restore solution. You can dynamically create and delete subvolumes, so, ideally you will be able to set it up later after installation too or anytime in future.

graysky · 2013-09-25 19:49:43

Miblo wrote:

Also, do you mind if I ask why you guys are only keeping your /var/cache on its own partition / subvolume, rather than the whole of /var? I'd've thought the logs at least could benefit from being in there to decrease writes to the SSD.

I am a bit OCD about keep my packages on btrfs/SSD. I see no reason to do it. So I mount a small ext4 (HDD) to /var/cache and I also symlink /var/log to /var/cache/log to keep jounrnalctl from rapping my SSD as well.

Pse · 2013-09-25 20:30:55

Miblo wrote:

Cheers for this, guys. None of you are warning me away from Btrfs, so I will go with it. Just intrigued that you used Btrfs for a while, Pse. What made you turn back?

I started to experience lower than usual performance in some areas: virtual machines and boot times. Lower performance is to be expected in VMs unless you disable CoW for the images, which I didn't. I was actually impressed that VMs were usable with CoW enabled (I did enable auto-defrag though). I think boot times were suffering from the fact I run prelink, which causes writes to almost all binaries once in a while (increasing fragmentation), though this is just a guess, as after leaving btrfs I found some problems with systemd-readahead that caused it to fail in my previous setup. I think what finally made me do the switch were the freezes I experienced when trying to run 'btrfs filesystem defrag' recursively. It made the computer totally unusable, in spite of niceness settings (both CPU and IO niceness). I did not investigate the causes any further so this may not be entirely related to btrfs, but the defrag process triggered it, and I haven't experienced it with other filesystems.

I do miss some features: compression, seamlessly adding or removing drives, cp --reflink, to name a few.

I should also mention that I experienced ATA errors due to a faulty connection and Btrfs handled everything wonderfully. No data loss whatsoever. The only case which resulted in data loss was related to the use of an old btrfs-progs package that corrupted some files during defragmentation. Make sure to keep it up to date.

Edit: accessing the logs through journalctl was also much slower in certain cases.

Last edited by Pse (2013-09-25 20:34:13)

Miblo · 2013-09-25 21:09:56

Ah, okay. Cheers again, guys. Maybe I'd be better off using ext4 for /var – 32GBs' worth, perhaps – with the remaining 1968GB a Btrfs partition for /home. And I think I will stick with all of /var on the one partition. I'm not touching /boot for now so will probably be better reinvestigating ext4 vs Btrfs vs ZFS vs F2FS vs whatever when I get that small-ish SSD. But, yeah, Pse, I tend not to let my system fall more than a couple of days out of date, so should always have the most recent btrfs-progs and linux-ck-kx.

Miblo · 2013-10-14 20:16:28

/var was my banana skin. I'd forgotten to take into account the quite obvious variable nature of the directory and tried to copy it to the new drive while it was live. Besides that the only minor problem I had was in my first `rsync -uvrRh` (if memory serves me right) invocation failing to preserve permissions of /home. `rsync -a` is now my friend.

% lsblk -o NAME,FSTYPE,LABEL,SIZE,MOUNTPOINT
NAME   FSTYPE LABEL      SIZE MOUNTPOINT
sda                     74.5G 
├─sda1 ext4               99M /boot
├─sda2 ext4               16G /
├─sda3 ext4                8G 
└─sda4 ext4             50.4G 
sdb                    149.1G 
└─sdb1 ext3   /Storage   149G /Storage
sdc                      1.8T 
├─sdc1 btrfs  var         32G /var
└─sdc2 btrfs  home       1.8T /home
sr0                     1024M

First impressions: I'm absolutely loving having all this space. Regarding Btrfs, it seems to be doing its thing. After copying /home to its new home it did register as taking up less space on the Btrfs partition, so compress=lzo is working. I haven't actually had to copy anything from and to Btrfs yet, so don't know about COW, but alias cp="cp --reflink=auto" is ready and waiting in my ~/.zshrc. Just must figure out how best to get ranger set up for COW. And I'm looking forward to an appropriate time when I can try seamlessly adding a device to a Btrfs partition.

Like Pse, I do seem to be getting longer boot times. It seems the Btrfs partitions take longer than the ext{3,4} to mount – 1.939s and 1.901s for /var and /home vs. 101ms and 79ms for /boot and /Storage according to `systemd-analyze blame` – although (presumably because of quiet on my kernel boot line) there's no mention of them at all. The ext{3,4} partitions, however, do get a mention on boot. All-in-all, though, I'm glad to be in the future at last.

Thank you again for your advice and reassurance, folks. Arch is far and away the simplest, clearest and best distro (and obviously OS) I've ever used.

Arch Linux

#1 2013-09-24 19:15:39

Future Proof Disk Configuration (with Btrfs)

#2 2013-09-24 21:52:00

Re: Future Proof Disk Configuration (with Btrfs)

#3 2013-09-24 22:07:47

Re: Future Proof Disk Configuration (with Btrfs)

#4 2013-09-25 04:33:31

Re: Future Proof Disk Configuration (with Btrfs)

#5 2013-09-25 19:23:33

Re: Future Proof Disk Configuration (with Btrfs)

#6 2013-09-25 19:42:31

Re: Future Proof Disk Configuration (with Btrfs)

#7 2013-09-25 19:49:43

Re: Future Proof Disk Configuration (with Btrfs)

#8 2013-09-25 20:30:55

Re: Future Proof Disk Configuration (with Btrfs)

#9 2013-09-25 21:09:56

Re: Future Proof Disk Configuration (with Btrfs)

#10 2013-10-14 20:16:28

Re: Future Proof Disk Configuration (with Btrfs)

Board footer