Arch Linux

cloverskull

Member

Registered: 2018-09-30

Posts: 212

Ideal partition layout and filesystem tweaks - help appreciated!

Hey friends,

I'm building a new rig as we speak (yay!) and am trying to decide on filesystems/partitions/etc. I could use a bit of advice, please

I have two NVME m.2 drives - Samsung 990 Pro 4TB.

What I plan to do is use 6TB for Arch, 2TB for work (aka, Windows, ugh).

This feels pretty straightforward - use LVM, to make a 6TB volume for Arch, leave 2TB for Windows. So far so good.

FYI - I'm basically following the structure/layout of https://wiki.archlinux.org/title/Ext4 to figure this out

My question is really about ext4 settings, I suppose. What would be the correct bytes-per-inode setting? My use case is one giant partition with everything (except /boot) on LVM using ext4. So, 6TB single partition with everything including /home. This rig will be for containers, AI/ML, and, of course, gaming.

What about reserve blocks?

As this is a desktop without an UPS, my assumption is that I shouldn't change the commit interval, but if it's safe to set it to 60, I may as well, right? Or should I leave that at default?

Obviously fast commit is a good plan here, I don't see any drawbacks there.

Anyone have any other thoughts?

As this drive is on a desktop in my home and my threat vector is limited, I don't intend to use any encryption.

Thanks for the advice! Cheers.

ozwigh

Member

Registered: 2014-10-07

Posts: 33

Re: Ideal partition layout and filesystem tweaks - help appreciated!

My humble opinion - why use LVM nowadays at all? ZFS if you're ok with LTS kernel (or being 1-2 releases behind the mainline) + a need to learn (a lot) or Btfrs (easy). I switched to Btrfs (and dropped LVM) a few years ago and hadn't any issues (both on the work laptop and a generic one) since (the home server/NAS is on BSD/ZFS). Plus, it's super easy to create/mount new subvolumes on need.
I'd RAID0 the drives and create a layout like this (a working example, skip crypt if you aren't paranoid enough):

nvme0n1     259:0    0  1.8T  0 disk  
├─nvme0n1p1 259:2    0  512M  0 part  /esp
└─nvme0n1p2 259:3    0  1.8T  0 part  
  └─md127     9:127  0  3.6T  0 raid0 
    └─raid  252:0    0  3.6T  0 crypt /home
                                      /ext
                                      /data
                                      /
nvme1n1     259:1    0  1.8T  0 disk  
├─nvme1n1p1 259:4    0  512M  0 part  /boot
└─nvme1n1p2 259:5    0  1.8T  0 part  
  └─md127     9:127  0  3.6T  0 raid0 
    └─raid  252:0    0  3.6T  0 crypt /home
                                      /ext
                                      /data
                                      /

There is an extensive use of Btrfs subvolumes. And... I'll run Windows under QEMU/KVM. Unless you need good GPU performance for your work (or plan gaming on Windows sometimes). I'm even using Windows DAW under QEMU on one of the laptops with the same latency on the external audio interface as on a Windows desktop DAW. I can guess that the rest of your hardware is pretty good, too. There should not be any problems. You can even use VirtualBox. On the other laptop, there are 2x4TB drives with a similar setup, and Btrfs handle it pretty well. Also, CoW filesystem is better for a PC w/o battery/UPS.
That way you'll have a common shared space without wasting anything. Also, Windows can read Btrfs (with an open-source driver), while it can't read ext4.
Of course, "RAID0 is dangerous" but ZFS/Btrfs snapshots + regular backups and monitoring drives' health + regular maintenance = zero problems. And drives will wear evenly + significant speed increase, which is good for your tasks. Well, there are not many laptops with 4 NVMe slots (and most of those are gaming ones), otherwise I'd use RAID10. /sigh
For temporary files, I'd use Zram (plain tmpfs is possible, too). Something like this:

zram0       251:0    0   16G  0 disk  [SWAP]
zram1       251:1    0   32G  0 disk  /tmp
zram2       251:2    0   24G  0 disk  /var/cache
                                      /cache
zram3       251:3    0  128G  0 disk  /ram

Sorry, I can't tell you much about ext4 tweaking, I'd stay on defaults except the block size. You'll hardly notice any performance changes on those drives. Or you may experiment with settings, 'cause there's no universal solution - something works better here, the other there. I'm not an expert on ext4 and a long time out of the loop. But all tweakings may provide less gain than nvme_core.default_ps_max_latency_us=0 kernel parameter 'cause there is no need for any power saving (or turning off VMD if you're on Intel).
As for the commit interval, I'll leave it as is (5 seconds), as it's not a laptop. Or even lower it (if possible) - on those drives you'll not notice any changes but it'll be a bit safer w/o UPS.
Of course, I don't insist on anything, but maybe you'll find some ideas interesting and useful.
Cheers!

cloverskull

Member

Registered: 2018-09-30

Posts: 212

Re: Ideal partition layout and filesystem tweaks - help appreciated!

Thanks for the advice, you've given me something to think about. I'll be honest, I was avoiding btrfs because I don't intend to use any of its unique features and it feels unnecessary if it won't provide any substantial performance benefit. I do have btrfs on my work laptop and it's working great for that purpose, but I was hoping to keep things a bit simpler with ext4. I may even skip LVM and just partition the additional nvme as additional storage media.

As far as ZFS, is there a performance benefit? My hardware is very recently released so LTS kernels are probably not going to work for me. I doubt I'd use things like snapshots so the extra features of ZFS are probably lost on me here as well

Nevertheless I really appreciate your reply and the excellent effort you put forth to write it up! I still don't have my mind fully made up so I'll consider your advice thoughtfully. Thanks!

ozwigh

Member

Registered: 2014-10-07

Posts: 33

Re: Ideal partition layout and filesystem tweaks - help appreciated!

You're welcome! I understand you don't want to overcomplicate things, keeping everything simple.
Ext4 is a good, rock-solid filesystem, and a safe choice. But btrfs compression may provide some (sometimes significant) performance benefits, though it depends on the files you work with. Also, it'll prolong the NVMe drives' life by reducing writing. With the recent hardware, you can go for zstd:9 without noticing any slowdowns. It can do raids (avoid 5/6 as they're bugged for now), even with different drive sizes, so no mdadm, no LVM, and you can have Windows(r). Plain and simple. And safe with linear "RAID". You may just ignore snapshots, subvolumes, and other features. Even turn off CoW and compression (per file/dir or at all). It's quite flexible. Not that I'm promoting btrfs, the choice is yours in the end.
As for ZFS, it's probably not worth the pain. Yes, it's super reliable and has a lot of features, but it needs tweaking and maintenance. Performance depends on the workload. I didn't do any performance tests, but I think for a typical home PC (music, video, games, etc, i.e., not very compressible data), ext4 may beat the others (though, you may not notice that on recent hardware!). And the main problem - it's out-of-the-tree, so here comes problems: stick with older kernels, sign ZFS module for Windows' "secure boot", and so on. "If you need ZFS, go FreeBSD way".
Maybe good old ext4 is the best choice for your case. Do mkfs.ext4 and forget about it.
Cheers!

stfischr

Member

Registered: 2020-04-29

Posts: 29

Re: Ideal partition layout and filesystem tweaks - help appreciated!

But btrfs ... 'll prolong the NVMe drives' life by reducing writing.

Not really in the grand scheme. You have to regularly run balance and defrag or your btrfs will become slow and eventually tell you there's no free space despite having huge amounts of free space. Balance and defrag do a lot of writing.

Ext4 or even xfs are super robust FSs that you don't need to do any maintenance for. If you want compression there is a LVM layer called VDO that provides de-duplication and compression for any filesystem. I originally wanted to use BTRFS for the transparent compression but it has too many draw backs so I will try VDO in the future.

A tip from someone that tries all the mkfs and mount options and thoroughly benches them. Go with the default unless you know that it benefits your workload. Commit interval has no performance impact unless you make it shorter than the default (makes it slower), everything that has a measurable performance impact comes with a drawback which is the reason it's not a standard option. nobarrier for example. Inodes are automatically reserved depending on the volume size. Unless you know that your files/workload will have a problem or you want to resize the filesystem to giant size at a later point go with the default, else there are usage-type settings (-T) that are better than just pulling numbers outa your ass

There is noatime (atime is basically useless these days) which you should use but it has hardly any performance impact.

One exception is, if you use a raid underneath you should tell ext4 the parameters with stride and stripe.

cloverskull

Member

Registered: 2018-09-30

Posts: 212

Re: Ideal partition layout and filesystem tweaks - help appreciated!

Awesome, thanks for the feedback folks. I do think I may read more about btrfs here, I didn't realize it had all these (what I'll call) "fringe" benefits lol. I was really just tracking it as being useful for people that want to use snapshots and COW.