Ideal bytes-per-inode ratio for a 8TB partition

HisDudeness · 2016-04-14 19:56:49

I'm willing to create a 8 TB partition for storage. I've created it without su reserved blocks

# mkfs.ext4 -m 0 /dev/sdX1

but still, 58.89 GiB were occupied with nothing written in the drive.

I understood from this post it was because of a low bytes-per-inode ratio, but the wiki page didn't clarify all my doubts

Is the only downside of a high inode-per-bytes ratio the risk of running out of inodes if many small files are written, thus being unable to write any more files?
Or, if I create a fil system with a 6 MiB ratio every file below 6 MiB will occupy 1 inode and therefore 6 MiB in my partition?
How to tell which must be the mean file size to avoid running out of inodes?

I know I'm making an awful confusion between inodes and sectors, but reading what an inode is really didn't clarify my practical doubts.

Here is the output of fdisk on my drive

# fdisk -l /dev/sdc

Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

However, creating my partition with a 6 MiB ratio resulted in just 1.12 GiB being unusable.

R00KIE · 2016-04-15 13:35:12

You might want to also look into the "resize=" option. If you don't plan to grow the partition later I believe you can save some space there too.

Regarding the bytes-per-inode ratio, form what is written on the man page you don't want to go lower than 4096 (blocksize of the filesystem). My understanding of it, and I may be wrong, is that you will need one inode per file/{hard,sym}link/directory. If you plan to store small files you need more inodes (a smaller bytes-per-inode ratio), if you plan to store large files you need less inodes (a larger bytes-per-inode ratio).

HisDudeness · 2016-04-15 14:22:23

I've probably got it now, and I feel stupid for how trivial the matter was.

Given that an inode serves to store attributes and location of a file/directory, one is needed for each of them, but it has nothing to do with minimum size a file occupies. An inode is always 128 bytes, 256 or such. A file occupies the blocks it has to fill given its dimension. If R is the bytes-per-inode ratio I give (banally, Size[bytes]/inodes), I have Size[Bytes]/R inodes, and I can allocate that much files, regardless of their dimensions. They will all occupy as much as they should.

If I have a 8TB drive and I choose a 6MiB ratio, I will have around 1271565 inodes. Not one each 6 MiB (meaning every file/folder I will put will occupy multiples of that size), but just 1271565, averaging one each 6 MiB in my drive. Banally, I will have to put files averaging 6 MiB size to avoid running out of inodes.

I feel ashamed for the idiocy I outputted in my question and I'm happy I got a grasp at it. I leave it just in case someone will be groping in the dark like me in the future (I hope not for him/her).

Thanks for the resize option, I'll look into it.

Edit: I can't find anywhere how to disable the possibility to resize, nor how much space I can save that way.

Last edited by HisDudeness (2016-04-15 14:32:52)

R00KIE · 2016-04-15 18:08:21

HisDudeness wrote:

Edit: I can't find anywhere how to disable the possibility to resize, nor how much space I can save that way.

If I'm not mistaken you pass a size to that option. The size will be the maximum size the filesystem will have, thus if you want to minimize the reserved space you pass it the size of the partition. See http://thunk.org/tytso/blog/2009/02/20/ … lock-size/

HisDudeness · 2016-04-16 19:05:28

So, if I append resize=15628053168, in my case, I should save some space?

Side question: I'm running badblocks on it to see if everything's okay or if I have to send it back before it gets too late. I've read if I specify a block size different from my drive's physical one the test could be useless. I'm doing a little bit of confusion between blocks, sectors, read... being the output of fdisk -l this

Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

block size is 4096, right? Which on of the numbers determines that, in case one is different? Physical sector size? I/O minimum size? I/O optimal?

HisDudeness · 2016-04-18 08:33:47

I have a question about the resize option: to save as much space as possible, what should I input? The device blocks number outputted by fdisk -l? What if I input more? It looks like there's no limit to the number I can give it, no preventive error message if I put an absurdly high number in there.

Last edited by HisDudeness (2016-04-18 08:50:43)

R00KIE · 2016-04-18 10:27:37

You can always give a high number. I suppose it should be limited by the filesystem maximum size.

If I remember correctly, if you give a number that is smaller than the partition you are trying to format it will complain. In the post I linked to it says 50MiB are saved for a 500GiB partition, so the savings will probably not be very large (around 800MiB if the math doesn't fail me). In the post I linked there is an example where the author uses 500G for a 500G partition, that's a starting point.

Awebb · 2016-04-18 10:51:54

Is less than 1% really a problem that needs to be fixed so urgently, that you consider creating unpredictable scenarios for the future? Don't get me wrong, I don't know what you actually know and what you don't, but you seem to be uncertain about what you are going to do with the storage. Are you even sure Ext4 is the right choice? What kinds of files will be on there?

HisDudeness · 2016-04-18 12:46:47

Mostly big files that don't need to be moved or modified, like videos, fsarchiver backups, FLACs of my audio CDs and disc images, which will compensate for photos and such.

I'd want to make it ext4 because I don't want to use any proprietary filesystem, and I'd want to eliminate all space wastes (given the use it's targeted to) so that I don't have to ever touch it a second time. That is, wanting just an external backup storage drive, leave just journaling and get rid of superuser reserved space, unnecessary inodes, the possibility to resize and such. After all, I want a single partition to fill any byte available in the drive and I've no reason whatsoever to ever resize it (I can't grow it and I don't want to shrink it to create another partition or such).

Only, I can't find how resize option should really work.

It's not that I'm depserate for 800 MiBs, but just that, you know, rather than wasting them reserving the space for a feature I'll never use I'd much better put files in them.

Last edited by HisDudeness (2016-04-18 12:48:23)

frostschutz · 2016-04-18 13:12:31

mke2fs (mkfs.ext4) has some profiles for common use cases, see /etc/mke2fs.conf or -T usage

There are some profiles for large files which is suitable if you're not going to extract a kernel source tarball on it (lots lots lots of tiny tiny tiny files).

Filesystems always have considerable overhead, it's best not to think about it too much. If you really fill it up to 100% you will see performance and fragmentation issues. Normally you don't notice fragmentation but if a single file has thousands of fragments (because the filesystem ran out of space and your file had to be squeezed in the small gaps between all the other files) it will become an issue...

HisDudeness · 2016-04-18 14:31:01

I see that a partition of 4-16 TB triggers the big filesystem type, which only changes bytes-per-inode ratio to 32768.

Is there some place where to look to find out which functions occupy space (like journaling, resize and root reserved blocks) which can be freed if the feature will not be used, and other tricks to optimize space (like changing number of inodes)?

DungeonLords · 2024-05-23 01:53:31

More details how to choose bytes-per-inode ratio here.
Ext2 handbook here.

Last edited by DungeonLords (2024-05-23 01:57:34)

Lone_Wolf · 2024-05-23 09:07:27

Please don't necrobump, see https://wiki.archlinux.org/title/Genera … bumping%22

Closing this old thread.

Arch Linux

#1 2016-04-14 19:56:49

Ideal bytes-per-inode ratio for a 8TB partition

#2 2016-04-15 13:35:12

Re: Ideal bytes-per-inode ratio for a 8TB partition

#3 2016-04-15 14:22:23

Re: Ideal bytes-per-inode ratio for a 8TB partition

#4 2016-04-15 18:08:21

Re: Ideal bytes-per-inode ratio for a 8TB partition

#5 2016-04-16 19:05:28

Re: Ideal bytes-per-inode ratio for a 8TB partition

#6 2016-04-18 08:33:47

Re: Ideal bytes-per-inode ratio for a 8TB partition

#7 2016-04-18 10:27:37

Re: Ideal bytes-per-inode ratio for a 8TB partition

#8 2016-04-18 10:51:54

Re: Ideal bytes-per-inode ratio for a 8TB partition

#9 2016-04-18 12:46:47

Re: Ideal bytes-per-inode ratio for a 8TB partition

#10 2016-04-18 13:12:31

Re: Ideal bytes-per-inode ratio for a 8TB partition

#11 2016-04-18 14:31:01

Re: Ideal bytes-per-inode ratio for a 8TB partition

#12 2024-05-23 01:53:31

Re: Ideal bytes-per-inode ratio for a 8TB partition

#13 2024-05-23 09:07:27

Re: Ideal bytes-per-inode ratio for a 8TB partition

Board footer