Low Level Formatting and Partitioning of drives

rollexus · 2017-08-23 15:26:24

Hello Everyone,

Was browsing the subject and came across a post by srs5694 in the following topic topic and also a blog. I would really like the community's input on the topic. Is it possible/advisable to do a low level format of the drive to 4096 byte sector size. Also, what tools should we use for proper alignment on those sectors (ie, fdisk, gparted, gidsk).

Basically I would really like your input on HDD sectoring and partitioning using the current advancements in the HDD sector (no pun intended . We are currently looking to use the M5210 in raid 10 configuration to spin 20 1TB ST1000NX0453] and would like to get everything aligned for max performance (ie, sector size vs partition size vs filesystem data structures).

It would have been nice to get those new 4K native drives but we were not sure if our server/raid/OS/vmware would be able to take advantage or even support the drive... An interesting post by seagate:
http://www.seagate.com/ca/en/tech-insig … master-ti/

Thanks in Advance.

Nick

Last edited by rollexus (2017-08-23 15:29:20)

Slithery · 2017-08-23 15:38:29

Both (c)fdisk and (c)gdisk have aligned to 4096b sector size by default for years now, there should be no need for any manual tuning.

rollexus · 2017-08-23 15:51:36

Thanks slithery for your response. Is there a way to do a low level format of the drive to present 4096 byte sized sectors to the computer (vs 8 512 bytes)?

loqs · 2017-08-25 18:13:47

rollexus wrote:

Is there a way to do a low level format of the drive to present 4096 byte sized sectors to the computer (vs 8 512 bytes)?

Why would you want to do this? My understanding is the OS would then see a 4k device so the minimum write size becomes 4k so 8 sectors would be written to the drive even if only 1 has changed.

R00KIE · 2017-08-25 21:09:38

@rollexus
You have to look in the manufacturer's page if there is a tool to do that. If I'm not mistaken some drives used to allow changing that via software with a tool that would use commands only the manufacturer knows. The caveat here is, if that is a boot drive if the bios/firmware on your machine (or the firmware in any controller card) will be able to detect the disk properly and/or boot from it.

@loqs
Even if you are writing only the 512B presented by the emulation layer, the disk itself still has to do a read-modify-write operation so I don't see much of a difference, besides most filesystems these days use (at least) 4k blocks by default.

loqs · 2017-08-25 21:21:52

@rookie I missed the devices in question were 512 emulated not 512 native. Yes the point would be mute if the any layer only worked at a level of 4k granularity.
Edit:
Although in the case of sending 8 512 writes to the device and the device not recombining them back to a single 4k write would be a worse outcome assuming less than 8 512 writes was actually needed.
But as already noted if the filesystem only sends 4k writes anyway this is a none issue.

Last edited by loqs (2017-08-25 23:30:13)

rollexus · 2017-08-28 16:38:44

@ioqs My understanding is that the filesystem with only send 4k writes. If anyone knows about the topic please elaborate. The device in question is ST1000NX0453 and from the spec sheet they seem to be 5xx native. With that in mind would there be any performance benefits to do a low level format to 4K sector size.

@rookie from what we can see the raid card (M5210) does support it and we think the machine does as well. I think today these machines should support these advanced formats as more and more machines are transitioning to ssd storage.

With that said, would there be any performance gains that could be had by going with 4k sector sizes?

Your help is greatly appreciated.

loqs · 2017-08-28 18:49:51

@rollexus see man 4 mkfs.ext4 -b option for example it will try and query the underlying block size of the device if not specified.
The underlying device in this case would be the RAID? So that passes the issue down to what the RAID controller returns to the OS as block size and if that is accurate.
I suspect you will not be certain until you have the hardware available and can pull telemetry from it. If mkfs detects 512 byte block size you can override it anyway.

R00KIE · 2017-08-28 20:10:24

If your partitions are properly aligned there should be no performance difference. By default all modern utilities align things so that it works "properly" on a 4k drive, even if the drive emulates 512B sectors.

The catch here is that the drive should report both sector sizes, the emulated sector size and the physical sector size, if you add a raid card to the mix all bets are off to what it will report. Even if it reports sane values I suppose somewhere along the way you will still have to configure something with sensible values which will most probably be different from the defaults. As an example for ext4 you probably want to adjust stride and stripe_width so that it matches your underlying raid configuration.

rollexus · 2017-09-02 13:42:57

@Ioqs: That's correct, The filesystem will be presented by the raid controller, which if I remember correctly has an option to format using 4096 byte sector size. Will look into that
@Rookie: Agreed on the sector size alignment, what interested me in the topic is that most OS filesystems use data structures that are sized in multiples of 4096 bytes anyways, So the question being not if there is a performance issue if we partition in something other than multiple of 8 sectors, the answer to this question is pretty self explanatory.... But would there be any performance benefits by just partitioning the sectors to 4096 bytes right from the beginning (ie, raid controller) and down to the OS (ie, mkfs)?

I hope this topic is not a TOTAL waste of time. I think it would be interesting to stay on top of as more and more storage devices are rolled out under the Advanced Format, and how our existing storage devices can be configured to take advantage of performance gains (if any)....

Last edited by rollexus (2017-09-02 13:43:48)

R00KIE · 2017-09-02 20:02:56

rollexus wrote:

But would there be any performance benefits by just partitioning the sectors to 4096 bytes right from the beginning (ie, raid controller) and down to the OS (ie, mkfs)?

I'd say that if you use tools that do the right thing and if you know what you are doing there should be no discernible performance difference.

Arch Linux

#1 2017-08-23 15:26:24

Low Level Formatting and Partitioning of drives

#2 2017-08-23 15:38:29

Re: Low Level Formatting and Partitioning of drives

#3 2017-08-23 15:51:36

Re: Low Level Formatting and Partitioning of drives

#4 2017-08-25 18:13:47

Re: Low Level Formatting and Partitioning of drives

#5 2017-08-25 21:09:38

Re: Low Level Formatting and Partitioning of drives

#6 2017-08-25 21:21:52

Re: Low Level Formatting and Partitioning of drives

#7 2017-08-28 16:38:44

Re: Low Level Formatting and Partitioning of drives

#8 2017-08-28 18:49:51

Re: Low Level Formatting and Partitioning of drives

#9 2017-08-28 20:10:24

Re: Low Level Formatting and Partitioning of drives

#10 2017-09-02 13:42:57

Re: Low Level Formatting and Partitioning of drives

#11 2017-09-02 20:02:56

Re: Low Level Formatting and Partitioning of drives

Board footer