You are not logged in.

#1 2022-01-08 13:35:00

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Fstrim (periodic trim) and SSD raid (raid5)

Hello,
Fstrim doesn't apparently work with RAID SSDs (soft raid5 with mdadm). I've read several articles on the subject but none were very conclusive.
I wonder whether periodic trim can be applied to SSD RAID?
If not, is it so big an issue? Is it worth it anyway?
What do you think?

Thank you
Cheers

Offline

#2 2022-01-08 14:58:26

frostschutz
Member
Registered: 2013-11-15
Posts: 1,136

Re: Fstrim (periodic trim) and SSD raid (raid5)

It should work, mdadm raid has supported trim for a long time now (maybe not in all levels and layouts, but standard ones should work). (EDIT: I was wrong, as @progandy pointed out below, there is a safety flag for the raid5 case)

How are you testing it? Any error messages in particular?

If you have LUKS in between, that's usually what prevents trim unless you open with allow discard flag.

Last edited by frostschutz (2022-01-08 19:58:37)

Offline

#3 2022-01-08 15:34:49

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Re: Fstrim (periodic trim) and SSD raid (raid5)

I am checking fstrim.timer output: journalctl -u fstrim

 
janv. 03 19:49:57 nordi systemd[1]: Starting Discard unused blocks on filesystems from /etc/fstab...
janv. 03 19:49:59 nordi fstrim[60198]: /boot : 50,2 GiB (53878419456 octets) réduits sur /dev/sdb1
janv. 03 19:49:59 nordi systemd[1]: fstrim.service: Deactivated successfully.
janv. 03 19:49:59 nordi systemd[1]: Finished Discard unused blocks on filesystems from /etc/fstab.

Only my boot partition gets trimmed. My raid /dev/md0 is mounted on /
I dont use LUKS

Offline

#4 2022-01-08 15:44:02

frostschutz
Member
Registered: 2013-11-15
Posts: 1,136

Re: Fstrim (periodic trim) and SSD raid (raid5)

Maybe it's not in /etc/fstab?

Also check lsblk, lsblk --discard

Offline

#5 2022-01-08 16:44:19

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Re: Fstrim (periodic trim) and SSD raid (raid5)

Yes, it's in fstab:
UUID=ddfcab13-4845-4728-ba5e-87860092abb3       /               ext4            defaults,relatime       0 1

I don't know how to interpret the result of lsblk though:

NAME    DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda            0      512B       2G         0
└─sda1         0      512B       2G         0
  └─md0        0       32K       2G         0
sdb            0      512B       2G         0
├─sdb1         0      512B       2G         0
└─sdb2         0      512B       2G         0
sdc            0      512B       2G         0
└─sdc1         0      512B       2G         0
  └─md0        0       32K       2G         0
sdd            0      512B       2G         0
└─sdd1         0      512B       2G         0
  └─md0        0       32K       2G         0

Does this mean, trim should work?
However,

fstrim -v /
fstrim: /: the discard operation is not supported

Offline

#6 2022-01-08 16:58:02

progandy
Member
Registered: 2012-05-17
Posts: 4,965

Re: Fstrim (periodic trim) and SSD raid (raid5)

You need to check if your ssds can handle discards in an mdadm  compatible way:
https://www.kernel.org/doc/html/latest/ … rd-support
Then add raid456.devices_handle_discards_safely=y to your kernel parameters.

Last edited by progandy (2022-01-08 17:24:32)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#7 2022-01-08 17:51:24

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Re: Fstrim (periodic trim) and SSD raid (raid5)

Yes, that's it, you found it. It's certainly disabled by default.
Last question: I don't know where to look to find "Storage devices that properly support ‘discard_zeroes_data’ " ? Is there a SSD catalogue somewhere or a command to tell us?

Offline

#8 2022-01-08 18:01:11

progandy
Member
Registered: 2012-05-17
Posts: 4,965

Re: Fstrim (periodic trim) and SSD raid (raid5)

Probably like this: https://wiki.polaire.nl/doku.php?id=linux_test_ssd_trim And then you have to trust that i doesn't behave differently in e.g. a high stress situation.

Last edited by progandy (2022-01-08 18:01:54)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#9 2022-01-08 18:08:31

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Re: Fstrim (periodic trim) and SSD raid (raid5)

OK, will do that, thank you. Works for a single device though. My raid is already assembled.

Last edited by ivanoff (2022-01-08 18:22:19)

Offline

#10 2022-01-08 19:18:51

frostschutz
Member
Registered: 2013-11-15
Posts: 1,136

Re: Fstrim (periodic trim) and SSD raid (raid5)

progandy wrote:

https://www.kernel.org/doc/html/latest/ … rd-support
Then add raid456.devices_handle_discards_safely=y to your kernel parameters.

Thanks for pointing it out, I was unaware of this change.

Even so this should be a documentation typo (discards -> discard) and would not have any effect unless specified correctly.

static bool devices_handle_discard_safely = false;
module_param(devices_handle_discard_safely, bool, 0644);
MODULE_PARM_DESC(devices_handle_discard_safely,
                 "Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");

In order to test, after enabling this flag (disable fstrim service timer beforehand for this so it does not run accidentally), you should be using a filesystem that has already been populated with data and ideally been in use for a while.

Then run an mdadm check and verify mismatch_cnt is 0.

head /sys/block/md0/md/mismatch_cnt
mdadm --action=check /dev/md0
mdadm --wait /dev/md0
head /sys/block/md0/md/mismatch_cnt

If the result for mismatch_cnt is 0, you can try your luck with discard: (back up all your files first, just in case)

fstrim -v /

Then re-run the check above and verify the mismatch_cnt is still 0.

If in doubt you can also drop caches in between (echo 3 > /proc/sys/vm/drop_caches) as the Linux cache tends to return previously cached data that has already been discarded by the storage device.

If mismatches appear, then using discard on this raid array might not be entirely safe (in a subsequent disk failure and recovery case which is after all the point of raid).

In that case you can follow it up with a repair instead of a check to restore parity, and periodically discard your data in this fashion (backup, check, discard, repair).

Or simply make do without discard. People use fstrim/discard because everyone uses it, not because they actually need it...

Last edited by frostschutz (2022-01-08 19:38:54)

Offline

#11 2022-01-09 12:44:02

ivanoff
Member
Registered: 2012-03-18
Posts: 66

Re: Fstrim (periodic trim) and SSD raid (raid5)

Or, since I can probably live without automatic trim on the raid disks, I think I will wait for the function to be stable and safe enough to be enabled by default in the kernel...
Because the fstrim can work a few times with mismatch_cnt=0 and oneday generate parity errors I suppose... so hardware manufacturers: get your thing straight!

Offline

Board footer

Powered by FluxBB