mdadm requires the array to be in sync at all times. If your drive returns random data after discard, it's not in sync.
To work around this, md would have to keep track of discarded regions in metadata, check parity with every write (at the cost of performance), get rid of optimizations that are based on the assumption that the array is always in sync (always recalculate parity from scratch instead of just updating it based on old data parity).
Comment in sourcecode drivers/md/raid5.c:
/*
* zeroing is required, otherwise data
* could be lost. Consider a scenario: discard a stripe
* (the stripe could be inconsistent if
* discard_zeroes_data is 0); write one disk of the
* stripe (the stripe could be inconsistent again
* depending on which disks are used to calculate
* parity); the disk is broken; The stripe data of this
* disk is lost.
*
* We only allow DISCARD if the sysadmin has confirmed that
* only safe devices are in use by setting a module parameter.
* A better idea might be to turn DISCARD into WRITE_ZEROES
* requests, as that is required to be safe.
*/
If in doubt, run a raid check, and verify mismatch_cnt = 0.
If there are mismatches, data will be corrupted on any rebuild - and rebuilds also happen after power loss, etc, not just on disk failure, so. In a way, it's worse than raid 0.
]]>https://www.kernel.org/doc/html/latest/ … rd-support
Then add raid456.devices_handle_discards_safely=y to your kernel parameters.
Thanks for pointing it out, I was unaware of this change.
Even so this should be a documentation typo (discards -> discard) and would not have any effect unless specified correctly.
static bool devices_handle_discard_safely = false;
module_param(devices_handle_discard_safely, bool, 0644);
MODULE_PARM_DESC(devices_handle_discard_safely,
"Set to Y if all devices in each array reliably return zeroes on reads from discarded regions");
In order to test, after enabling this flag (disable fstrim service timer beforehand for this so it does not run accidentally), you should be using a filesystem that has already been populated with data and ideally been in use for a while.
Then run an mdadm check and verify mismatch_cnt is 0.
head /sys/block/md0/md/mismatch_cnt
mdadm --action=check /dev/md0
mdadm --wait /dev/md0
head /sys/block/md0/md/mismatch_cnt
If the result for mismatch_cnt is 0, you can try your luck with discard: (back up all your files first, just in case)
fstrim -v /
Then re-run the check above and verify the mismatch_cnt is still 0.
If in doubt you can also drop caches in between (echo 3 > /proc/sys/vm/drop_caches) as the Linux cache tends to return previously cached data that has already been discarded by the storage device.
If mismatches appear, then using discard on this raid array might not be entirely safe (in a subsequent disk failure and recovery case which is after all the point of raid).
In that case you can follow it up with a repair instead of a check to restore parity, and periodically discard your data in this fashion (backup, check, discard, repair).
Or simply make do without discard. People use fstrim/discard because everyone uses it, not because they actually need it...
]]>I don't know how to interpret the result of lsblk though:
NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda 0 512B 2G 0
└─sda1 0 512B 2G 0
└─md0 0 32K 2G 0
sdb 0 512B 2G 0
├─sdb1 0 512B 2G 0
└─sdb2 0 512B 2G 0
sdc 0 512B 2G 0
└─sdc1 0 512B 2G 0
└─md0 0 32K 2G 0
sdd 0 512B 2G 0
└─sdd1 0 512B 2G 0
└─md0 0 32K 2G 0
Does this mean, trim should work?
However,
fstrim -v /
fstrim: /: the discard operation is not supported
Also check lsblk, lsblk --discard
]]>
janv. 03 19:49:57 nordi systemd[1]: Starting Discard unused blocks on filesystems from /etc/fstab...
janv. 03 19:49:59 nordi fstrim[60198]: /boot : 50,2 GiB (53878419456 octets) réduits sur /dev/sdb1
janv. 03 19:49:59 nordi systemd[1]: fstrim.service: Deactivated successfully.
janv. 03 19:49:59 nordi systemd[1]: Finished Discard unused blocks on filesystems from /etc/fstab.
Only my boot partition gets trimmed. My raid /dev/md0 is mounted on /
I dont use LUKS
How are you testing it? Any error messages in particular?
If you have LUKS in between, that's usually what prevents trim unless you open with allow discard flag.
]]>Thank you
Cheers