Moved to SSD, not sure if TRIM works

emes · 2014-02-13 18:57:56

Hi,

I have changed my laptop's hdd to ssd. My disk layout is not very basic:

/dev/sda1 is BIOS boot
/dev/sda2 is /boot ext2
/dev/sda3 is LUKS with LVM with 3 LVs for /, /home (xfs both) and swap

So far I have managed to enable trim on LUKS device by cryptdevice=[...]:allow-discards kernel option.
I also added "discard" option to fstab and it works for /home and /boot but root volume doesn't receive that parameter (obviously, it is mounted in initcpio, before /etc/fstab is read).

~:# mount | grep -e ^/
/dev/mapper/plecha-root on / type xfs (rw,noatime,attr2,inode64,noquota)
/dev/sda2 on /boot type ext2 (rw,noatime,discard)
/dev/mapper/plecha-home on /home type xfs (rw,noatime,attr2,discard,inode64,noquota)

~:# lsblk -D             
NAME              DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda                      0      512B       2G         0
├─sda1                   0      512B       2G         0
├─sda2                   0      512B       2G         0
└─sda3                   0      512B       2G         0
  └─plecha               0      512B       2G         0
    ├─plecha-root        0      512B       2G         0
    ├─plecha-home        0      512B       2G         0
    └─plecha-swap        0      512B       2G         0

So, do I have TRIM on root volume or not?
If not, how can I modify initcpio to mount root with discard option?
How can I make sure the LVM layer passes discards? I'm not sure how it was initialized by initcpio tools.

jasonwryan · 2014-02-13 19:37:02

Not an Installation issue, moving to Kernel & Hardware...

mychris · 2014-02-13 22:47:48

The mount options in your /etc/fstab will be applied to the / mount point. The init process remounts the root filesystem with all the options specified in /etc/fstab during the boot.
http://www.freedesktop.org/software/sys … nt-fs.html

Can you post your /etc/fstab? It looks like you don't have the discard option in it. If the FS doesn't know it should discard, the SSD can't be TRIMed automatically, because the FS is the only part which knows about free and used blocks.
The discard option for the FS mount is not enough, I think you also have to tell the LVM about it. Look into the man page of lvm.conf, maybe you find something there. Maybe the allow-discards you put into the kernel options is enough. My SSDs are not encrypted any more so I can't check it. Check the wiki for those informations, I also found it there.

emes · 2014-02-13 23:14:56

mychris wrote:

The mount options in your /etc/fstab will be applied to the / mount point. The init process remounts the root filesystem with all the options specified in /etc/fstab during the boot.

Certainly it does not. Or it tries to but fails, because manual remount with discard option doesn't work:

~:# mount | grep root
/dev/mapper/plecha-root on / type xfs (rw,noatime,attr2,inode64,noquota)
~:# mount -o remount,discard /
~:# mount | grep root         
/dev/mapper/plecha-root on / type xfs (rw,noatime,attr2,inode64,noquota)

Can you post your /etc/fstab? It looks like you don't have the discard option in it.

/dev/mapper/plecha-root	/         	xfs       	rw,discard,noatime,attr2,inode64,noquota	0 1
/dev/mapper/plecha-home	/home     	xfs       	rw,discard,noatime,attr2,inode64,noquota	0 2
/dev/sda2           	/boot     	ext2      	rw,discard,noatime	0 2
/dev/mapper/plecha-swap	  swap  swap

The discard option for the FS mount is not enough, I think you also have to tell the LVM about it. Look into the man page of lvm.conf, maybe you find something there.

I have "issue_discards = 1" in /etc/lvm/lvm.conf but the situation is similar as with fstab. The file resides on the LVM itself, so the volume group is initialized by initcpio code before the config file is read.

How can I check if a specific vg actually issues discards?

mychris · 2014-02-14 00:30:21

emes wrote:

Certainly it does not. Or it tries to but fails, because manual remount with discard option doesn't work:

Can you look into the logs if there are any errors?

emes wrote:

How can I check if a specific vg actually issues discards?

You can check the current configuration with

# lvm dumpconfig

The configuration files are stored within the kernel image, have you tried to rebuild it?

The LVM configuration should not have any effect on the FS mount, because the FS doesn't know anything about the LVM. You should be able to mount a FS with the discard option even if the device doesn't support it (at least I can mount my ext4 FS with discard on a normal HD)

What does

# fstrim -v /

tell you?

frostschutz · 2014-02-14 00:31:17

As for testing whether TRIM works, I've outlined a method here: http://unix.stackexchange.com/a/85880/30851

LVM never blocks discards. The issue_discards is for TRIM on lvremove. I set that to 0 in order to be able to recover from a wrong command. If you are sure you want to discard the space no longer in use by LVM, you can always create a temporary discard-LV with lvcreate -l100%FREE and blkdiscard that. I do not use the discard mount option either. I prefer fstrim as a weekly cron task. Works just as well without harming performance.

Yes, that means not everything is TRIM the instant it becomes free. No, that isn't a problem in a normal usage scenario.

Last edited by frostschutz (2014-02-14 00:33:40)

emes · 2014-02-14 00:56:46

So, the test fails on both filesystems. The one with discard and the one without it.
By failing I mean that I re-read the same pattern after the file had been deleted and fstrim + cache drop were issued.

fstrim behaves in a strange way:

~:# fstrim -v /    
/: 936,7 MiB (982220800 bytes) trimmed
~:# fstrim -v /
/: 936,7 MiB (982220800 bytes) trimmed
~:# fstrim -v /home
/home: 5,4 GiB (5830422528 bytes) trimmed
~:# fstrim -v /home
/home: 5,4 GiB (5830422528 bytes) trimmed

I guess that means the trim was issued but has been blocked by some layer and never reached the device?

frostschutz · 2014-02-14 01:39:24

In that case the culprit is most likely LUKS, which allows discards only if the --allow-discards option was used.

cryptsetup itself allows this option only at the time you luksOpen, i.e. with root on LUKS you'd have to make your initramfs do it.

Although it's possible to add the discards at runtime with some dmsetup foo. But you should really know what you're doing with this one: http://code.google.com/p/cryptsetup/iss … ail?id=196

Edit: Oh, you mentioned allow discards already in your first post. Check with 'dmsetup table | grep crypt' whether it's actually active or not. For each SSD backed crypt container it should say allow_discards in there somewhere.

As for fstrim it's normal to report the same size when you call it twice; it always trims all free space; the filesystem doesn't remember/care what was trimmed already.

Last edited by frostschutz (2014-02-14 01:47:09)

emes · 2014-02-14 12:12:58

~:# dmsetup table | grep crypt
plecha: 0 58353871 crypt serpent-cbc-plain 0000000000000000000000000000000000000000000000000000000000000000 0 8:3 4096 1 allow_discards

Now it all looks even more mysterious to me.
Seems like all the layers report that they support trims, but some of them is a liar.

frostschutz · 2014-02-14 17:23:35

Well, if you want to skip the layers, create a partition (or: sacrifice your swap or boot partition) and use blkdiscard on that. If that doesn't work either, you have an SSD that either does not support TRIM, or somehow returns the old data even after TRIM, a controller or bridge that does not let TRIM commands pass through it, or a kernel issue.

umount /boot
sync
dd bs=1M if=/dev/boot of=/somewhere/safe/boot.img
cmp /dev/boot /somewhere/safe/boot.img && echo identical || echo different
blkdiscard /dev/boot
echo 3 > /proc/sys/vm/drop_caches
cmp /dev/boot /somewhere/safe/boot.img && echo identical || echo different
hexdump -C /dev/boot | head
00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
dd bs=1M if=/somewhere/safe/boot.img of=/dev/boot # add conv=sparse if /dev/boot was 00 throughout

Edit: I forgot the drop caches in between again ...

Last edited by frostschutz (2014-02-14 20:31:16)

R00KIE · 2014-02-14 18:15:30

Mind you that trim may not happen immediately after you issue the trim command and ranges to trim, but the drive will know that space is not being used anymore the next time garbage collection/wear leveling reaches that space.

Arch Linux

#1 2014-02-13 18:57:56

Moved to SSD, not sure if TRIM works

#2 2014-02-13 19:37:02

Re: Moved to SSD, not sure if TRIM works

#3 2014-02-13 22:47:48

Re: Moved to SSD, not sure if TRIM works

#4 2014-02-13 23:14:56

Re: Moved to SSD, not sure if TRIM works

#5 2014-02-14 00:30:21

Re: Moved to SSD, not sure if TRIM works

#6 2014-02-14 00:31:17

Re: Moved to SSD, not sure if TRIM works

#7 2014-02-14 00:56:46

Re: Moved to SSD, not sure if TRIM works

#8 2014-02-14 01:39:24

Re: Moved to SSD, not sure if TRIM works

#9 2014-02-14 12:12:58

Re: Moved to SSD, not sure if TRIM works

#10 2014-02-14 17:23:35

Re: Moved to SSD, not sure if TRIM works

#11 2014-02-14 18:15:30

Re: Moved to SSD, not sure if TRIM works

Board footer