NVMe SSD USB enclosure TRIM support

telgar · 2019-03-03 05:30:31

I'm investigating possibilities to enable TRIM on an enclosure I just bought for NVMe SSDs. It is based on the JMS583 adapter chip, which supports TRIM by all accounts. My enclosure does not appear to support it out of the box, however:

❯ lsblk --discard /dev/sda
NAME   DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda           0        0B       0B         0
└─sda1        0        0B       0B         0

❯ sudo fstrim -v /run/media/telgar/externalssd
fstrim: /run/media/telgar/externalssd: the discard operation is not supported

This is the enclosure I have:
https://www.amazon.com/gp/product/B07JM … le_o01_s00

I couldn't find a full datasheet for the chip, but here is a brief:
http://www.jmicron.com/PDF/brief/jms583.pdf

I found this post about TRIM support for UASP enclosures:
https://bbs.archlinux.org/viewtopic.php?id=236280

There I learned these commands for checking a few relevant bits:

❯ sudo sg_readcap -l /dev/sda                                                                                        
Read Capacity results:                                                                                               
   Protection: prot_en=0, p_type=0, p_i_exponent=0                                                                   
   Logical block provisioning: lbpme=0, lbprz=0                                                                      
   Last LBA=1953525167 (0x74706daf), Number of logical blocks=1953525168                                             
   Logical block length=512 bytes                                                                                    
   Logical blocks per physical block exponent=3 [so physical block length=4096 bytes]                                
   Lowest aligned LBA=0                                                                                              
Hence:                                                                                                               
   Device size: 1000204886016 bytes, 953869.7 MiB, 1000.20 GB

❯ sudo sg_vpd -a /dev/sda
Supported VPD pages VPD page:
  Supported VPD pages [sv]
  Unit serial number [sn]
  Device identification [di]
  Block limits (SBC) [bl]
  Block device characteristics (SBC) [bdc]
  Logical block provisioning (SBC) [lbpv]
  0xde
  0xdf

Unit serial number VPD page:
  Unit serial number: DD5641988397B

Device Identification VPD page:
  Addressed logical unit:
    designator type: NAA,  code set: Binary
      0x3044564198839720
    designator type: T10 vendor identification,  code set: ASCII
      vendor id: Samsung
      vendor specific: SSD 970 EVO 1B2QDD5641988397B

Block limits VPD page (SBC):
  Write same non-zero (WSNZ): 0
  Maximum compare and write length: 0 blocks [Command not implemented]
  Optimal transfer length granularity: 8 blocks
  Maximum transfer length: 65535 blocks
  Optimal transfer length: 65535 blocks
  Maximum prefetch transfer length: 65535 blocks
  Maximum unmap LBA count: -1 [unbounded]
  Maximum unmap block descriptor count: 63
  Optimal unmap granularity: 0 blocks [not reported]
  Unmap granularity alignment valid: false
  Unmap granularity alignment: 0 [invalid]
  Maximum write same length: 0 blocks [not reported]
  Maximum atomic transfer length: 0 blocks [not reported]
  Atomic alignment: 0 [unaligned atomic writes permitted]
  Atomic transfer length granularity: 0 [no granularity requirement
  Maximum atomic transfer length with atomic boundary: 0 blocks [not reported]
  Maximum atomic boundary size: 0 blocks [can only write atomic 1 block]

Block device characteristics VPD page (SBC):
  Non-rotating medium (e.g. solid state)
  Product type: Not specified
  WABEREQ=0
  WACEREQ=0
  Nominal form factor not reported
  ZONED=0
  RBWZ=0
  BOCS=0
  FUAB=0
  VBULS=0
  DEPOPULATION_TIME=0 (seconds)

Logical block provisioning VPD page (SBC):
  Unmap command supported (LBPU): 1
  Write same (16) with unmap bit supported (LBPWS): 0
  Write same (10) with unmap bit supported (LBPWS10): 0
  Logical block provisioning read zeros (LBPRZ): 0
  Anchored LBAs supported (ANC_SUP): 0
  Threshold exponent: 0 [threshold sets not supported]
  Descriptor present (DP): 0
  Minimum percentage: 0 [not reported]
  Provisioning type: 0 (not known or fully provisioned)
  Threshold percentage: 0 [percentages not supported]

Only hex output supported
VPD page code=0xde:
 00     00 de 00 0c 4a 4d 00 00  00 00 00 00 00 00 00 00    ....JM..........

Only hex output supported
VPD page code=0xdf:
 00     00 df 00 11 4a 4d 00 00  00 00 00 00 00 00 00 00    ....JM..........
 10     00 00 00 00 00                                      .....

So the unmap command is reported as supported in the VPD, but sg_readcap (scsi READ CAPICTY(16)) returns libpme=0. I tried setting provisioning_mode="unwrap" in udev:

ACTION=="add|change", ATTRS{idVendor}=="152d", ATTRS{idProduct}=="0583", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap"
ACTION=="add|change", ATTRS{idVendor}=="152d", ATTRS{idProduct}=="0583", SUBSYSTEM=="scsi_disk", ATTR{thin_provisioning}="1"

This did set:

❯ cat /sys/devices/pci0000:00/0000:00:14.0/usb2/2-1/2-1:1.0/host2/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0/provisioning_mode
unmap

Before applying this udev rule, the result was "full". I don't pretend to know what that means. I added the "thin_provisioning" rule because in the same /sys dir, there is a file called "thin_provisioning" which is "0". The rule did not change this value, however.
With this change, there is a change in lsblk:

❯ lsblk --discard /dev/sda
NAME   DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sda           0        0B       4G         0
└─sda1        0        0B       4G         0

fstrim now reports trimming all the empty space on the filesystem every time, instead of just freed blocks.

❯ sudo fstrim -v /run/media/telgar/externalssd/
/run/media/telgar/externalssd/: 902.8 GiB (969375936512 bytes) trimmed

The full size of the storage was trimmed before I copied 28GB of stuff to it. Despite this reporting, I don't know if it is actually trimming that space.

I'm not sure where to look next. I would look at the kernel code or the driver, but I'm not sure where. Does anyone have any ideas of what to try next? Or this may be the best we can do for now. But I would like to identify the missing element and who can fix it.

d_fajardo · 2019-03-03 08:25:52

You can check the mount option perhaps for ssd optimization? Is it mounted via fstab? I use btrfs and there is an ssd option to mount a drive which optimizes it as ssd.

telgar · 2019-03-03 18:31:36

You're talking about "discard"? That just trims every block as it's deleted from the file system. You have to verify it supports trim at all first. Basically all SATA and nvme drives will. The problem is that I'm connecting this via USB in an enclosure. The adapter (bridge) chipset has to support forwarding that command, and Linux has to detect it. That's what this post is about.

A similar round of posts came out as people tried to determine which SATA enclosures properly implement trim. Now (mid last year), nvme enclosures are coming out, and we get to do it all over again.

salfdjf · 2019-03-05 23:51:52

Basically you've done everything that can be done.

I have an enclosure with the same chipset (152d:0583) and with the same issue. However, applying only the first udev rule which activates "unmap" worked for me - the first time I run fstrim the whole SSD is trimmed, but after that only the blocks of actually deleted files.

The udev rule is equivalent to the following quick test:

echo "unmap" > /sys/class/scsi_disk/9\:0\:0\:0/provisioning_mode

One way to test the working TRIM is to use this in Windows: https://github.com/CyberShadow/trimcheck

Googling more about the topic, it seems to be well known:

1) Windows has less checks in order to enable TRIM
2) Some enclosures do have a bad habit of not properly reporting LBPME
3) Yes, it was reported multiple times and no, Linux-USB devs won't change the current behavior: https://marc.info/?l=linux-scsi&m=145749400028192&w=2

One thing one could do is report the issue to the enclosure (not the chipset!) manufacturer, hoping they would fix it in the firmware.

By the way, one of the feedbacks on your Amazon link also complains about missing TRIM support.

telgar · 2019-03-06 04:25:04

Thanks for the reply. I've been reading the relevant source code (I actually found the exact place modified by the patch in the list thread), trying to figure out why DISC-GRAN is not set properly when setting provisioning_mode to "unmap". I haven't figured it out yet, but once I have details, I'll either come up with a fix or complain to the enclosure manufacturer.

I have been testing a lot of SATA enclosures as well, and none of them have properly reported the LBPME bit. I am not sure which chip is responsible for that yet; do you know if it is the adapter chip that responds to READ CAPACITY(16), or is it forwarded to the SSD controller?

That comment complaining about lack of TRIM is actually mine.

Last edited by telgar (2019-03-06 04:25:43)

reagentoo · 2021-01-25 16:50:57

And I'll put my two cents in as well.

I made this rule for myself:

ACTION=="add|change", ATTRS{idVendor}=="152d", ATTRS{idProduct}=="0580", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap", ATTR{manage_start_stop}="1"

"manage_start_stop=1" prevents growth of "Unsafe Shutdowns" in the smart info (which is zero by default for usb-devices).

Use this commands to check:

find /sys/ -name provisioning_mode -exec grep -H . {} + | sort
find /sys/ -name manage_start_stop -exec grep -H . {} + | sort
smartctl --all -d sntjmicron /dev/sdd

I have RaidSonic IB-1817M-C31 with JMicron JMS583:

# lsusb | grep 152d
Bus 006 Device 004: ID 152d:0580 JMicron Technology Corp. / JMicron USA Technology Corp. USB 3.1 Storage Device

# lsblk --discard /dev/sdd
NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO
sdd         0        4K       4G         0

If anyone has any thoughts why lsblk output doesn't match the reality, please share.

logician · 2021-03-13 07:15:55

reagentoo wrote:

And I'll put my two cents in as well.

I made this rule for myself:

ACTION=="add|change", ATTRS{idVendor}=="152d", ATTRS{idProduct}=="0580", SUBSYSTEM=="scsi_disk", ATTR{provisioning_mode}="unmap", ATTR{manage_start_stop}="1"

That udev rule worked for me (adjusting the idProduct value to 0583 in my case). Thanks!!

WorMzy · 2021-03-13 12:42:38

Mod note: closing this old topic now.

Arch Linux

#1 2019-03-03 05:30:31

NVMe SSD USB enclosure TRIM support

#2 2019-03-03 08:25:52

Re: NVMe SSD USB enclosure TRIM support

#3 2019-03-03 18:31:36

Re: NVMe SSD USB enclosure TRIM support

#4 2019-03-05 23:51:52

Re: NVMe SSD USB enclosure TRIM support

#5 2019-03-06 04:25:04

Re: NVMe SSD USB enclosure TRIM support

#6 2021-01-25 16:50:57

Re: NVMe SSD USB enclosure TRIM support

#7 2021-03-13 07:15:55

Re: NVMe SSD USB enclosure TRIM support

#8 2021-03-13 12:42:38

Re: NVMe SSD USB enclosure TRIM support

Board footer