External USB 3.0 Drive hanging when SMART is run

lhl · 2016-05-12 00:09:25

I've been fighting with this for a while (months) and have seem similar problems while searching, but no solution, so any suggestions would be helpful. I have an NTFS formatted 2.5" USB 3.0 Seagate Backup Plus 4TB external drive (STDR4000100) that hangs up regularly whenever I mount it with Thunar (which uses GVFS, which uses udisks2 to automount). I've confirmed via looking at the logs and running smartctl manually that a USB reset is being triggered whenever SMART is queried. This issue happens whether the drive is plugged into a USB 2.0 or USB 3.0 physical port.

My current kernel, although I've also tried the mainline w/ the same issue:

Linux ono3 4.5.3-1-ARCH #1 SMP PREEMPT Sat May 7 20:43:57 CEST 2016 x86_64 GNU/Linux

My motherboard is an Asus MAXIMUS IMPACT VII (Intel Z97 mini-ITX board) and here's the lspci for the USB controllers:

00:14.0 USB controller: Intel Corporation 9 Series Chipset Family USB xHCI Controller (prog-if 30 [XHCI])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0, IRQ 27
	Memory at f7120000 (64-bit, non-prefetchable) [size=64K]
	Capabilities: [70] Power Management version 2
	Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
	Kernel driver in use: xhci_hcd
	Kernel modules: xhci_pci

00:1a.0 USB controller: Intel Corporation 9 Series Chipset Family USB EHCI Controller #2 (prog-if 20 [EHCI])
	Subsystem: ASUSTeK Computer Inc. Device 8534
	Flags: bus master, medium devsel, latency 0, IRQ 20
	Memory at f7136000 (32-bit, non-prefetchable) [size=1K]
	Capabilities: [50] Power Management version 2
	Capabilities: [58] Debug port: BAR=1 offset=00a0
	Capabilities: [98] PCI Advanced Features
	Kernel driver in use: ehci-pci
	Kernel modules: ehci_pci

By default, the drive is detected as a UAS drive when plugged in, here's the error that happens - it ends up taking about 5-10s for the drive to remount/unhitch (where anything that's accessing the data is locked up of course). Sadly, udisks2 seems to poll SMART every 10 minutes and I haven't figured out a way to stop it (more on that in a bit). Here's what journalctl outputs:

Nov 05 01:02:19 ono2 kernel: sd 1:0:0:0: [sdb] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
Nov 05 01:02:19 ono2 kernel: sd 1:0:0:0: [sdb] tag#0 CDB: opcode=0xa1 a1 08 2e d0 01 00 4f c2 00 b0 00 00
Nov 05 01:02:43 ono2 kernel: sd 1:0:0:0: [sdb] tag#1 uas_eh_abort_handler 0 uas-tag 2 inflight: CMD IN 
Nov 05 01:02:43 ono2 kernel: sd 1:0:0:0: [sdb] tag#1 CDB: opcode=0x88 88 00 00 00 00 00 52 30 5c 20 00 00 02 00 00 00
Nov 05 01:02:43 ono2 kernel: scsi host1: uas_eh_bus_reset_handler start
Nov 05 01:02:43 ono2 kernel: usb 4-6: reset SuperSpeed USB device number 2 using xhci_hcd
Nov 05 01:02:43 ono2 udisksd[609]: Error performing housekeeping for drive /org/freedesktop/UDisks2/drives/ST4000LM016_1N2170_W8001AFE: Error updating SMART data: sk_disk_smart_read_data: Input/output error (udisks-error-quark, 0)
Nov 05 01:02:43 ono2 kernel: scsi host1: uas_eh_bus_reset_handler success

Here's an example running "smartctl --all /dev/sda -d sat" manually:

May 11 01:14:52 ono3 kernel: sd 0:0:0:0: [sda] tag#0 uas_eh_abort_handler 0 uas-tag 1 inflight: CMD IN 
May 11 01:14:52 ono3 kernel: sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x85 85 08 0e 00 d0 00 01 00 00 00 4f 00 c2 00 b0 00
May 11 01:14:52 ono3 kernel: scsi host0: uas_eh_bus_reset_handler start
May 11 01:14:52 ono3 kernel: usb 4-5: reset SuperSpeed USB device number 2 using xhci_hcd
May 11 01:14:52 ono3 kernel: scsi host0: uas_eh_bus_reset_handler success

Through my research, some suggestions to fix uas_eh_bus_reset_handler problems were were to disable UAS for the drive. I did so by adding the vendor id/model id to a /etc/modprobe.d/ignore_uas.conf blacklist:

options usb-storage quirks=0bc2:ab28:u

updating the kernel, and rebooting: "mkinitcpio -p linux && reboot"

This lets me use usb-storage instead of uas successfully:

May 11 15:32:08 ono3 kernel: usb 3-13: new high-speed USB device number 5 using xhci_hcd
May 11 15:32:08 ono3 kernel: usb 3-13: UAS is blacklisted for this device, using usb-storage instead
May 11 15:32:08 ono3 kernel: usb-storage 3-13:1.0: USB Mass Storage device detected
May 11 15:32:08 ono3 mtp-probe[10803]: checking bus 3, device 5: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-13"
May 11 15:32:08 ono3 mtp-probe[10803]: bus: 3, device: 5 was not an MTP device
May 11 15:32:08 ono3 kernel: usb-storage 3-13:1.0: Quirks match for vid 0bc2 pid ab28: 800000
May 11 15:32:08 ono3 kernel: scsi host2: usb-storage 3-13:1.0
May 11 15:32:09 ono3 kernel: scsi 2:0:0:0: Direct-Access     Seagate  BUP BK           0304 PQ: 0 ANSI: 6
May 11 15:32:09 ono3 kernel: sd 2:0:0:0: [sda] Spinning up disk...
May 11 15:32:13 ono3 kernel: ....ready

However, I get the reset errors and hiccups still (possibly less severe hangups, but still interrupts say a movie playing from the drive):

May 11 15:45:46 ono3 kernel: usb 3-13: reset high-speed USB device number 5 using xhci_hcd
May 11 15:45:46 ono3 udisksd[5057]: Error performing housekeeping for drive /org/freedesktop/UDisks2/drives/ST4000LM016_1N2170_W8001AFE: Error updating SMART data: sk_disk_smart_read_data: Input/output error (udisks-error-quark, 0)

Of course, if the issue is just with SMART being run, the obvious solution would be to disable SMART access. I made the following udev rules in /etc/udev/rules.d/60-usb-hdd.rule (related bug?):

KERNEL=="sd*[!0-9]", ATTR{removable}=="0", ENV{ID_BUS}=="usb", ENV{DEVTYPE}=="disk", ENV{ID_ATA_FEATURE_SET_SMART}="0", ENV{ID_ATA_FEATURE_SET_SMART_ENABLED}="0", ENV{UDISKS_DISABLE_POLLING}="1"
KERNEL=="sd*[!0-9]", ATTR{removable}=="0", ENV{ID_BUS}=="ata", ENV{DEVTYPE}=="disk", ENV{ID_ATA_FEATURE_SET_SMART}="0", ENV{ID_ATA_FEATURE_SET_SMART_ENABLED}="0", ENV{UDISKS_DISABLE_POLLING}="1"
KERNEL=="sd*[!0-9]", ATTR{removable}=="0", ENV{ID_BUS}=="scsi", ENV{DEVTYPE}=="disk", ENV{ID_ATA_FEATURE_SET_SMART}="0", ENV{ID_ATA_FEATURE_SET_SMART_ENABLED}="0", ENV{UDISKS_DISABLE_POLLING}="1"

I assigned variables to both disable SMART and to explicitly tell udisks to stop polling. Unfortunately, while these are applying (confirmed with a "
udevadm info --query=all --path=/sys/block/sda | grep SMART" query), udisks ignores this. Another posting seems to confirm this as a udisks2 issue - I can run "sudo smartctl --smart=off /dev/sda" and disable SMART (confimed w/ smartctl --all), which fixes the hang, but the next time udisksd polls, it renables SMART. (I am running with the latest udisks2-2.1.7-1)

While I'd be interested to see if anyone else is having issues w/ SMART requests causing disconnects, I think at this point, I'd be happy if I could just find a way to get udisks2 to stop doing a SMART request every 10 minutes for housekeeping. Any suggestions welcome, as I think I've done all I can think of here.

R00KIE · 2016-05-12 00:54:01

Welcome to the forums and congratulations on a well written first post.

From the description you provide you have already tried most of what I would suggest you could try. I'd say that disabling smart completely is not a good idea and you should only use it as a last resort workaround (if that would help, which it seems it doesn't).

One thing you can check is if there are any firmware updates for your drive. Given that you have problems even when you stop using uas it might be a firmware problem. Another thing you can try is to plug the disk in a usb2 port and see if it works or not, both with and without the uas driver.

If it still doesn't work in a usb2 port then your best bet might be to find a way to disable udisks' polling every ten minutes(1), if you can get it to work in a usb2 port you might want to consider reporting the problem in the kernel bug tracker as it might be a problem with the xhci driver.

(1) You will have to report the bug upstream if one isn't open. If there is an open bug already follow the squeaky wheel approach and add any relevant information to that bug report, I suppose that if enough people complain (with reason of course, +1 or me too posts don't help) then someone might get around to fix it sooner rather than later.

lhl · 2016-05-12 08:14:02

I just checked on the off chance that there was a firmware update - unfortunately, there isn't.

Also, I had already tried it and unfortunately, the problem seems to persist w/ both USB2 and USB3 (not sure if I tried it w/ uas or usb-storage).

It looks like the linux-usb mailing list might be the ultimate place to report on the overall issue (SMART hanging the USB) but from the threads that I followed, it's probably unlikely to give a very satisfying or timely fix.

As a proximate workaround, it looks like figuring out how to get udisks2 to stop polling should be the easiest way to go, but it looks like the udisks team is not interested and there are several untriaged-but-related bugs that are assigned to a developer that has "not reading bugmail" in his name. Oh well.

I did find one promising avenue, which is that an Ubuntu user has submitted a patch for adjusting/disabling udisks2 housekeeping on a unrelated bug. I'll be giving this a try next against a fork from the udisks2 source.

In case this starts showing up on searches if anyone is having similar issues, I have some more notes on research I've done online.

Arch Linux

#1 2016-05-12 00:09:25

External USB 3.0 Drive hanging when SMART is run

#2 2016-05-12 00:54:01

Re: External USB 3.0 Drive hanging when SMART is run

#3 2016-05-12 08:14:02

Re: External USB 3.0 Drive hanging when SMART is run

Board footer