Whole I/O subsystem blocked by usb transfer

lynix · 2011-05-21 17:05:35

When transfering data to a usb stick (dd if=arch-cd.iso of=/dev/sdd), it seems as if the whole I/O subsystem gets a massive overload.

First the transfer rate is about 4 MB/s, then it constantly drops, under 1 MB/s. All other tasks seem to get no real I/O priority, a simple "ls /tmp" takes about a minute.
I even get these errors in dmesg:

[29960.226302] INFO: task thunderbird-bin:2249 blocked for more than 120 seconds.
[29960.226306] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[29960.226309] thunderbird-bin D ffff8801b0c15618     0  2249   2239 0x00000000
[29960.226314]  ffff8801b23d5e28 0000000000000082 ffffffff00000000 ffffffff8114ae3e
[29960.226320]  ffff8801b11f9400 ffff8801b310e230 ffff8801b23d5fd8 ffff8801b0c15340
[29960.226324]  ffff8801b23d5fd8 ffff8801b23d5fd8 ffff8801b0c15620 ffff8801b23d5fd8
[29960.226329] Call Trace:
[29960.226336]  [<ffffffff8114ae3e>] ? generic_permission+0x1e/0xc0
[29960.226340]  [<ffffffff8114a34d>] ? path_put+0x1d/0x30
[29960.226344]  [<ffffffff8114e0a4>] ? finish_open+0xe4/0x1b0
[29960.226347]  [<ffffffff8114cbb1>] ? do_path_lookup+0x71/0x130
[29960.226351]  [<ffffffff8114e6e8>] ? do_filp_open+0x188/0x760
[29960.226356]  [<ffffffff813b3825>] rwsem_down_failed_common+0xc5/0x160
[29960.226360]  [<ffffffff813b38d3>] rwsem_down_write_failed+0x13/0x20
[29960.226366]  [<ffffffff81207113>] call_rwsem_down_write_failed+0x13/0x20
[29960.226370]  [<ffffffff813b2f00>] ? down_write+0x20/0x22
[29960.226374]  [<ffffffff8110e0ea>] sys_mmap_pgoff+0x11a/0x220
[29960.226379]  [<ffffffff8101015d>] sys_mmap+0x1d/0x20
[29960.226383]  [<ffffffff8100ae12>] system_call_fastpath+0x16/0x1b

As the transfer is done via dd, the filesystem can not be the cause. The stick I used is a standard USB 2.0 flash drive, which gets recognized like this:

[29361.640311] usb 2-2: new high speed USB device using ehci_hcd and address 6
[29361.767234] scsi12 : usb-storage 2-2:1.0
[29362.766714] scsi 12:0:0:0: Direct-Access     Ut163    USB2FlashStorage 0.00 PQ: 0 ANSI: 2
[29362.767021] sd 12:0:0:0: Attached scsi generic sg4 type 0
[29362.768026] sd 12:0:0:0: [sdd] 7897087 512-byte logical blocks: (4.04 GB/3.76 GiB)
[29362.768892] sd 12:0:0:0: [sdd] Write Protect is off
[29362.768897] sd 12:0:0:0: [sdd] Mode Sense: 00 00 00 00
[29362.769705] sd 12:0:0:0: [sdd] Asking for cache data failed
[29362.769710] sd 12:0:0:0: [sdd] Assuming drive cache: write through
[29362.774057] sd 12:0:0:0: [sdd] Asking for cache data failed
[29362.774061] sd 12:0:0:0: [sdd] Assuming drive cache: write through
[29362.849510]  sdd: sdd1
[29362.853056] sd 12:0:0:0: [sdd] Asking for cache data failed
[29362.853059] sd 12:0:0:0: [sdd] Assuming drive cache: write through
[29362.853063] sd 12:0:0:0: [sdd] Attached SCSI removable disk

I got uhci_hcd and ehci_hcd loaded, kernel is kernel26-2.6.38.6-2. Any ideas what could be the cause?

Thanks,

lynix

stqn · 2011-05-22 19:31:27

No idea, but to increase the transfer speed you could try adding "bs=1M".

vadmium · 2011-05-23 07:40:05

I’ve get that laggy problem with the occasional hung task alarms on my 2.5 year old Eee PC, usually because Firefox likes writing lots of crap to the larger /home SSD partition. I’ve wondered if it might be the flash memory is wearing out and taking ages to successfully write. But I still don’t understand why Linux has to block everything else to wait for that particular request to finish.

Pierre · 2011-05-23 08:08:57

I have recently seen this effect when using dd to copy the meego image to an old usb sdrive (at about 5MB/s). Even though there was no load everything lagged; e.g. surfing the web. The system was an i7 with 16GB RAM. So it might not matter how fast the hw really is.

R00KIE · 2011-05-23 12:40:42

AFAIK that is a recurrent and old problem and so is heavy disk I/O hanging the whole system. I have experienced it myself before but lately things seems to have been working fine or at least better.

@vadmium
Firefox is or used to be a bit of a pig in that respect as it used to or still does issue a sync after all write requests so it can make systems with slower disks (flash/usb) slow down to a crawl.

pyther · 2011-11-03 01:17:00

Any updates on this? When transferring to a usb flash drive it is impossible to browse the web (firefox/chromimum). Anything that can be done to help improve performance?

karol · 2011-11-03 01:20:50

pyther wrote:

Any updates on this? When transferring to a usb flash drive it is impossible to browse the web (firefox/chromimum). Anything that can be done to help improve performance?

There are many threads on this, I don't know what have you tried already.
https://bbs.archlinux.org/viewtopic.php … 38#p998738

echo 1 > /proc/sys/vm/dirty_ratio && echo "echo 1 > /proc/sys/vm/dirty_ratio" >> /etc/rc.local

defears · 2011-11-03 01:29:45

Not a real solution, but at least made the system usable for me. Add this to /etc/security/limits.conf

username - priority -2

pyther · 2012-01-15 16:42:08

Try setting hugepage defrag to madvise

Set to madvise

echo madvise >  /sys/kernel/mm/transparent_hugepage/defrag

Disable

echo never >  /sys/kernel/mm/transparent_hugepage/defrag

See https://bbs.archlinux.org/viewtopic.php … 8#p1033648

Setting hugepage defrag to madvise has drasticly reduced the problem, if not completely solved it (further testing needed), for me!

MadCat_X · 2012-01-15 22:23:27

If you experience problems like this, try upgrading to kernel 3.2, this problem received some attention and I personally find the system to considerably more responsive under heavy I/O. You can also try Mel Gorman's patch (although it trades huge pages for responsiveness in case of many dirty pages caused by heavy I/O activity).

Last edited by MadCat_X (2012-01-15 22:31:49)

Arch Linux

#1 2011-05-21 17:05:35

Whole I/O subsystem blocked by usb transfer

#2 2011-05-22 19:31:27

Re: Whole I/O subsystem blocked by usb transfer

#3 2011-05-23 07:40:05

Re: Whole I/O subsystem blocked by usb transfer

#4 2011-05-23 08:08:57

Re: Whole I/O subsystem blocked by usb transfer

#5 2011-05-23 12:40:42

Re: Whole I/O subsystem blocked by usb transfer

#6 2011-11-03 01:17:00

Re: Whole I/O subsystem blocked by usb transfer

#7 2011-11-03 01:20:50

Re: Whole I/O subsystem blocked by usb transfer

#8 2011-11-03 01:29:45

Re: Whole I/O subsystem blocked by usb transfer

#9 2012-01-15 16:42:08

Re: Whole I/O subsystem blocked by usb transfer

#10 2012-01-15 22:23:27

Re: Whole I/O subsystem blocked by usb transfer

Board footer