USB file copy very slow

firewalker · 2010-10-27 10:55:21

So, is there an "official" solution? I have the same problem.

alleyoopster · 2010-11-21 08:15:37

I noticed a slow down in speeds and searching found this thread. I ran some tests and here are the results. I haven't got hectic slow speeds, but I am only getting 3MB/s write on a contiguous file, which I would say is slow for USB 2.0 Is it unreasonable to expect 12MB/s? No idea what it was before this problem.

The problem
Start copying 621MB file to stick within Thunar or CLI and it takes over 3 minutes. Starts fast and then slows after 140MB.

System Details
2.6.35-ARCH #1 SMP PREEMPT Sat Oct 30 21:22:26 CEST 2010 x86_64 AMD Turion(tm) 64 X2 Mobile Technology TL-58 AuthenticAMD GNU/Linux
Openbox using Thunar to mount

Stick
Bus 002 Device 014: ID 054c:0243 Sony Corp. MicroVault Flash Drive

usb 2-3: USB disconnect, address 13
usb 2-3: new high speed USB device using ehci_hcd and address 14
scsi17 : usb-storage 2-3:1.0
scsi 17:0:0:0: Direct-Access     Sony     Storage Media    0100 PQ: 0 ANSI: 0 CCS

Testing speeds
(rw,nosuid,nodev,uhelper=hal,uid=1000,shortname=winnt)
Copying 621MB iso to USB stick - 189 seconds (3.2 MB/s)

# dd if=/dev/zero of=/media/Quatre/tmp/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 415.704 s, 5.2 MB/s

I have tried
Mounting outside of Thunar
With mount I tried sync (slowed 0.3MB/s!)
With mount options noatime,async,nodiratime no change
Reformatted stick to ext4 - no change
CLI copying
Monitoring CPU usage - 3-5% while copying

Haven't tried
Going to try in other systems and update later
Playing with ehci_hcd

alfadude · 2010-11-30 08:55:35

I am having the same problems when copying marge files to my USB hard disks.
I noticed that when I changed the io scheduler for my drive (/sys/block/sde/queue/scheduler) to noop, the transferrate went from 5MB/s to 18MB/s.
No scheduler is better? Thats weird.
Maybe the kernel developers should take a good look at this problem.

duendetuc · 2011-02-23 22:51:31

Same problem here

firewalker · 2011-02-24 08:28:40

I have noticed that the problem exists with memory sticks that aren't USB High Speed certified. With certified devices it is as fast as it can (for my pc at least).

ohadbasan · 2011-04-01 15:22:03

I am experiencing the same problem for the past 3 years which I've been using linux.
I'm using a sandisk cruzer. if that's not hi-speed certified then what is?

firewalker · 2011-06-23 20:09:39

I noticed today that it has to do with FAT-32 file system. I had a new 320 gbyte FAT-32 usb disk and the speed was 0~5 mbyte/sec. I formated the disk to EXT-3 and the speed is now 20~22 mbytes/sec.

Can someone else verify it?

Sayo · 2011-07-13 00:19:33

Started happening to me as well. Tried a copy from EXT3 -> EXT3, EXT3 -> NTFS and NTFS -> NTFS. It copies around 300~500 MB and almost hangs after that. I've checked the logs and didn't find anything that could indicate a problem.

What can I do to see better what's happening?

thisoldman · 2011-07-13 03:41:30

@Sayo, 'iotop' might be a useful tool. It is much like 'top' but for I/O.

I've just read again all the posts in this very long thread. I have both USB hard drives and flash drives. I haven't experienced any slowdowns or long pauses with file transfers to or from USB devices. I don't know if my setup would work for other people – I'm just an Arch user, not a linux guru by any means.

My internal hard drive partitions are all jfs or ntfs, except for /boot, which is ext2. My flash drives are always vfat and I use ntfs, ext3 or jfs for USB hard drive partitions. I get 5+ MB/s when transferring a 700 MB file from internal drives to flash drives and 20+ MB/s transferring to USB hard drives.

I always mount USB flash drives with the 'flush' option. I also use 'elevator=deadline' on the kernel line in /boot/grub/menu.lst:

kernel /vmlinuz26 root=/dev/disk/by-uuid/19dfef2c-fe93-4d43-b5ab-d9493f7c504e ro vga=795 elevator=deadline

I've seen some users' stories on the internet that suggest changing the kernel's I/O scheduler from 'cfq' to 'deadline' or even 'noop' can improve USB transfer rates. See these articles for some more information:
http://www.wlug.org.nz/LinuxIoScheduler
http://blog.nexcess.net/2010/11/07/chan … scheduler/

Viper_Scull · 2011-07-13 10:13:03

Deadline scheduler is not the best choice for HDDs.

You can switch USB's I/O scheduler once mounted instead of using deadline for every disk.

echo deadline > /sys/block/sdc/queue/scheduler

I'd go with noop though for USBs and SSDs.

Last edited by Viper_Scull (2011-07-13 10:16:31)

Xemertix · 2011-07-14 10:28:22

Same problem here..when copying a big file (e.g. 1gb) to a ntfs usb flash drive (corsair flash voyager etc.) kde4 UI freezes completely after few seconds and the flash drive activity light is on..then kde unfreezes when this slow copy is finished..I noticed high I/O activity.
I'm using 2.6.39.2-zen kernel,with bfq scheduler.

http://forums.gentoo.org/viewtopic-t-79 … cb1e66c1e8

https://bugs.launchpad.net/ubuntu/+sour … bug/541937

Last edited by Xemertix (2011-07-14 10:47:42)

smakked · 2011-08-05 02:57:16

This issue still exists? Well in Arch it does, USB transfer speeds are shocking average about 2 meg a second, yet on Fedora on the same machine we get about 8 meg a second?
Any solutions for this yet.

neoanima · 2011-08-24 10:58:34

any one fixed this yet?

Clouseau · 2011-09-07 12:28:25

I also have the problem and it exists for cca 9 months. Very frustrating and it doesn't have anything to do with the computer because on another computer and a laptop the behaviour is the same... https://bbs.archlinux.org/viewtopic.php?id=110590

Kardell · 2011-09-10 18:26:27

For problems with my USB controllers I just fiddle with corresponding usb modules ehci, uhci, removing ones responsible for slower transfer rates and then remount devices, so I no longer see 960kB/s speed and 3h to copy dummy things.

Clouseau · 2011-09-12 11:12:40

Yeah, that's the first thing I tried, before 9 months, the problem still persists...

dmesg | grep uhci = nothing
dmesg | grep ohci = nothing
dmesg | grep ehci

[ 1.277109] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 1.277124] ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 1.277142] ehci_hcd 0000:00:1a.0: setting latency timer to 64
[ 1.277144] ehci_hcd 0000:00:1a.0: EHCI Host Controller
[ 1.277151] ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3
[ 1.277179] ehci_hcd 0000:00:1a.0: debug port 2
[ 1.281083] ehci_hcd 0000:00:1a.0: cache line size of 32 is not supported
[ 1.281091] ehci_hcd 0000:00:1a.0: irq 16, io mem 0xf99f8000
[ 1.297494] ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[ 1.297793] ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23
[ 1.297857] ehci_hcd 0000:00:1d.0: setting latency timer to 64
[ 1.297862] ehci_hcd 0000:00:1d.0: EHCI Host Controller
[ 1.297874] ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6
[ 1.297908] ehci_hcd 0000:00:1d.0: debug port 2
[ 1.301834] ehci_hcd 0000:00:1d.0: cache line size of 32 is not supported
[ 1.301855] ehci_hcd 0000:00:1d.0: irq 23, io mem 0xf99fa000
[ 1.314195] ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[ 1.607387] usb 3-1: new high speed USB device number 2 using ehci_hcd
[ 1.837255] usb 6-1: new high speed USB device number 2 using ehci_hcd
[ 2.040693] usb 3-1.1: new high speed USB device number 3 using ehci_hcd
[ 2.227265] usb 6-1.1: new high speed USB device number 3 using ehci_hcd
[ 2.383745] usb 6-1.5: new full speed USB device number 4 using ehci_hcd
[ 2.537039] usb 6-1.7: new high speed USB device number 5 using ehci_hcd
[ 2.900241] usb 6-1.7.2: new low speed USB device number 6 using ehci_hcd
[ 3.089993] usb 6-1.7.4: new low speed USB device number 7 using ehci_hcd
[ 3.283280] usb 6-1.7.6: new full speed USB device number 8 using ehci_hcd
[ 3.443179] usb 6-1.7.6: new high speed USB device number 9 using ehci_hcd
[ 50.024493] usb 3-1.2: new high speed USB device number 4 using ehci_hcd

Last edited by Clouseau (2011-09-12 11:13:09)

nathan28 · 2011-09-12 22:34:27

Going to add my n=1 to this.

I have slow transfer rates on large files to USB flash disks on two machines (one is RIP now) with Arch since the 3.0 kernel... [edited]

OTOH, writing to and from USB HDD seems fine.

For the hell of it, I am showing ehci and uhci USB devices in dmesg.

EDIT/UPDATE

I have played around a little with this.

I mount with vanilla automagic udev rules from the wiki. Trying a couple times with a $time cp, Copying 3+ GB was half the speed (15 MB/sec) of copying 1.5GB (29 MB/sec) from FAT32 flash to ext4 HDD. But I've had +8GB transfers to the disk take hours, just don't have any numerical data there. Another identical USB disk formated to UFS in FreeBSD would sometimes slow down to under 60 kiloByte/sec.

Last edited by nathan28 (2011-09-19 15:26:54)

thisoldman · 2011-09-25 03:25:43

An article on lwn.net from February 2011, by Arnd Bergmann, a kernel developer, says that a cure for slow transfers won't be seen soon: "Optimizing Linux with cheap flash drives" – http://lwn.net/Articles/428584/.

After reading the article and comments, my impession is that FAT may be the best filesystem for most small sticks and cards.

sirocco · 2011-10-04 04:20:42

Some solution for smp systems
http://ag-up.com/?tag=copy-commander

pocus · 2011-11-14 07:30:52

I had the same problem and I managed to fix it by adding the flush option when mounting the device.

Apparently, this issue seems to be related to fat/vfat filesystems.
http://kernelnewbies.org/Linux_2_6_19#h … 9499d0912e

/dev/zero · 2011-11-14 07:42:27

Thanks for the hint; it has also seemed to me like my FAT usb takes forever. Now I will know to give "-o flush" a go.

If only Thunar's volmanrc provided opportunities to force the option ...

Convergence · 2011-11-24 18:50:15

I think that I'm experiencing this problem as well. I automatically mount sdcards and thumb drives with KDE's handy little USB device manager. I modified this UDEV rule: https://wiki.archlinux.org/index.php/Ud … SB_devices to use the flush option, since that seems to be the suggested option here. Here is the rule:

KERNEL!="sd[a-z][0-9]", GOTO="media_by_label_auto_mount_end"

# Import FS infos
IMPORT{program}="/sbin/blkid -o udev -p %N"

# Get a label if present, otherwise specify one
ENV{ID_FS_LABEL}!="", ENV{dir_name}="%E{ID_FS_LABEL}"
ENV{ID_FS_LABEL}=="", ENV{dir_name}="usbhd-%k"

# Global mount options
ACTION=="add", ENV{mount_options}="relatime,flush"
# Filesystem-specific mount options
ACTION=="add", ENV{ID_FS_TYPE}=="vfat|ntfs", ENV{mount_options}="$env{mount_options},utf8,gid=100,umask=002"

# Mount the device
ACTION=="add", RUN+="/bin/mkdir -p /media/%E{dir_name}", RUN+="/bin/mount -o $env{mount_options} /dev/%k /media/%E{dir_name}"

# Clean up after removal
ACTION=="remove", ENV{dir_name}!="", RUN+="/bin/umount -l /media/%E{dir_name}", RUN+="/bin/rmdir /media/%E{dir_name}"

# Exit
LABEL="media_by_label_auto_mount_end"

I then confirmed that my sdcard was being mounted with the flush option:

$ cat /etc/mtab | grep -i flush
/dev/sdd1 /media/usbhd-sdd1 vfat rw,relatime,flush,utf8,gid=100,umask=002 0 0

However, I'm still getting slow transfer times, (and as far as I can remember always have under linux).

I'm transferring medium sized files (10-50 megs) 10-100 at a time to a class 10 sdcard (which is fairly new, less than half full, and should write pretty fast.)

It seems like the transfer takes place in jumps and starts. It will quickly transfer a few megs, slow down and then stop. Then it will wait for a while (during this time the card reader continues to flash suggesting that it is still in fact working) and start the process over again.

What causes this cycle of fast, slow, stop, start-over? Maybe after this transfer is done, I'll format an sdcard with a different FS, and see if I experience the same problems.

Edit:
So I did format an older sd card with ext2. It is a smaller disk; only 2 gigs, but I think that it was enough to confirm that this is indeed a vfat problem. File transfers were much faster. I transferred 650 megs in less than 30 seconds. Too bad that my camera, and pmp won't read ext2!

By the way: I was frustrated to find that my normal user didn't have write permission to write to this disk. I had to use root to make the copy. Duh!

Edit2: (Straying from the topic a little) This would make a good argument to convince the rockbox devs to support ext*! Their reason for not adding ext functionality is that very few people would be interested, and that everyone would have to pay the penalty in memory usage. However, this problem effects all Linux users (I think) which probably make up a good proportion of their user base.

Last edited by Convergence (2012-01-24 16:14:43)

quasifilmie · 2011-11-25 02:15:05

I just formatted my 4 GB kingston usb to ntfs and I have noticed improvement. Before the copying would go really fast and that the file transfer (using gnome) would just hang at the end for few a minutes. Now it doesn't hang at the end of but when I go to safely remove the drive that hangs and it says files are being transferred. One thing that has not changed is that during the course of copying a big file the transfer will continuously go from say 22 mb/s to 13 mb/s. Quite odd. I am also using the the new kernel from testing so maybe this is the reason for the change. Anyone noticed a change when copying large files to thumb drive with the new kernel?

chuckiv · 2015-05-01 20:14:48

I am getting 100-700 KiB/s. It is very slow and very inconsistent. Is there still no solution 4 years later?? Is it a GUI issue? Will terminal go faster? Or is this Arch issue?

firekage · 2015-05-01 21:44:40

chuckiv wrote:

I am getting 100-700 KiB/s. It is very slow and very inconsistent. Is there still no solution 4 years later?? Is it a GUI issue? Will terminal go faster? Or is this Arch issue?

Try in terminal. I ude KDE and also found that sometimes copy is not slow...but impossible because when copying files takes place all DM stops to respond and have to kill X (i checked usb hard disk drives with usb 3.0, not only pendrives). I tried even to install other file managers like gnome, thunar, pacmanfm and with dolphin i saw this problem. Luckily, it is not so consistant over and over.

Last edited by firekage (2015-05-01 21:45:06)

Arch Linux

#101 2010-10-27 10:55:21

Re: USB file copy very slow

#102 2010-11-21 08:15:37

Re: USB file copy very slow

#103 2010-11-30 08:55:35

Re: USB file copy very slow

#104 2011-02-23 22:51:31

Re: USB file copy very slow

#105 2011-02-24 08:28:40

Re: USB file copy very slow

#106 2011-04-01 15:22:03

Re: USB file copy very slow

#107 2011-06-23 20:09:39

Re: USB file copy very slow

#108 2011-07-13 00:19:33

Re: USB file copy very slow

#109 2011-07-13 03:41:30

Re: USB file copy very slow

#110 2011-07-13 10:13:03

Re: USB file copy very slow

#111 2011-07-14 10:28:22

Re: USB file copy very slow

#112 2011-08-05 02:57:16

Re: USB file copy very slow

#113 2011-08-24 10:58:34

Re: USB file copy very slow

#114 2011-09-07 12:28:25

Re: USB file copy very slow

#115 2011-09-10 18:26:27

Re: USB file copy very slow

#116 2011-09-12 11:12:40

Re: USB file copy very slow

#117 2011-09-12 22:34:27

Re: USB file copy very slow

#118 2011-09-25 03:25:43

Re: USB file copy very slow

#119 2011-10-04 04:20:42

Re: USB file copy very slow

#120 2011-11-14 07:30:52

Re: USB file copy very slow

#121 2011-11-14 07:42:27

Re: USB file copy very slow

#122 2011-11-24 18:50:15

Re: USB file copy very slow

#123 2011-11-25 02:15:05

Re: USB file copy very slow

#124 2015-05-01 20:14:48

Re: USB file copy very slow

#125 2015-05-01 21:44:40

Re: USB file copy very slow

Board footer