USB file copy very slow

danielomen · 2009-11-03 18:10:30

I have exactly the same problems as everyone above describes and worse. Plugging in a known working uvc webcam completely crashes the usb bus so i need to reboot if i want to use anything usb, and it isn't limited to arch. trying fedora 12 beta i686 & x86_64 is no different.
I will file a bug report at kernel.org as everything is fine with my samsung nc10

For reference my machine is arch64 dual AMD Athlon 7750
southbridge chipset is AMD SB700

That sound similar to anyone else?

Evzin · 2009-11-05 10:12:09

It seems that the problem occurs when HAL mounts stick with "sync" option. When i mount it manually without that option everything seems works ok... Also i found following workaround which disables sync option for hotplugged devices:
(switch to root)
1. mkdir -p /usr/share/hal/fdi/policy/95userpolicy/
2. Create file /usr/share/hal/fdi/policy/95userpolicy/nosync.fdi with following content:

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">   
  <device>   
    <!-- disable sync for mount -->   
    <match key="block.is_volume" bool="true">   
      <match key="volume.fsusage" string="filesystem"> 
        <merge key="volume.policy.mount_option.sync"
type="bool">false</merge>
      </match>   
    </match>   
  </device>   
</deviceinfo>

3. /etc/rc.d/hal restart
Just tested it. Seems works.

P.S. You can disable sync for sepcific volume UUID using: <match key="volume.uuid" string=".........."> .... </match>

Zaxariadis · 2009-11-05 10:39:28

What is sync option, and what are the risks of disabling it ?

seiichiro0185 · 2009-11-05 10:56:10

with sync enabled the data is written to the device immideately, without it is cached in RAM frist. if you allways unmount your devices correctly there should be no problem with disabling sync

Zaxariadis · 2009-11-05 11:25:14

Tried it using the UUID option and worked fine with my 8GB SONY
but how can I use this option when i need to mount manually some other USB stick?

I don't wont all my external devices to be mounted without the "sync" option but I do wont to have the choice to umount and remount them manually without that option.

Any suggestions?

Evzin · 2009-11-05 15:40:08

Zaxariadis wrote:

but how can I use this option when i need to mount manually some other USB stick?

Use -o async option. If u need to remount already mounted volume u could do mount -o remount,async ....

Solid1986Snake · 2009-11-11 20:05:18

Hi,

when I type "mount" with my stick inserted I get following:

/dev/sdb on /media/EMTEC type vfat (rw,nosuid,nodev,uhelper=hal,uid=1000,utf8,shortname=mixed,flush)

seems not the right options for my usb stick? Am I right? How could I tell hal to mount with the correct options?

mezcal · 2009-11-13 11:59:06

Hi,

I have had the same problem.
It was necessary to allow USB2.0 support in a BIOS.
Working with USB stick is quite fast now.

Garret · 2009-11-14 18:10:28

I created the file /usr/share/hal/fdi/policy/95userpolicy/nosync.fdi with the uuid of my 8gb usb kingston stick.

<?xml version="1.0" encoding="UTF-8"?>
<deviceinfo version="0.2">   
  <device>   
    <!-- disable sync for mount -->   
    <match key="block.is_volume" bool="true">   
      <match key="volume.fsusage" string="filesystem"> 
      <match key="volume.uuid" string="F601-66A4"> F601-66A4 </match>
        <merge key="volume.policy.mount_option.sync"
type="bool">false</merge>
      </match>   
    </match>   
  </device>   
</deviceinfo>

I restarted hal but nothing has changed

Mountainjew · 2009-11-15 03:30:05

danielomen wrote:

I have exactly the same problems as everyone above describes and worse. Plugging in a known working uvc webcam completely crashes the usb bus so i need to reboot if i want to use anything usb, and it isn't limited to arch. trying fedora 12 beta i686 & x86_64 is no different.
I will file a bug report at kernel.org as everything is fine with my samsung nc10
For reference my machine is arch64 dual AMD Athlon 7750
southbridge chipset is AMD SB700
That sound similar to anyone else?

I'm having the exact same issue. My motherboard also uses the SB700 chipset. Been trying to solve this and low throughput/high cpu issues for months. Getting nowhere. Tried many different kernels. Using zen-stable at the minute and still got the same issues.

Btw i couldn't get that HAL workaround to work either, made no difference here.

Oh i was watching the log viewer too, and my usb stick mounts as usb 2.

Edit: It's definately a kernel issue. I just installed the lts 2.6.27 kernel and the usb transfer speeds are great again, and my webcam doesn't kill my usb controller :)

Edit2: Meh, it's still slow doing large file transfers.

Last edited by Mountainjew (2009-11-15 14:11:26)

Solid1986Snake · 2009-11-17 08:05:11

Well,

I switched to kernel26zen stable (package from aur with own config) and am using the mentioned hal rule above (shouldn't it be copied to /etc/hal/fdi/policy ?).

Usb copy went incredibly fast now! 1,3 Gb in 5 Minutes to the same stick that needed more than half an hour for 800 Mb before...

Maybe try on your own

iphitus · 2009-11-17 10:42:42

Check the mount options (output of 'mount'). If you see 'sync' - that's why.

Remount them with async by doing 'mount /dev/sdxx -o remount,async' and things should perform much better.

Garret · 2009-11-17 11:43:31

iphitus wrote:

Check the mount options (output of 'mount'). If you see 'sync' - that's why.
Remount them with async by doing 'mount /dev/sdxx -o remount,async' and things should perform much better.

I gave:

[garret@desktop ~]$ mount
/dev/sda2 on / type ext4 (rw)
none on /dev type tmpfs (rw,relatime,mode=755)
none on /proc type proc (rw,relatime)
none on /sys type sysfs (rw,relatime)
none on /dev/pts type devpts (rw)
none on /dev/shm type tmpfs (rw)
/dev/sda1 on /windows type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096)
gvfs-fuse-daemon on /home/garret/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=garret)
/dev/sdb1 on /media/ENZO type vfat (rw,nosuid,nodev,uhelper=devkit,uid=1000,gid=100,shortname=mixed,dmask=0077,utf8=1,flush)
[garret@desktop ~]$ sudo mount /dev/sdb1 -o remount,async
Password: 
[garret@desktop ~]$ mount
/dev/sda2 on / type ext4 (rw)
none on /dev type tmpfs (rw,relatime,mode=755)
none on /proc type proc (rw,relatime)
none on /sys type sysfs (rw,relatime)
none on /dev/pts type devpts (rw)
none on /dev/shm type tmpfs (rw)
/dev/sda1 on /windows type fuseblk (rw,nosuid,nodev,allow_other,blksize=4096)
gvfs-fuse-daemon on /home/garret/.gvfs type fuse.gvfs-fuse-daemon (rw,nosuid,nodev,user=garret)
/dev/sdb1 on /media/ENZO type vfat (rw,nosuid,nodev,uhelper=devkit,uid=1000,gid=100,shortname=mixed,dmask=0077,utf8=1,flush)
[garret@desktop ~]$

neoanima · 2009-12-02 14:51:20

I have this problem too. But I didn't try the solutions, I wish a better fix of this bug.

ok, just now, i tried that policy sulution, but no luck.

i copied a 240Mb file to usb, the transfer dialog closed within about 30secs, but the light of usb kept blinking and i can't umount the usb.

Last edited by neoanima (2009-12-04 13:09:45)

mechmg93 · 2009-12-13 07:15:11

After months of trying to find a solution, i still have a huge problem with copying to my external usb disk.

When i copy a file for example of 900mb it shows 99% in an normal amount of time and then it stays for minutes in this state till it finishes. There are some times that it never finishes.

I did some tests with parted live cd. I tried to do the exact same copy and i had no problem. So its is arch specific. Up to now i am dowloading sidux live cd in order to do another one test..

edit:

after some tests with both my usb flash and usb disk, there is a possibility that the disk is broken, so more tests needed in order to understand what is happening.

Last edited by mechmg93 (2009-12-13 07:22:06)

lilsirecho · 2009-12-13 21:24:40

Do you have an enclosure for the USB disk or is it a flash unit?

Perhaps it is an external hdd?

Confused as to Usb flash versus Usb disk!!!

Godlikearg · 2009-12-17 01:51:52

I doubt that this would be a HAL problem, as it was reported the same problem with dd writing directly to the device file.

I also suffer this problem when using dd to write to a partition of an external USB hard drive (NOT flash drive). Kernel bug?

Rasi · 2010-01-04 01:07:44

i notice the same issue on my box and dug a bit deeper.
dmesg shows this line for me:

[ 310.720036] usb 1-2: new high speed USB device using ehci_hcd and address 3
[ 314.008267] hub 1-0:1.0: unable to enumerate USB device on port 2

and then continues to register the drive with ohci instead.

that got me interested and i tried it with an ubuntu live cd (linux mint helena actually): exact same issue. Right, defect USB hub? lets try an older arch. found a livecd with kernel 2.6.24 and guess what? all worked as it should.

tavianator · 2010-01-04 01:55:39

The reluctancy to unmount is caused by linux's default extensive caching. Run "sync" before you unmount; that will wait until all I/O operations have been flushed, or mount with -o sync. In terms of the kernel bug, it may be helpful to try kernel26-lts (an older kernel) or kernel26-git from AUR (a much newer kernel).

Rasi · 2010-01-04 12:41:10

Update: this is definately odd.

i tried to modprobe -r ehci_hcd to see if the error goes away. It does not.

BUT:

if i modprobe -r both modules (ohci and ehci) and then load ohci again, everything works as expected, including fast transfer rates...

Luis Sousa · 2010-01-14 19:53:52

Same problem here. I have 2 USB pendrives, 4Gb & 8Gb, and the copying times for a file of 400Mb goes up to 8min on each one. Tried same file on Ubuntu (i have 2 partitions, Ubuntu+Arch) and it went smoothly, 2min max. Tried to use async option manually and it did no difference at all, with sync option it was a LOT worse (my default is async). Bug in Arch?

Puppybasher · 2010-02-09 07:44:02

This problem remains unsolved, I am experiencing this problem as we speak

flamelab · 2010-02-09 07:47:37

No problem here with the stock arch kernel

Aidenn · 2010-02-09 11:43:07

let's start with the fact, that people are experiencing three different issues here.

first one is the transfer "starts fast, then slows down". it's a normal async behaviour, at first the file is copied to cache, and after the cache fills up, the real transfer speed shows up. if the file is small enough, it may not get past the cache at all, so it appears to be copied instantly. after "finishing" the transfer, the system still has to sync the cache with the stick, that's why you can't unmount the stick. just wait or use -o sync, so you won't see the fake speed at the beginning.

second one is the general slowness. well, ~400kB/s is really bad and looks like a bug, but ~2MB/s is pretty normal, depending on the stick. Standard write speeds for modern sticks are somewhere between 2 and 6 MB/s (even if the manufacturer says, for example, 11MB/s). it gets even worse with big files or a lot of small ones. it's OS-independent, and you can't do anything about it.

third one is slow-only-in-Arch. I can think of two things - first, other OSes have better caching schemes, but it doesn't mean that it's really faster. and you can lose your data if you take the stick out after the transfer seemingly ended, but the cache still hasn't synced. second is that there really is something wrong with the Arch kernel, but everything works fine for me. I'm getting ~4MB/s with my Goodram Twister 4GB, same as in Windows. though that forced OHCI thing described above seems really weird.

Last edited by Aidenn (2010-02-09 11:51:48)

Luis Sousa · 2010-02-09 18:16:01

Aidenn wrote:

third one is slow-only-in-Arch. I can think of two things - first, other OSes have better caching schemes, but it doesn't mean that it's really faster. and you can lose your data if you take the stick out after the transfer seemingly ended, but the cache still hasn't synced. second is that there really is something wrong with the Arch kernel, but everything works fine for me. I'm getting ~4MB/s with my Goodram Twister 4GB, same as in Windows. though that forced OHCI thing described above seems really weird.

I think that it is a kernel bug, because this error is recent to me, 2 months old max... And in ubuntu works fine.

Last edited by Luis Sousa (2010-02-09 18:16:48)

Arch Linux

#51 2009-11-03 18:10:30

Re: USB file copy very slow

#52 2009-11-05 10:12:09

Re: USB file copy very slow

#53 2009-11-05 10:39:28

Re: USB file copy very slow

#54 2009-11-05 10:56:10

Re: USB file copy very slow

#55 2009-11-05 11:25:14

Re: USB file copy very slow

#56 2009-11-05 15:40:08

Re: USB file copy very slow

#57 2009-11-11 20:05:18

Re: USB file copy very slow

#58 2009-11-13 11:59:06

Re: USB file copy very slow

#59 2009-11-14 18:10:28

Re: USB file copy very slow

#60 2009-11-15 03:30:05

Re: USB file copy very slow

#61 2009-11-17 08:05:11

Re: USB file copy very slow

#62 2009-11-17 10:42:42

Re: USB file copy very slow

#63 2009-11-17 11:43:31

Re: USB file copy very slow

#64 2009-12-02 14:51:20

Re: USB file copy very slow

#65 2009-12-13 07:15:11

Re: USB file copy very slow

#66 2009-12-13 21:24:40

Re: USB file copy very slow

#67 2009-12-17 01:51:52

Re: USB file copy very slow

#68 2010-01-04 01:07:44

Re: USB file copy very slow

#69 2010-01-04 01:55:39

Re: USB file copy very slow

#70 2010-01-04 12:41:10

Re: USB file copy very slow

#71 2010-01-14 19:53:52

Re: USB file copy very slow

#72 2010-02-09 07:44:02

Re: USB file copy very slow

#73 2010-02-09 07:47:37

Re: USB file copy very slow

#74 2010-02-09 11:43:07

Re: USB file copy very slow

#75 2010-02-09 18:16:01

Re: USB file copy very slow

Board footer