the 100th time in forums - usb storage device is slow [NOT solved]

drakosha · 2007-03-09 07:51:24

My usb flash is mounted by hal (i believe) - i use xfce. And it's mounted without 'async' i believe which causes slow speed. I did not find the solution on the forums.
I added /etc/hal/fdi/policy/90-user-methods.fdi (in order to add 'async' and 'noatime'):

<device> 
  <match key="block.is_volume" bool="true">
      <match key="@block.storage_device:storage.hotpluggable" bool="true">
            <merge key="volume.policy.mount_option.sync" type="bool">false</merge>
            <merge key="volume.policy.mount_option.noatime" type="bool">true</merge>
      </match>
      <match key="@block.storage_device:storage.removable" bool="true">
            <merge key="volume.policy.mount_option.sync" type="bool">false</merge>
            <merge key="volume.policy.mount_option.noatime" type="bool">true</merge>
      </match>
  </match>
</device>

but it did not help. Here's mount result:

>mount
/dev/hda4 on / type reiserfs (rw)
...
/dev/sda on /media/disk type vfat (rw,noexec,nosuid,nodev,shortname=winnt,uid=1000)

What do i do?T

Last edited by drakosha (2007-03-11 18:36:04)

lanrat · 2007-03-09 13:09:45

"sync" is the slowest when you write to the usb drive. "async" should be the fastest and set by default. But there is a new option for both security and speed. Try "flush" mode (for vfat only). Unfortunatelly I don't use hal for automounting just pure udev so I can't help with your setup. This solution works for me quite well for a long time.

drakosha · 2007-03-10 19:17:36

Thanks, but i'd like to know the hal/dbus solution. I'll try udev if there's no other options

drakosha · 2007-03-10 21:27:30

First finding: 'async' is not a valid mount flag at all in hal: found out here /usr/share/hal/fdi/policy/10osvendor/20-storage-methods.fdi
Adding 'async' there and restarting hal did not help.

drakosha · 2007-03-14 14:00:29

I looked a little at thunar code and it looks it calls 'exo-mount' with relevant HAL device id. I got lost in 'exo-mount', but i know nothing about hal, glib, etc. Any XFCE hackers here?

Update: it looks like there's a solution here, i'll check it later:
https://bugzilla.novell.com/show_bug.cgi?id=105871

Last edited by drakosha (2007-03-15 11:01:40)

drakosha · 2007-12-14 14:34:50

i was wrong, async is not the issue - any suggestions what might be the problem?

hal mount:

cat /proc/mounts | grep vfat
/dev/sda1 /media/disk vfat rw,nosuid,nodev,uid=1000,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1,shortname=winnt 0 0
/tmp>time dd if=/dev/zero bs=32k count=5120 of=/media/disk/tst
5120+0 records in
5120+0 records out
167772160 bytes (168 MB) copied, 103.187 s, 1.6 MB/s

real    1m43.210s
user    0m0.017s
sys    0m0.910s

manual mount:

cat /proc/mounts | grep vfat
/dev/sda1 /tmp/xxx vfat rw,noatime,nodiratime,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1 0 0

/tmp>time sudo dd if=/dev/zero bs=32k count=5120 of=/tmp/xxx/tst1
5120+0 records in
5120+0 records out
167772160 bytes (168 MB) copied, 102.382 s, 1.6 MB/s

real    1m42.390s
user    0m0.020s
sys    0m0.787s

drakosha · 2007-12-14 14:42:27

One more input, dmesg shows:

sd 5:0:0:0: [sda] 16777216 512-byte hardware sectors (8590 MB)
sd 5:0:0:0: [sda] Write Protect is off
sd 5:0:0:0: [sda] Mode Sense: 0b 00 00 08
sd 5:0:0:0: [sda] Assuming drive cache: write through
sd 5:0:0:0: [sda] 16777216 512-byte hardware sectors (8590 MB)
sd 5:0:0:0: [sda] Write Protect is off
sd 5:0:0:0: [sda] Mode Sense: 0b 00 00 08
sd 5:0:0:0: [sda] Assuming drive cache: write through

maybe that's the problem?

byte · 2007-12-14 17:32:48

I've never seen anything else than "write through". I have no problems getting 10-20 MB/s with hdparm with my USB stick and external hard disk and didn't configure anything.
Have you verified it's an Arch-only problem? Either with Windows, some LiveCD or on another computer?

drakosha · 2007-12-14 17:37:03

What flags exactly are you using with hdparm for testing - can you please write the exact command? Great idea, i'll try with another computer when i'll be at work.
It's another usb diskonkey though.

Last edited by drakosha (2007-12-14 17:40:32)

byte · 2007-12-14 17:49:10

hdparm --direct -t /dev/sd...
hdparm -t /dev/sd...

There shouldn't be a major difference.

lilsirecho · 2007-12-14 17:52:44

Perhaps it is running at USB 1.1 speed due to being connected to a 1.1 port or not being recognized as 2.0 device.

If any hubs in the system, try without same. Dmesg should ID the USB and usbview give info as well as lsusb. That would include ID for the hub and its usb mode.

I am probably going through things you already have looked into.

Using other ports? Using other devices?

drakosha · 2007-12-14 18:03:04

i'm open to all suggestions, i do not know how to debug this issue..

dmesg:

usb 3-6: new high speed USB device using ehci_hcd and address 16
usb 3-6: configuration #1 chosen from 1 choice
scsi13 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 16
usb-storage: waiting for device to settle before scanning
scsi 13:0:0:0: Direct-Access USB 2.0 Flash Disk 2.00 PQ: 0 ANSI: 2
sd 13:0:0:0: [sda] 16777216 512-byte hardware sectors (8590 MB)
sd 13:0:0:0: [sda] Write Protect is off
sd 13:0:0:0: [sda] Mode Sense: 0b 00 00 08
sd 13:0:0:0: [sda] Assuming drive cache: write through
sd 13:0:0:0: [sda] 16777216 512-byte hardware sectors (8590 MB)
sd 13:0:0:0: [sda] Write Protect is off
sd 13:0:0:0: [sda] Mode Sense: 0b 00 00 08
sd 13:0:0:0: [sda] Assuming drive cache: write through
sda: sda1
sd 13:0:0:0: [sda] Attached SCSI removable disk
usb-storage: device scan complete

usbview:
EHCI Host Controller is USB 2 480Mb/s (usb disk on key is under it)
OHCI Host Controller is USB Version: 1.10 Speed: 12Mb/s (full) (and hci_usb is under it)

Currently i use usb mouse and usb bluetooth dongle... What should i look into?
What else should i look at?

lilsirecho · 2007-12-14 18:09:48

Byte has provided a test that will help in a previous post.

drakosha · 2007-12-14 18:14:59

Is this speed ok?

>sudo hdparm --direct -t /dev/sda1
/dev/sda1:
Timing O_DIRECT disk reads: 50 MB in 3.05 seconds = 16.38 MB/sec
>sudo hdparm -t /dev/sda1
/dev/sda1:
Timing buffered disk reads: 46 MB in 3.01 seconds = 15.26 MB/sec

2 questions:
1. What is the expected speed?
2. why 'async' and 'noatime' does not affect performance?

Last edited by drakosha (2007-12-14 18:19:11)

lilsirecho · 2007-12-14 18:26:44

The expected speed is between 10 and 20MB/S so yours are correct.

The problem lies elsewhere.

drakosha · 2007-12-14 23:12:22

I just visited a friend with windows. It looks like my flash is much slower than his. Maybe it's in fact usb 1.1?

Furthermore, the problem is in *write*, not read. Can anyone please do timing similarly to one i did wtu dd above? Note: it'll create a new file!

Last edited by drakosha (2007-12-14 23:40:16)

broch · 2007-12-15 16:16:53

sudo hdparm -t /dev/sdb1
/dev/sdb1:
Timing buffered disk reads: 62 MB in 3.08 seconds = 20.12 MB/sec
michal[~]$ sudo hdparm --direct -t /dev/sdb1
/dev/sdb1:
Timing O_DIRECT disk reads: 70 MB in 3.05 seconds = 22.97 MB/sec
GEIL 1GB USB drive/hal (no fstab entry, no tweaking with config )
time dd if=/dev/zero bs=32k count=5120 of=deleteme.now
5120+0 records in
5120+0 records out
167772160 bytes (168 MB) copied, 14.0872 s, 11.9 MB/s
real 0m14.090s
user 0m0.005s
sys 0m0.563s

I think that this is waste of time, different (USB) drives will have different read/write speed. Maybe first check with your manufacturer instead of guessing what "it should be"?

hdparm is not really a benchmark and results will vary when repeated several times.

Last edited by broch (2007-12-15 16:18:59)

drakosha · 2007-12-15 16:33:52

Is there a possibility to check the flash's controller - which usb version does it support?

lilsirecho · 2007-12-15 17:37:10

Usb view should give the mode provided by the controller for each drive.

Be most aware that wriye speeds are notoriously slow with some flash drives. This is because the chips are slower in some drives and the fact that erase has to occur before write.

The application best utilizing flash drives is Live OS systems using squashfs or cramfs which utilize reads from the flash drive.

Some insight into the performance of your drive is to run different file sizes of data through the flash drive. Large files move faster, many short files in a large format will move very slowly. Flash drives differ in performance by large margins.

Controllers are also involved. Some are great performers but others are slow.

Perhaps you should try a different flash unit. Compact Flash is 1.1 speed by the way.

It may help to format the drive before running the files you have poor performance with.

Basically, write is usually 40% to 50% less speedy than write and the content of the write filescan reduce the write speed dramatically.

Lots of data in googling on the subject of write speeds.

santodelaespada · 2009-04-15 05:50:39

Hi, I had the same problem last year.
A brand new cheap pendrive, before I crossed over to linux, worked fine under windows XP.
One fine day the writing speed droped down to near zero.
googling I learned that pendrives SHALL NOT BE FORMATED (neither any other flash media), and my usual "full erase" tecnic under windows was to "quick format" it. The result is that somehow it creates a partition structure that (i belive) nests everytime you format the drive this way. Any way the result is an exponentialy decreassed writing speed!!!

solution came along with linux and ignorance, I searched the web on "how to format usb flash drives" and learned about mkdosfs. Magic solution is...... :

unmount the volume (if mounted)
make a full drive filesystem with mkdosfs

In my 2GiB pendrive I use:

mkdosfs -f 1 -F 16 -n Pendrive -r 512 -v -I /dev/sdX

where "-f 1" makes only one fat; "-F 16" makes FAT16 format; "-n Pendrive" labels it Pendrive; "-r 512" makes the maximun root entries equal to 512; "-v" for verbose output; "-I" to make full drive filesystem (no partitions); and /dev/sdX is the device you'd like to erase

--- REMEMBER ---
is "/dev/sdb" NOT "/dev/sdb1" (with out the numbers)

after that speed is restored.
Hope is usefull

in spanish: http://taringa.net/posts/linux/1874931/ … Linux.html

apaige · 2009-04-15 09:46:59

USB 1.1 is limited to 12 megabits per second, or 1.5 megabyte per second (in practice, more like 1 megabyte per second). You're getting over 15 MB/s with hdparm, so that's not your problem; your USB flash drive is definitely connected as a USB 2 device.

Your device is probably slow at writing to begin with, but there's an additional factor to take into account: the erase block size. Flash devices have to erase blocks of data before they can write over them; the erase block size varies between devices, it's usually 128KiB or 512KiB, but it can be even higher than that.

So, if your dd test uses a block size (32KiB in your case) that is smaller than your device's erase block size, you're already getting a performance penalty. Better try a large block size (like 1MiB) to make sure you're not crippling your device.

Also, you have to make sure your partition starts at an erase block boundary. I don't feel like going into the details right now, but I suggest you back up your data and run dd on the device itself instead of the mounted filesystem.

dd if=/dev/zero of=/dev/sda bs=1M count=128

Arch Linux

#1 2007-03-09 07:51:24

the 100th time in forums - usb storage device is slow [NOT solved]

#2 2007-03-09 13:09:45

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#3 2007-03-10 19:17:36

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#4 2007-03-10 21:27:30

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#5 2007-03-14 14:00:29

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#6 2007-12-14 14:34:50

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#7 2007-12-14 14:42:27

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#8 2007-12-14 17:32:48

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#9 2007-12-14 17:37:03

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#10 2007-12-14 17:49:10

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#11 2007-12-14 17:52:44

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#12 2007-12-14 18:03:04

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#13 2007-12-14 18:09:48

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#14 2007-12-14 18:14:59

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#15 2007-12-14 18:26:44

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#16 2007-12-14 23:12:22

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#17 2007-12-15 16:16:53

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#18 2007-12-15 16:33:52

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#19 2007-12-15 17:37:10

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#20 2009-04-15 05:50:39

Re: the 100th time in forums - usb storage device is slow [NOT solved]

#21 2009-04-15 09:46:59

Re: the 100th time in forums - usb storage device is slow [NOT solved]

Board footer