USB stick - poor transfer speeds

Kardell · 2015-10-10 19:56:58

Hi All,
Do you have any suggestions how to speedup transfer speeds for USB sticks?
I do have a brand new Kingston 64GB with FAT32 (I have to keep this file system for backward compatibility with Windows )
I am copying video 4.5GB and it goes high wire at about 20% and transfer time increases from 1min to 5, but I think it will take 10 to finish this.
I know that SanDisk is selling some crappy sticks, but I have faith in Kingston as my other 8GB stick is performing very well.
Is there anything I can tweak on the system to easy this obstruction?
Thank you

Scimmia · 2015-10-10 20:01:52

If this is a USB 2.0 drive, that seems reasonable.

If you really have to deal with a version of Windows that doesn't support NTFS, I feel really sorry for you.

karol · 2015-10-10 20:03:29

A single 4.5 GB file on a FAT32?

Scimmia · 2015-10-10 20:04:16

Ah, good catch.

Kardell · 2015-10-10 20:06:19

karol you are right FAT32 cannot handle bigger files. Transfer speed for a smaller files is also poor,
I am going to format it now, test it and let you know if any difference.

Why are they selling 64GB sticks with FAT32 on it?

Last edited by Kardell (2015-10-10 20:07:27)

karol · 2015-10-10 20:11:30

Kardell wrote:

Why are they selling 64GB sticks with FAT32 on it?

Why not? It's a common filesystem and it works OK unless you want to store huge files on it.
https://en.wikipedia.org/wiki/File_size

Wikipedia wrote:

the maximal file size on FAT32 is 4 GB−1 B

Kardell · 2015-10-10 20:21:01

I don't mind having FAT32 on 8GB stick, but max 4GB per file limitation on 64GB, sorry.

Same thing is happening on NTFS or it is even worse, for 1.2GB it ran fast up to 55% then is got stuck for like 8min.
For a 4.5GB file it is still estimating the remaining time (2min already)

It is showing now 1h52min.

Kardell · 2015-10-10 20:37:39

This is what dmesg is saying:

[234906.695654] usb 1-2: USB disconnect, device number 19
[234909.784804] usb 1-2: new high-speed USB device number 20 using xhci_hcd
[234909.964317] usb-storage 1-2:1.0: USB Mass Storage device detected
[234909.964609] scsi host10: usb-storage 1-2:1.0
[234910.984180] scsi 10:0:0:0: Direct-Access     Kingston DataTraveler 2.0 PMAP PQ: 0 ANSI: 4
[234912.677632] sd 10:0:0:0: [sde] 122915328 512-byte logical blocks: (62.9 GB/58.6 GiB)
[234912.677757] sd 10:0:0:0: [sde] Write Protect is off
[234912.677759] sd 10:0:0:0: [sde] Mode Sense: 23 00 00 00
[234912.677886] sd 10:0:0:0: [sde] No Caching mode page found
[234912.677888] sd 10:0:0:0: [sde] Assuming drive cache: write through
[234912.698513]  sde: sde1

htop is not showing any unusual high memory or CPU utilization.

Kardell · 2015-10-11 01:23:27

Checked on a different PC with Windows 7. Same thing.
While copying a ISO 7GB in size it boosted to almost 108MB/s and then it slowed down completely at about 25%.
What a piece of junk!

mpan · 2015-10-11 02:23:10

Have you by any chance reformatted/repartitioned the USB stick?

Last edited by mpan (2015-10-11 02:23:21)

Kardell · 2015-10-11 03:17:27

Yes I have formatted it once again in Windows, no difference

R00KIE · 2015-10-11 11:28:28

As the saying goes, you get what you pay for. Many usb 3 flash drives have speeds similar to the slower usb 2 flash drives. In your case the flash drive being usb 3 is just for marketing as usb 2 would be good enough.

The usb 3 flash drives that do have good write speeds are more expensive and you need to do your research well to make sure you are actually buying a fast one.

Kardell · 2015-10-11 12:25:33

You are right, I have researched on that quite some time when I have made a boo-boo and bought crap SanDisk pendrives, but it was at least 2 years ago and I though "ok, eventually this junk will disappear from the market, so I just wait"
Please bear in mind that I am looking for compact storage not just a pen drive to boot something from, in this case 2GB pendrive is big enough.

It looks like these companies keep selling junk for years now and I am not sure why people are buying it as this issue render them useless especially those 16+GB.
I am going to return it asap as it looks like USB hard drive is a better option in this price range.

Please note that this is USB2.0, but the problem is not with the speed, but with some buffer filling up completely. In most of the cases I will not manage to finish transferring as it is slow like a dial-up.

R00KIE · 2015-10-11 14:10:56

I though that it was a usb 3 drive because of this: "new high-speed USB device number 20 using xhci_hcd", but I guess you just plugged it into a usb 3 port.

The buffer issue is normal I'd say, linux buffers writes to the filesystem, up to a point, once the buffer is full it slows down to the speed of the flash drive, and then after the file seems to have transferred you still have to wait until the buffer is emptied to the flash drive. I would suggest you try another filesystem, as it might make some difference, but given that you want compatibility with windows your choices are limited to FAT, exFAT and NTFS.

That said, some brands are worse than others, I do try to avoid flash drives and sd cards from kingston as I've had mostly bad experiences with them, they are cheap but you get what you pay for. As for other brands, better research well before buying otherwise the risk of bad surprises is high.

mpan · 2015-10-11 18:03:33

Kardell wrote:

Yes I have formatted it once again in Windows, no difference

I mean: did you formatted it before?

Flash memory uses blocks for read/write operations, on hardware level. On read a whole block has to be opened, even if a single byte is to be read. On write not only a whole block has to be opened, but actually it has to be erased as a whole and rewritten — again, even if even a single byte is to be written. To obtain any decent performance, partitions and filesystems has to be tuned to match block size of a given device. If they're not, device will never work well; this is especially true for cheaper specimens, which have firmware capable of opening only one block at a time. This is why there is a rule of thumb to never format flash memory, until you exactly know what you're doing. Otherwise you'll degrade transfer speeds, and also greatly decrease device's lifetime.

Kardell · 2015-10-11 18:08:41

@mpan
Originally there was FAT32 on it and the performance was bad from the beginning.
I understand your point.
Is there any way to establish what is the spec and tune it accordingly?
Thanks

mpan · 2015-10-12 13:38:46

If the performance was bad from the begining, then it's not the problem with blocks: flash memories come out of the factory with propery alignment already.

However, if you would ever need to repartition a flash memory, make partitions aligned to the same block size as they were originally. This is the easiest way to avoid problems. Usually you'll have something like this:

[MBR …and some unused space] [partiton] [a bit of unused space]

The size of the first part is typically the size of a block (or a multiple of it, but that's rare) and all your partitions should be aligned to that size.
When setting up a a filesystem and there is a corresponding option available, you should also set it properly (a description for ext4).

If you don't know the original block size, you can use flashbench. Note however, that it requires some interpretation by the user - it's not a magic tool that will simply print out a single answer.

Arch Linux

#1 2015-10-10 19:56:58

USB stick - poor transfer speeds

#2 2015-10-10 20:01:52

Re: USB stick - poor transfer speeds

#3 2015-10-10 20:03:29

Re: USB stick - poor transfer speeds

#4 2015-10-10 20:04:16

Re: USB stick - poor transfer speeds

#5 2015-10-10 20:06:19

Re: USB stick - poor transfer speeds

#6 2015-10-10 20:11:30

Re: USB stick - poor transfer speeds

#7 2015-10-10 20:21:01

Re: USB stick - poor transfer speeds

#8 2015-10-10 20:37:39

Re: USB stick - poor transfer speeds

#9 2015-10-11 01:23:27

Re: USB stick - poor transfer speeds

#10 2015-10-11 02:23:10

Re: USB stick - poor transfer speeds

#11 2015-10-11 03:17:27

Re: USB stick - poor transfer speeds

#12 2015-10-11 11:28:28

Re: USB stick - poor transfer speeds

#13 2015-10-11 12:25:33

Re: USB stick - poor transfer speeds

#14 2015-10-11 14:10:56

Re: USB stick - poor transfer speeds

#15 2015-10-11 18:03:33

Re: USB stick - poor transfer speeds

#16 2015-10-11 18:08:41

Re: USB stick - poor transfer speeds

#17 2015-10-12 13:38:46

Re: USB stick - poor transfer speeds

Board footer