USB transfer slows down

noc3 · 2022-05-31 07:52:24

Hi, it's my first post. (I hope i'm doing well)

My problem is with usb transfer, it has a strange behaviour: when i move large files in a usb the transfer starts without problems, reach a byte limit (like a treshold) then the speed slows down costantly from ~200MB/s to ~10MB/s until the transfer stops. With smaller files (probably smaller than this "treshold") the tranfer speed is high but it "freeze" when it's complete, so the operation never end.

I find this problem tranfering file using nautilus, i use GNOME as DE.
I have 2 USB 3.1 and one 2.0, and seems the drivers and the firmware are correct (xhci-hcd?).
My kernel is Linux 5.18.0-arch1-1 #1 SMP PREEMPT_DYNAMIC Tue, 24 May 2022 22:00:36 +0000.
The only thing it seems produce some effect is set an high value of vm.dirty_bytes, but it seems that only set an higher treshold.

I really don't know how to solve the problem, on internet i can't find anything. Sorry if the information are not so clear but i only see this behaviour without knowing what's produce it.

JoeyCorleone · 2022-05-31 08:05:32

This could be due to hardware limitation or degeneration.

Could you try to reproduce this on another PC? I'm assuming we are talking about portable USB flash drives here.

noc3 · 2022-05-31 08:23:34

Yes is portable USB flash drives. I don't think is about hardware because on this laptop i have previously installed ubuntu and windows and there was no problem.
Unfortunatly i didn't have another pc for reproducing the error.

I didn't konw if this information could help but i cant to eject usb correclty (after transfering something) and if i reboot, the pc won't reboot until i remove phisically the usb pen, but if i remove the pen the gurb crashes.

I also find nothing in journalctl

dimich · 2022-05-31 12:55:37

noc3 wrote:

With smaller files (probably smaller than this "treshold") the tranfer speed is high but it "freeze" when it's complete, so the operation never end.

Have you tried different USB flash drives? Maybe this particular drive is degrading and FTL hides details from host.

noc3 wrote:

I find this problem tranfering file using nautilus, i use GNOME as DE.

What if write directly to the drive by dd'ing to /dev/sdX, where sdX is you flash storage device? This will erase partition table and filesystem, but you can re-create them after test.

noc3 wrote:

The only thing it seems produce some effect is set an high value of vm.dirty_bytes, but it seems that only set an higher treshold.

Generally it looks like a write cache is filled quickly first, then actual write to device is slow. It would be expected behaviour except for "the operation never end".

noc3 · 2022-05-31 13:09:18

yes i've tried with multiple USB flash drives on different port.
i also tried to to change file system (ntfs or fat) and also remove and reinstall ntfs-3g.
I'm sorry but i don't undestand what "dd'ing" means.
Yes probably is something like the file is written in the cache but the cache never write on the actual USB flash drive

dimich · 2022-05-31 13:22:11

noc3 wrote:

I'm sorry but i don't undestand what "dd'ing" means.

I mean unmount your flash drive and run something like

# dd if=/dev/urandom of=/dev/sdX status=progress

where sdX is your USB storage device name. You can see it by `lsblk` command or in `dmesg` output after plugging in. WARNING! Be careful with this name, don't confuse it with another drives because raw write will destroy all data on the drive.

dimich · 2022-05-31 13:23:58

But if the issue is observed on different flash drives, it makes no sense to raw write on particular drive. Nevermind.

seth · 2022-05-31 13:27:19

but i cant eject usb correclty (after transfering something) and if i reboot, the pc won't reboot until i remove phisically the usb pen, but if i remove the pen the grub crashes.

Can you please elaborate on this?
1. system has a 90s timeout before it forcefully termiates processes - did you wait this long for a reboot (or maybe 5 minutes, get a cup of coffee)
2. How does "the grub crash"?? (Assuming that's what "gurb" was supposed to be)

In general, check dmesg or the system journal for IO errors after failing to write to the USB key.

noc3 · 2022-05-31 14:27:09

Later i try and i'll post the output of dmesg.

Sorry, "the grub crashes" means that after shutting down the screen goes black whit a broadcast message (something like "the system is shutting down NOW") and remains like this until i unplug (phisically) the USB key, after shuts down.
If i do this operations, when i restart the laptop seems like it tries to recover itself: the bios parameters return to the default values and the screen shows "no bootable device" message. I fix this changing the parameters in bios, restarting, booting in arch installation USB key and manually reinstall the grub bootloader.

Sorry for the ignorance: what commands (apart from dmesg) shows me useful informations? (information that eventually i'll post to fix this)useful

ps: i say "bios" but i have UEFI

Last edited by noc3 (2022-05-31 14:33:37)

mpan · 2022-05-31 16:04:49

Seems to me like an expected behavior for transferring files to a medium that can handle only 10MB/s write speeds:

Initially data is piled up in file cache, where the limiting speed is source medium read bandwidth. 200MB/s seems reasonable read speed for a HDD.
File cache gets filled up and transfer is limited to the actual write speed of the destination medium. 10MB/s is quite common for USB sticks.
No more data is there to be read from source medium, but flushing from file cache still continues. Seems as if nothing is happening, as from the perspective of the copying application all data has been transferred.

You can’t unmount the filesystem to which data is still being written, much less eject the medium. And waiting for that to end may be a blocking I/O operation that can’t be interrupted, consistent with what you describe.

noc3 · 2022-05-31 16:31:48

There is a way to check the write speed of the usb?
If is this the problem, there is a way to limit the write rate in cache (or something else)? I need to see the correct "operation time" in nautilus (graphical file manager)

mpan · 2022-05-31 17:09:38

noc3 wrote:

There is a way to check the write speed of the usb?

head -c 1GB /dev/urandom >/tmp/testfile
time bash -c 'cd /path/to/mounted/usb && mv /tmp/testfile . && sync -f testfile'

Divide 1GB by time reported as “real” to get estimation of the average write speed to the mounted filesystem on the USB stick. For 10MB/s one that will be in the range of 100 seconds.

noc3 wrote:

If is this the problem, there is a way to limit the write rate in cache (or something else)?

Literally? No, because the question makes no sense: there is no particular “write to cache” operation. It’s a “read from source medium” operation and that is determined by how fast it can deliver data. And even if you could slow down your source medium, to do that you must know the destination transfer rate… so you achieve nothing.

noc3 wrote:

I need to see the correct "operation time" in nautilus (graphical file manager)

Divide size of the data by your USB stick’s write speed. Unless you specifically bough a high-speed one, you may safely assume it’s 5 minutes per 3GB.

There is no sane⁽¹⁾ way to do that automatically. Not in Nautilus, not in any other file manager, but in general by the very nature of the problem. An exact estimation requires frequently flushing data to the destination: not only it slows things down and prevents efficient use of extents, but also increases write amplification.
____
⁽¹⁾ Of course “sane” is subjective. But I do not find such costs reasonable.

noc3 · 2022-05-31 17:50:10

I give that commands and this are the results:

real	3m10,905s
user	0m0,006s
sys	0m1,709s

And this is dmesg

 
[  140.690019] usb 1-1: new high-speed USB device number 5 using xhci_hcd
[  140.830837] usb 1-1: New USB device found, idVendor=0781, idProduct=5575, bcdDevice= 1.00
[  140.830851] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  140.830857] usb 1-1: Product: Cruzer Glide
[  140.830862] usb 1-1: Manufacturer: SanDisk
[  140.830865] usb 1-1: SerialNumber: 4C530001241005103242
[  140.857321] usb-storage 1-1:1.0: USB Mass Storage device detected
[  140.857419] scsi host2: usb-storage 1-1:1.0
[  140.857487] usbcore: registered new interface driver usb-storage
[  140.864473] usbcore: registered new interface driver uas
[  141.878017] scsi 2:0:0:0: Direct-Access     SanDisk  Cruzer Glide     1.00 PQ: 0 ANSI: 6
[  141.878596] sd 2:0:0:0: Attached scsi generic sg0 type 0
[  141.879506] sd 2:0:0:0: [sda] 30892032 512-byte logical blocks: (15.8 GB/14.7 GiB)
[  141.880743] sd 2:0:0:0: [sda] Write Protect is off
[  141.880746] sd 2:0:0:0: [sda] Mode Sense: 43 00 00 00
[  141.881064] sd 2:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[  141.888207]  sda: sda1
[  141.889301] sd 2:0:0:0: [sda] Attached SCSI removable disk
[  142.247161] FAT-fs (sda1): Volume was not properly unmounted. Some data may be corrupt. Please run fsck.
[  162.723614] audit: type=1334 audit(1654018171.877:98): prog-id=25 op=LOAD
[  162.723680] audit: type=1334 audit(1654018171.877:99): prog-id=26 op=LOAD
[  162.723701] audit: type=1334 audit(1654018171.877:100): prog-id=27 op=LOAD
[  162.757269] audit: type=1130 audit(1654018171.913:101): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  192.793503] audit: type=1131 audit(1654018201.946:102): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  192.891266] audit: type=1334 audit(1654018202.046:103): prog-id=0 op=UNLOAD
[  192.891270] audit: type=1334 audit(1654018202.046:104): prog-id=0 op=UNLOAD
[  192.891271] audit: type=1334 audit(1654018202.046:105): prog-id=0 op=UNLOAD
[  529.200405] audit: type=1100 audit(1654018538.358:106): pid=3837 uid=1000 auid=1000 ses=4 msg='op=PAM:authentication grantors=pam_faillock,pam_permit,pam_faillock acct="noce" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  529.200554] audit: type=1101 audit(1654018538.358:107): pid=3837 uid=1000 auid=1000 ses=4 msg='op=PAM:accounting grantors=pam_unix,pam_permit,pam_time acct="noce" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  529.201244] audit: type=1110 audit(1654018538.358:108): pid=3837 uid=1000 auid=1000 ses=4 msg='op=PAM:setcred grantors=pam_faillock,pam_permit,pam_faillock acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  529.202226] audit: type=1105 audit(1654018538.358:109): pid=3837 uid=1000 auid=1000 ses=4 msg='op=PAM:session_open grantors=pam_systemd_home,pam_limits,pam_unix,pam_permit acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

If i try (from nautilus) to eject the device it says that is impossible 'cause the the volume is busy, like i'm writing something, but the writing operation ends.
If i umount it says ("noce" is my user, "a" is volume name)

 umount: /run/media/noce/a: target is busy.

If i eject it anyway remains the "eject icon" but if i try to umount from terminal it says "no mount point specified" (so is unmounted?)

After this i shut down and it works without problems

Last edited by noc3 (2022-05-31 17:52:27)

noc3 · 2022-05-31 18:28:03

i tried with another two usb:
one ntfs: it ejected without problems
time:

 
real	4m52,061s
user	0m0,011s
sys	0m2,029s

the second is a usb 3.0 adapter for a 64GB micro sd:
on 3.0 port gives problem in ejecting
on 2.0 port it doesn't give problem in ejecting (but i wait up to 2 min before ejecting)

tried on usb 3.1 port:
  real	0m42,177s
  user	0m0,007s
  sys	0m1,128s

tried on usb 2.0 port: 
  real	0m48,303s
  user	0m0,005s
  sys	0m1,611s

I after tried again on port 3.1 with the "second usb", this time i waited 4 minutes and it ejects without problems

 
real	0m40,614s
user	0m0,004s
sys	0m1,190s

mpan · 2022-05-31 19:13:02

That gives 3–5 MB/s. That is low, but this is also what Cruzer Glide speed is reported to be. The manufacturer doesn’t provide write speed in the specs, which strongly confirms that data.

However, after a call to `sync` is finished, unmounting and ejecting should be instantaneous. Is some application using files on the stick? Don’t you have a terminal open with shell in a directory on the stick? That would prevent unmounting. You may use lsof to find out what is using the thumb drive:

sudo lsof /path/to/mounted/usb/stick

noc3 · 2022-06-07 09:46:35

Hi, i wait some times and try some solutions, unfortunally i can't solve the problem, i tried to change kernel too.
I also read that there are some cache problem with nvidia gpus so maybe is my situation.
I didn't know, i can't figure it out.
Thank you all, i think i leave the situation like this, paying attention on how i unmount the usb stick.

Arch Linux

#1 2022-05-31 07:52:24

USB transfer slows down

#2 2022-05-31 08:05:32

Re: USB transfer slows down

#3 2022-05-31 08:23:34

Re: USB transfer slows down

#4 2022-05-31 12:55:37

Re: USB transfer slows down

#5 2022-05-31 13:09:18

Re: USB transfer slows down

#6 2022-05-31 13:22:11

Re: USB transfer slows down

#7 2022-05-31 13:23:58

Re: USB transfer slows down

#8 2022-05-31 13:27:19

Re: USB transfer slows down

#9 2022-05-31 14:27:09

Re: USB transfer slows down

#10 2022-05-31 16:04:49

Re: USB transfer slows down

#11 2022-05-31 16:31:48

Re: USB transfer slows down

#12 2022-05-31 17:09:38

Re: USB transfer slows down

#13 2022-05-31 17:50:10

Re: USB transfer slows down

#14 2022-05-31 18:28:03

Re: USB transfer slows down

#15 2022-05-31 19:13:02

Re: USB transfer slows down

#16 2022-06-07 09:46:35

Re: USB transfer slows down

Board footer