Am I being "lied to" about file copies to USB?

earthpig · 2010-04-29 23:38:58

I know this is going to sound bizarre, but I think both pcmanfm and nautilus are misleading me.

Replicate:
1. copy/paste a large file (~700mb) from hard drive to thumb drive. watch the dialog box zip to 100% and close. play movie. it plays fine.

unmount thumb drive, remove, put back in. error messages when i try to play the file -- and sometimes it'll say the file is suddenly ~600 or ~650mb.

2. start over. copy/paste a large file (~700mb) from hard drive to thumb drive. watch the dialog box zip to 100% and close. play movie. it plays fine.

walk away, leaving thumb drive mounted and inserted for 20 minutes.

unmount and remove thumb drive. reinsert. play movie. it plays fine, no errors at all.

tried both 1 and 2 with two different usb ports, two different thumb drives, and a multitude of movies.

why does walking away for 20 minutes after it is "done" copying make a difference? any way i can get pcmanfm and/or nautilus to tell the truth about the progress?

Last edited by earthpig (2010-04-29 23:54:03)

Allan · 2010-04-29 23:52:28

How are you unmounting the thumb drive? From what I understand, some of the transfer is stored in a cache to speed up the process from a user perspective. But unmounting the drive should force all that to be written.

earthpig · 2010-04-29 23:56:13

thanks for the response.

in both nautilus and pcmanfm (whichever i use to copy): right click on the drive on the left -> unmount -> wait till its done unmounting (both the indicator light on the thumb drive and the graphical indication in nautilus or pcmanfm) -> remove

some of the transfer is stored in a cache to speed up the process from a user perspective.

any way i can disable that? is that on arch's end, the hardware's end, or the thumb drive's end?

Last edited by earthpig (2010-04-29 23:57:24)

karol · 2010-04-30 00:00:40

> any way i can disable that?
http://bbs.archlinux.org/viewtopic.php?id=96060

falconindy · 2010-04-30 00:00:40

You can mount the drive with the 'sync' option. This will force all write operations to be done synchronously.

karol · 2010-04-30 00:02:38

[OT]
Ummm, it's the first time I've seen a 'tie' (check the post time: mine and falconindy's) on the forums. Does it mean I'll die tomorrow or sth?
[/OT]

earthpig · 2010-04-30 00:04:20

sweet, thanks guys!

just to make sure i understand correctly:

1. i should start the copy process

2. wait till it claims to be done

3. run the command 'sync'

4. now unmount and remove

is that correct?

flamelab · 2010-04-30 00:06:41

Hmm, is there any way we can edit Udev (?) to use "sync(hronous)" mount for external hdd/usb?

I've experienced the exact same thing, and it's highly frustrating.

karol · 2010-04-30 00:11:54

I don't use automounting and I have no file manager.

#!/bin/bash

DRIVE="/dev/sdb1"

if mount | grep -q $DRIVE; then
  umount $DRIVE
else
  mount $DRIVE /mnt/usb
fi

(I stole this piece of code, it's not mine)

I stick the stick in, run the script, copy / remove files and run it again - I physically remove the USB stick only after I see the prompt (and this can often take a while) ready after it finished unmounting - which means everything is synced.

eirik · 2010-04-30 01:32:53

earthpig wrote:

sweet, thanks guys!
just to make sure i understand correctly:
1. i should start the copy process
2. wait till it claims to be done
3. run the command 'sync'
4. now unmount and remove
is that correct?

No, you're supposed to use the sync option at mount time. Type "man mount" in a terminal.

perbh · 2010-04-30 01:37:46

eirik wrote:

earthpig wrote:
sweet, thanks guys!
just to make sure i understand correctly:
1. i should start the copy process
2. wait till it claims to be done
3. run the command 'sync'
4. now unmount and remove
is that correct?
No, you're supposed to use the sync option at mount time. Type "man mount" in a terminal.

Yes, but - the 'sync' command will also flush caches to disk ...
In the 'old days' we always had to do "sync; sync; umount"

eirik · 2010-04-30 01:47:42

perbh wrote:

eirik wrote:
earthpig wrote:
sweet, thanks guys!
just to make sure i understand correctly:
1. i should start the copy process
2. wait till it claims to be done
3. run the command 'sync'
4. now unmount and remove
is that correct?
No, you're supposed to use the sync option at mount time. Type "man mount" in a terminal.
Yes, but - the 'sync' command will also flush caches to disk ...
In the 'old days' we always had to do "sync; sync; umount"

I stand corrected.

When copying files to my memory stick, the copying progress indicator goes to 100%, but keeps waiting there until the cache has been written completely. It's weird that the OP's system doesn't handle this properly, but I guess hardware differences are to blame.

ngoonee · 2010-04-30 02:08:21

When your DE does an unmount, it takes quite a bit of time for syncing to properly finish if you've copied a large file. Using Gnome here, there's a big 'finishing copying files, do not remove yet' warning. If I wait for that to go away (and the activity light to shut off) then things are fine.

Its not really 'lying', caching of file writes is a Good Thing for preserving HD/thumbdrive lifespans and reducing power consumption.

falconindy · 2010-04-30 02:13:42

flamelab wrote:

Hmm, is there any way we can edit Udev (?) to use "sync(hronous)" mount for external hdd/usb?
I've experienced the exact same thing, and it's highly frustrating.

Sure. There's a few udev different flavors of udev rules in the wiki. I've modified one slightly, which looks like this...

KERNEL!="sd[e-z][0-9]", GOTO="media_by_label_only_auto_mount_end"

ACTION=="add", IMPORT{program}="/sbin/blkid -o udev -p %N"
ENV{ID_FS_LABEL}=="", GOTO="media_by_label_only_auto_mount_end"

ACTION=="add", RUN+="/bin/mkdir -p /media/$env{ID_FS_LABEL}"

# Global mount options
ACTION=="add", ENV{mount_options}="noatime,users"
# Filesystem specific options
ACTION=="add", ENV{ID_FS_TYPE}=="vfat|ntfs", ENV{mount_options}="$env{mount_options},sync,utf8,uid=1000,gid=100,fmask=133,dmask=022"

ACTION=="add", RUN+="/bin/mount -o $env{mount_options} /dev/%k /media/$env{ID_FS_LABEL}"
ACTION=="remove", ENV{ID_FS_LABEL}=="?*", RUN+="/bin/umount -l /media/$env{ID_FS_LABEL}", RUN+="/bin/rmdir /media/$env{ID_FS_LABEL}"
LABEL="media_by_label_only_auto_mount_end"

Note that the first line probably needs to be modified -- I've been fighting something (dunno if its a bug) that makes this rule mount my internal drives at bootup -- fstab gets parsed, crying ensues because everything's already mounted, and I get dumped to a maintenance shell.

grey · 2010-04-30 08:46:15

OP: can you reproduce this from the command line? I.e. do "mount; copy; unmount; mount;" and then check the big file.

cesura · 2010-04-30 20:12:35

karol wrote:

I don't use automounting and I have no file manager.
#!/bin/bash

DRIVE="/dev/sdb1"

if mount | grep -q $DRIVE; then
  umount $DRIVE
else
  mount $DRIVE /mnt/usb
fi
(I stole this piece of code, it's not mine)
I stick the stick in, run the script, copy / remove files and run it again - I physically remove the USB stick only after I see the prompt (and this can often take a while) ready after it finished unmounting - which means everything is synced.

You could always put that in your shell config file. That way, when you start a terminal, it will be mounted

earthpig · 2010-04-30 22:47:10

eirik wrote:

When copying files to my memory stick, the copying progress indicator goes to 100%, but keeps waiting there until the cache has been written completely. It's weird that the OP's system doesn't handle this properly, but I guess hardware differences are to blame.

the most significantly unusual hardware i have is that i am typically copying things from an SSD to a thumb drive -- likely one of the more drastic speed differences.

OP: can you reproduce this from the command line? I.e. do "mount; copy; unmount; mount;" and then check the big file.

sure, i will later.

Arch Linux

#1 2010-04-29 23:38:58

Am I being "lied to" about file copies to USB?

#2 2010-04-29 23:52:28

Re: Am I being "lied to" about file copies to USB?

#3 2010-04-29 23:56:13

Re: Am I being "lied to" about file copies to USB?

#4 2010-04-30 00:00:40

Re: Am I being "lied to" about file copies to USB?

#5 2010-04-30 00:00:40

Re: Am I being "lied to" about file copies to USB?

#6 2010-04-30 00:02:38

Re: Am I being "lied to" about file copies to USB?

#7 2010-04-30 00:04:20

Re: Am I being "lied to" about file copies to USB?

#8 2010-04-30 00:06:41

Re: Am I being "lied to" about file copies to USB?

#9 2010-04-30 00:11:54

Re: Am I being "lied to" about file copies to USB?

#10 2010-04-30 01:32:53

Re: Am I being "lied to" about file copies to USB?

#11 2010-04-30 01:37:46

Re: Am I being "lied to" about file copies to USB?

#12 2010-04-30 01:47:42

Re: Am I being "lied to" about file copies to USB?

#13 2010-04-30 02:08:21

Re: Am I being "lied to" about file copies to USB?

#14 2010-04-30 02:13:42

Re: Am I being "lied to" about file copies to USB?

#15 2010-04-30 08:46:15

Re: Am I being "lied to" about file copies to USB?

#16 2010-04-30 20:12:35

Re: Am I being "lied to" about file copies to USB?

#17 2010-04-30 22:47:10

Re: Am I being "lied to" about file copies to USB?

Board footer