You are not logged in.
When copying files (especially external USB drives) I'm always issueing a:
sync
because the file transfer pretends to have finished but still didn't write everything to the disk. It happened to me more than once, that files got corrupted on the connected disk because I seemingly forgot to issue a sync command and disconnected the device too early.
It happens both with the cp command as well as with any graphical file manager. They pretend to write at insane speeds, finish very quickly, but only when I issue a sync the drive LED flashes like hell for at least 10 seconds while writing the remaining blocks to disk.
Doing some research, I've found the asynchronous mounting to be the culprit, which seems to be the default setting for mounting external drives. However I have not found any clue on how to change that default setting. It happened to me on all distros so far.
What I need is a robust method for copying files, thus synchronously mounted USB devices. I'm still backing up my data partition with robocopy on Windows (shame on me) because I cannot allow any files getting corrupted.
Arch - makes me feel right at /home
Offline
Use the flush mount option. This is a useful udev rule:
ACTION=="add", ENV{ID_FS_TYPE}=="vfat|ntfs", ENV{mount_options}="$env{mount_options},flush,utf8,gid=100,umask=002"
Offline
Just a point of clarification... It is the kernel doing this, not the copy tools (cp, rsync, dolphin, thunar etc).
And yes, you're right; async mounting (default) is the culprit and will almost certainly cause you corruption issues if the device isn't sync'ed first.
Which brings me to the fact that a sync is done when the device is unmounted; so if you're not unmounting your devices properly, what did you expect to happen? Even Windows does this with the "Safely Remove Device" dialog.
If you're mounting using fstab, you can use the "sync" flag in the option column, otherwise the udev above is a good choice.
But please... A unicorn cries everytime a device isn't cleanly unmounted
Are you familiar with our Forum Rules, and How To Ask Questions The Smart Way?
BlueHackers // fscanary // resticctl
Offline
Thanks both of you for your replies!
Which brings me to the fact that a sync is done when the device is unmounted; so if you're not unmounting your devices properly, what did you expect to happen? Even Windows does this with the "Safely Remove Device" dialog.
The last time I regularly needed to "safely remove" my devices on Windows was a decade ago on Windows 2000.
I do unmount my devices properly on Linux and Mac though, but I tend to forget to at times due to this.
What I want to achieve is that the file copy progress shown corresponds to the actual copy process. I don't care about the need to unmount that much but I do care about the incorrect progress display and the huge time amount the sync operation takes.
Can someone shed some light on the difference of flush and sync? Is their functionality filesystem dependent? (I've got devices with FAT, NTFS and exFAT)
Arch - makes me feel right at /home
Offline
according to man mount, the flush option is only supported on FAT filesystems.
the sync option is more versatile :
The following options apply to any filesystem that is being mounted (but not every filesystem actually honors them - e.g., the sync option today has effect only for ext2, ext3, fat, vfat and ufs):partitions
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
I've never had problems but I come from OS X so I'm paranoid about unmounting as OS X throws out alarming error messages if you forget to do it and so you learn pretty fast. I pretty much always use safely unmount in Windows although I don't remember needing to do this when I used Windows properly but that was Windows 95...
CLI Paste | How To Ask Questions
Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L
Offline