[solved] Large file transfers are corrupted

AbuG · 2018-01-27 01:28:09

Hey guys, am new to the community.

Fairly new to Arch (2 months) been using debian based systems for many years.

I been having a problem with file transfers of large media files they are getting corrupt during transfer .

the file is 11.5GB .mkv file on an NTFS system on an internal hdd and am copying to an external USB hdd on exFat filesystem.

I have tried with dolphin and rsync (using gRsync) and the target file is 0.3 GB smaller.
checksum with SHA256 are different to each other and likewise different from the source file.

I ran an exfatfsck on the target drive which is ok. also checked in windows10 .

the source partition is mounted in fstab as follows
UUID=060********DF /media/BigV ntfs-3g defaults,locale=en_GB.UTF-8 0 0

initially when i installed Arch 2months back this drive was read only and eventually i found some errors on it and fixed them and it mounts properly with write functionality
I have checked in windows10 again and there are no errors. Likewise I tested transfer of the file in windows10 and the checksums match.

The only other thing to note is that the transfer speed on arch is 70Mb/s for the first 4 GB and then 30Mb/s thereafter... while in windows10 it is about 30Mb/s throughout ..

I have been getting corruption on smaller files eg. 1.5 GB. also from different source partitions but also NTFS filesystem.

In debian i didnt use ntfs-3g in fstab.... was just ntfs with no additional packages.

Any assistance on how to troubleshoot would be greatly appreciated

Last edited by AbuG (2018-01-27 20:13:57)

i_love_r34 · 2018-01-27 03:35:39

What is the output of dmesg after the transfer?

justasug · 2018-01-27 09:40:47

Even though you said that there is no corruption when you transfer files on Windows, I would still check the hardware to be sure. Especially the RAM. Download memtest and run it to see if it's faulty.
I had the same problem of files getting corrupted (hash mismatches) upon copying. It turned out it was slightly faulty RAM.

seth · 2018-01-27 09:50:43

NTFS system on an internal hdd and am copying to an external USB hdd on exFat filesystem.
...
I have checked in windows10 again and there are no errors.

Disable windows fastboot. https://www.tenforums.com/tutorials/418 … -10-a.html

AbuG · 2018-01-27 20:13:22

seth wrote:

Disable windows fastboot. https://www.tenforums.com/tutorials/418 … -10-a.html

This appears to have done the trick. Many thanks .

windows10 is on a completely different drive to the source file so wonder why it causes a problem

i done memtest it was ok .

seth · 2018-01-27 20:19:55

Windows only hibernates and leaves *every* filesystem it touched in an inconsistent state. This basically breaks using two OS in parallel (incl. 2 different windows installations) for good.
I'm not at hand sure whether it's at least smart enough to sync and detach removable disks ...

Long story short: "booting windows takes ages and is unfortunately mandatory whenever you plug in a new hardware and auto-install a driver, so we came up with this fragile crap" m(

Arch Linux

#1 2018-01-27 01:28:09

[solved] Large file transfers are corrupted

#2 2018-01-27 03:35:39

Re: [solved] Large file transfers are corrupted

#3 2018-01-27 09:40:47

Re: [solved] Large file transfers are corrupted

#4 2018-01-27 09:50:43

Re: [solved] Large file transfers are corrupted

#5 2018-01-27 20:13:22

Re: [solved] Large file transfers are corrupted

#6 2018-01-27 20:19:55

Re: [solved] Large file transfers are corrupted

Board footer