[Solved] Torrent-like file copy

stundrum · 2013-09-19 17:34:28

The thing is i have very unreliable usb flash and it will always copy larger files with error or two (bad chip or overheating idk) but once file is copied it will always read it correctly so it is useful for smaller files...

I am wandering if i could somehow copy file in a manner like torrent client downloads large files, i mean dividing a file in a small chunks and generate checksum of every chunk, then copy chunk by chunk on the flash drive while checking its checksum on destination and when error occurs overwrite that chuck and so on... I know i can simply add it to archive and then split it to smaller pieces but i need whole file on a flash...

Any ideas?

Last edited by stundrum (2013-10-02 22:36:27)

jasonwryan · 2013-09-19 17:57:20

You could use btsync.

Not a GNU/Linux discussion; moving to Apps and DEs...

karol · 2013-09-19 18:00:45

jasonwryan wrote:

You could use btsync.

I think it's not what it's designed for and seems a bit of an overkill.
OP simply wants to copy files to a pendrive.

Last edited by karol (2013-09-19 18:03:09)

stundrum · 2013-09-19 18:11:35

jasonwryan wrote:

Not a GNU/Linux discussion; moving to Apps and DEs...

Oops, didn't saw Apps forum.

karol wrote:

jasonwryan wrote:
You could use btsync.
I think it's not what it's designed for and seems a bit of an overkill.
OP simply wants to copy files to a pendrive.

Exactly.

Last edited by stundrum (2013-09-19 18:21:56)

aesiris · 2013-09-19 19:49:39

Try this script

#!/bin/bash
maxerrors=20

ifile="$1"
ofile="$2"
quant="$3"
[[ $quant ]] && (($quant>1)) || quant=4194304

size=$(stat -c %s "$ifile")
i=0
j=0
while ((i*quant < size)) && ((j<maxerrors)); do
        dd if="$ifile" of="$ofile" bs=$quant count=1 seek=$i skip=$i
        if [[ "$(dd if="$ofile" bs=$quant count=1 skip=$i 2>/dev/null | md5sum)" != "$(dd if="$ifile" bs=$quant count=1 skip=$i 2>/dev/null | md5sum)" ]]; then
                ((j++))
        else
                ((i++))
        fi
done
((j>0)) && echo "Total of $j write error"
diff -q "$ifile" "$ofile"

It copies the file in blocks of $quant bytes (third parameter in the command line, 4MB if omitted) .

Earnestly · 2013-09-19 20:22:19

Or use rsync…

t0m5k1 · 2013-09-19 20:24:15

you could create split zip volumes!

stundrum · 2013-10-02 22:39:20

Thanks all for great suggestions, i've ended up with coping file to pendrive and doing file comparison with hex editor and overwriting bad blocks but i will sure check out your solutions in a future.

Arch Linux

#1 2013-09-19 17:34:28

[Solved] Torrent-like file copy

#2 2013-09-19 17:57:20

Re: [Solved] Torrent-like file copy

#3 2013-09-19 18:00:45

Re: [Solved] Torrent-like file copy

#4 2013-09-19 18:11:35

Re: [Solved] Torrent-like file copy

#5 2013-09-19 19:49:39

Re: [Solved] Torrent-like file copy

#6 2013-09-19 20:22:19

Re: [Solved] Torrent-like file copy

#7 2013-09-19 20:24:15

Re: [Solved] Torrent-like file copy

#8 2013-10-02 22:39:20

Re: [Solved] Torrent-like file copy

Board footer