You are not logged in.
For years now I've been using dd and nc to take images of my machines. Network speeds, however, have not kept up with storage speeds, at least the hardware I'm willing to pay for. I'm looking for a way of parallelising the nc component of the following:
source machine:
dd if=/dev/sdx bs=1M | nc destinationhost 9001
destination machine:
nc -l -p 9001 | dd bs=1M of=/dev/sdx
With respect to the network I can set up within reason anything required.
My current plan is to write a program to reads from a file, splits the file reassembles it at the other end.
If anyone has any better suggestions I'm all ears otherwise, I'll paste a short python script or link to github/aur when I'm done.
Cheers
Last edited by a_user_named_dazza (2024-12-23 15:00:58)
Offline
On phone, hence short: That sounds like you should look into NIC teaming / LACP. Bundle NICs together and you'll get near(ish) the combined speed.
Offline
Thanks, I've managed to get something working with the following script
down.sh
"""
ip link del bond0 type bond
"""
up.sh
"""
bash down.sh
e1="eth0"
e2="eth0"
ip link add bond0 type bond ; sleep 0.5
ip link set bond0 type bond miimon 100 mode balance-rr ; sleep 0.5
ip link set $e1 down ; sleep 0.5
ip link set $e2 down ; sleep 0.5
ip link set $e1 master bond0 ; sleep 0.5
ip link set $e2 master bond0 ; sleep 0.5
ip link set bond0 up ; sleep 0.5
ip addr add 10.0.0.2/24 dev bond0 ; sleep 0.5
"""
Still working out what's going wrong with the script but if one or both interfaces fail to appear in "cat /proc/net/bonding/bond0" then run the script again.
From what I can tell, this still doesn't allow the speedup of a single tcp connection but when testing I found that both scp and ssh were able to acheive a transfer rate of 2Gbps, so it's solved my problem.
To replace nc with ssh in the context of "dd if=/dev/sdx bs=1M | nc destinationhost 9001" :
ssh user@remotehost cat /dev/sdx > file_on_local_host
Offline