You are not logged in.

#1 2022-05-13 06:35:36

makh
Member
Registered: 2011-10-10
Posts: 299

[SOLVED] Faster data transfer between two pcs

Hi

Usually in the past we had good ol' desktops with IDE or sata ports. Simply plug out the disk and put it in to the other pc. Data used to be small and the transfer was perhaps fastest. Only bottleneck to attach & detach the desktop screws and hard disk.

Now with 500GB or plus hard disks and using laptop, isnt that simple; I havent opened any laptop yet!!!

I searched around to find that the ssh was a decent option; but it maxed to 2.5 Mbps or 3 Mpbs with slowing down to even 250 kBps; using my usual router. I stopped the process; my external may easily do that from 11 to 60/80 mbps, but with two steps.

Anyways: is there some other method I couldnt get to know of to transfer data fast directly; usually I dont get to do such a task often; but still if some thing can play magic and save time; the next time possibly!

Thanks in advance!

Last edited by makh (2022-06-14 09:06:29)


OS:  Arch  &/  Debian
System: LENOVO ThinkPad E14
Desktop: Xfce

Offline

#2 2022-05-13 14:48:46

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,740

Re: [SOLVED] Faster data transfer between two pcs

I assume you have a high bandwidth link between the machines.  At least 100 Mb/s, preferably 1000 Mb/s
Just as a sanity check, did ssh (probably sftp) report in Mb/s (megabits per second) or in MB/s (megabytes per second)?

You might try rsync.   
If that is not fast enough, you might try setting up a ]NFS

Neither of those are encrypted.  rync can be made to use an ssh tunnel but, encryption will probably slow things down a little.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#3 2022-05-13 23:05:02

makh
Member
Registered: 2011-10-10
Posts: 299

Re: [SOLVED] Faster data transfer between two pcs

ewaller wrote:

I assume you have a high bandwidth link between the machines.  At least 100 Mb/s, preferably 1000 Mb/s
Just as a sanity check, did ssh (probably sftp) report in Mb/s (megabits per second) or in MB/s (megabytes per second)?

You might try rsync.   
If that is not fast enough, you might try setting up a ]NFS

Neither of those are encrypted.  rync can be made to use an ssh tunnel but, encryption will probably slow things down a little.

Hi,

I think probably the router is 100 MBps, not more.

2. ssh used MB/s (megabytes per second)

3. I shall give rsync a shot, just to test it.

4. I dont know how: but probably if one machine is set as hotspot, probably it would have been pretty faster; though I dont know how to do that yet!

5. I understand: You mean that rsync will be used via samba/NFS by default, but could be used with ssh also.

6. NFS is also a definite option. But my concern was that since ssh didnt take much speed; than NFS will not play pretty better either!

7. Looking around more: someone suggests to use USB 3 to USB 3 cable or USB bridged cable for the fastest transfer; but theres no easy application to do that in linux. A utility usbnet needs to be configured and given IP somehow. [PRECAUTION: Do not connect USB 1.x or USB 2.x directly; use a Bridged cable. But USB 3.x can be connected directly]

Thank you

Edit-1: Point #7 above.

Last edited by makh (2022-05-13 23:11:11)


OS:  Arch  &/  Debian
System: LENOVO ThinkPad E14
Desktop: Xfce

Offline

#4 2022-05-14 00:18:25

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,740

Re: [SOLVED] Faster data transfer between two pcs

To put things into perspective, ssh reporting 3MB/s-ish implies you are seeing 30Mb/s or better on your Ethernet link.


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#5 2022-05-14 00:28:59

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: [SOLVED] Faster data transfer between two pcs

makh wrote:

I havent opened any laptop yet!!!

And I've not had my hands on one that I've not opened.  Generally swapping out drives in a laptop is pretty strait forward ... so long as it's not an Apple.  I'f it's an Apple, don't even think about opening it if you value your sanity.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#6 2022-05-14 01:42:24

jonno2002
Member
Registered: 2016-11-21
Posts: 684

Re: [SOLVED] Faster data transfer between two pcs

you never actually stated your using ethernet, are those speeds using wireless ? if so use ethernet cables

Offline

#7 2022-05-14 05:08:12

seth
Member
Registered: 2012-09-03
Posts: 49,997

Re: [SOLVED] Faster data transfer between two pcs

https://www.amazon.com/s?k=sata+usb+adapter
Ok, I guess *both* PCs are notebooks?

Make sure to enhance the network connection - 100Mbps will never be "fast" and 1Gbps has been the ethernet "standard" for a loooooooong time.
So if you've two new-ish notebooks the weak link is the carrier (wifi) or the switch (router).

Last edited by seth (2022-05-14 05:12:50)

Online

#8 2022-06-14 09:06:03

makh
Member
Registered: 2011-10-10
Posts: 299

Re: [SOLVED] Faster data transfer between two pcs

Hi,

I wasnt getting the time but at last...!

Though I dont need it right now; but tested it for future use:
This wiki was incomplete:
https://wiki.archlinux.org/title/Networ … ridged_LAN
Seeking point from:
https://bbs.archlinux.org/viewtopic.php?id=160796
This wasnt helpful as ifconfig is not available in Arch:
https://www.softprayog.in/troubleshooti … -lan-cable
I couldnt understand this one:
https://wiki.archlinux.org/title/Internet_sharing

Connect by the Cross Cable: lights should blink on both ports.
(Router method may perhaps be easier, Network Manager by default only connects to one connection, it doesnot allow to use )

Machine -1:

ip link
ip link set up dev enp4s0
(use the link name you want to make up; the ethernet device name may vary)
ip addr add 192.168.0.1/24 dev enp4s0

Machine -2:

ip link
ip link set up dev enp0s25
ip addr add 192.168.0.2/24 dev enp0s25

check connection from both the machines:

ping -c3 192.168.0.1
ping -c3 192.168.0.2

Should be OK...

ssh is not working... I am unable to connect to the machine!

systemctl enable sshd
systemctl start sshd

In xfce4, ssh using the thunar.
ssh@192.168.0.1
It will prompt for password.

Copy paste files speed @ 41 MBPs...!!!

Data transfer using scp etc should also be working; but I didnt test it.

Seems to work after a struggle and search of 5 hours!

Do let me know if something shorter or faster is possible.

Thankyou


OS:  Arch  &/  Debian
System: LENOVO ThinkPad E14
Desktop: Xfce

Offline

#9 2022-06-15 02:12:45

dawnofman
Member
Registered: 2019-07-26
Posts: 140

Re: [SOLVED] Faster data transfer between two pcs

makh wrote:

Hi

Usually in the past we had good ol' desktops with IDE or sata ports. Simply plug out the disk and put it in to the other pc. Data used to be small and the transfer was perhaps fastest. Only bottleneck to attach & detach the desktop screws and hard disk.

Now with 500GB or plus hard disks and using laptop, isnt that simple; I havent opened any laptop yet!!!

I searched around to find that the ssh was a decent option; but it maxed to 2.5 Mbps or 3 Mpbs with slowing down to even 250 kBps; using my usual router. I stopped the process; my external may easily do that from 11 to 60/80 mbps, but with two steps.

Anyways: is there some other method I couldnt get to know of to transfer data fast directly; usually I dont get to do such a task often; but still if some thing can play magic and save time; the next time possibly!

Thanks in advance!

My two-cents: if you need to transfer a huge amount of data between two laptops AND you can live without internet access for the time you are transferring the data:

- 1. make a point-to-point wired ethernet connection between the laptops: static addresses
- 2. make one NFS read-write share on the target laptop (easy; almost all default options) (could advice you otherwise)
- 3. connect to that share from the source laptop and use rsync

No need for SSH nor encryption not anything slowing you down because you are on a private point-to-point connection so none else have access at all.

For gigabit ethernet connections expect 100 MB/s or so.

Offline

#10 2022-06-19 10:59:46

makh
Member
Registered: 2011-10-10
Posts: 299

Re: [SOLVED] Faster data transfer between two pcs

dawnofman wrote:

My two-cents: if you need to transfer a huge amount of data between two laptops AND you can live without internet access for the time you are transferring the data:

- 1. make a point-to-point wired ethernet connection between the laptops: static addresses
- 2. make one NFS read-write share on the target laptop (easy; almost all default options) (could advice you otherwise)
- 3. connect to that share from the source laptop and use rsync

No need for SSH nor encryption not anything slowing you down because you are on a private point-to-point connection so none else have access at all.

For gigabit ethernet connections expect 100 MB/s or so.

Hi,
Thanks, I think NFS should be very faster, but never got a situation to try to use it.

Shouldnt simple cp / mv be faster then rsync?


Thankyou


OS:  Arch  &/  Debian
System: LENOVO ThinkPad E14
Desktop: Xfce

Offline

#11 2022-06-19 13:41:37

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,449
Website

Re: [SOLVED] Faster data transfer between two pcs

makh wrote:

Shouldnt simple cp / mv be faster then rsync?

Assuming it works perfectly the first time, then a cp / mv is likely to be of comparable speed to rsync.  If it's any faster it would be only by a trivial margin.  But the assumption here is the big issue.  If anything goes wrong, rsync will allow you to restart and complete the transfer where the other two will be much less convenient.  So the overall time spent for a complete transfer will either be comparable between the two or much in favor of rsync.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#12 2022-06-20 17:29:29

dawnofman
Member
Registered: 2019-07-26
Posts: 140

Re: [SOLVED] Faster data transfer between two pcs

Trilby wrote:
makh wrote:

Shouldnt simple cp / mv be faster then rsync?

Assuming it works perfectly the first time, then a cp / mv is likely to be of comparable speed to rsync.  If it's any faster it would be only by a trivial margin.  But the assumption here is the big issue.  If anything goes wrong, rsync will allow you to restart and complete the transfer where the other two will be much less convenient.  So the overall time spent for a complete transfer will either be comparable between the two or much in favor of rsync.

Great. I didn't know rsync could resume since I am starting to use it. Thanks for the tip.

Offline

#13 2022-06-20 17:38:01

dawnofman
Member
Registered: 2019-07-26
Posts: 140

Re: [SOLVED] Faster data transfer between two pcs

makh wrote:

Shouldnt simple cp / mv be faster then rsync?

If you ever end implementing it be aware of (possible): mv command fails silently for 4.2 GB file at 450 MB

Or, if you have some little spare time, please, try to move a large file to see what happens -I am having issues with mv and large files and I am suspecting something funny is going on --I expect to try to do the same on a totally different network on a distinct distro in a week-or-so, so any feedback on the matter will be welcome.

Offline

Board footer

Powered by FluxBB