Arch Linux

theking2

Banned

From: Romanshorn Switzerland

Registered: 2009-03-04

Posts: 372

scp traffic topped at 27MB/s over 1Gb Ethernet

I have the following situation
Two servers (Archlinux/QNAP) are connected over a dedicated 1Gb Ehternet connection without a switch.
I copy the complete contents of one server (Arch) to the other (QNAP).

When I issue one scp transaction it tops out at 27MB/s. but If I issue a second SCP of a different folder structure it also tops out a 27MB/s yielding a total of 55MB/s (according to the QNAP control panel).

I search a bit but could find a bandwidth limitation on scp or is there one?

Is there anotherway to utilies the optimal transfer speed between two servers over a dedicated Ethernet connection?

---

archlinux on a Gigabyte C1037UN-EU, 16GiB
a Promise  PDC40718 based ZFS set
root on a Samsung SSD PB22-J
running LogitechMediaServer(-git), Samba, MiniDLNA, TOR

R00KIE

Forum Fellow

From: Between a computer and a chair

Registered: 2008-09-14

Posts: 4,734

Re: scp traffic topped at 27MB/s over 1Gb Ethernet

Yes there is an implicit speed limit: how fast you can encrypt and decrypt the data.

If the network is secure use NFS instead.

---

R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

graysky

Wiki Maintainer

From: :wq

Registered: 2008-12-01

Posts: 10,696

Website

Re: scp traffic topped at 27MB/s over 1Gb Ethernet

https://bbs.archlinux.org/viewtopic.php?id=136713 gives a comparison of different ciphers FYI.

---

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packages • Zsh and other configs

Slithery

Administrator

From: Norfolk, UK

Registered: 2013-12-01

Posts: 5,776

Re: scp traffic topped at 27MB/s over 1Gb Ethernet

It's not just the encryption that adds an overhead, the compression does as well.

You can switch off compression using the -o 'Compression no' flag.

---

No, it didn't "fix" anything. It just shifted the brokeness one space to the right. - jasonwryan
Closing -- for deletion; Banning -- for muppetry. - jasonwryan

aur - dotfiles

Izzette

Member

Registered: 2015-10-02

Posts: 25

Re: scp traffic topped at 27MB/s over 1Gb Ethernet

You could pipe your file to ssh instead using -o Compression=no and use your own (faster compression), like this:

$ lz4 < file.to.transfer | ssh -o Compression=no user@host.domain bash -c 'lz4cat > transfered.file'

severach

Member

Registered: 2015-05-23

Posts: 198

Re: scp traffic topped at 27MB/s over 1Gb Ethernet

The encryption has nothing to do with it. Secure FTP (FTPS) is easily able to encrypt at gigabit wire speed. It all has to do with SSH being multiplexed.

StackOverflow: Why when I transfer a file through SFTP, it takes longer than FTP?

FTP and FTPS solve this by using a separate connection for the data. It is a good idea, but the way they did it cripples the protocol for IPv4-NAT.

To me the solution is so simple I'm surprised the SSH team hasn't already implemented it. The solution is a separate connection but absolutely not the way it was done for FTP or FTPS. Thanks to encryption and authentication everywhere and an already established connection over which pretty much anything can be communicated, it's easy to just use the same port number. The client and server to agree to split off the data stream and the server supplies a key to the client. The client opens another connection, authenticates with the key, and the data stream seamlessly switches over to the separate non multiplexed connection. Later the client and server can agree to terminate the separate data connection and the stream seamlessly switches back.