You are not logged in.

#1 2012-08-19 02:49:03

kdar
Member
From: Sweet home Alabama!
Registered: 2009-06-27
Posts: 356

Transferring file over network? what can I use besides ssh?

What can I use to transfer file over network besides using ssh?

This is only for experimental purposes. I have used ssh before, by using command like this:

(time cat /run/shm/test/input/totalInput.tar | ssh username@ip-server "cat > /dev/null") 2>>/run/shm/test/p3_UC_ctime_tar.txt

or this, if I tried to compress a file and then transfer:

(time bzip2 -kfc1 /run/shm/test/input/totalInput.tar | ssh username@ip-server "cat > /dev/null") 2>>/run/shm/test/p3_ctime_tar.txt

However, I think ssh is killing my transfer speed. I am getting only 10-20Mbps.. Router should support over 200-300Mbps.
What can I use besides ssh? I was thinking about netcat, but what I don't like is that I have to open port on the other side for each new file transfer.

Offline

#2 2012-08-19 03:06:56

skunktrader
Member
From: Brisbane, Australia
Registered: 2010-02-14
Posts: 1,538

Re: Transferring file over network? what can I use besides ssh?

man scp

Offline

#3 2012-08-19 03:07:43

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 19,739

Re: Transferring file over network? what can I use besides ssh?

Where to start.  Off the top of my head:  sftp. ftp. http. smb.  tftp. nfs.

Are you by chance confusing Bytes per second with Bits per second?
Why to you think 300 Mb/sec is a reasonable number.  What is you NIC? Your router? The NIC on the other computer? The disk on your computer?  The disk on the other computer?
Is the other computer on your subnet?  How many hops to the other computer? Are you depending on the Public Internet?


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#4 2012-08-19 03:18:08

kdar
Member
From: Sweet home Alabama!
Registered: 2009-06-27
Posts: 356

Re: Transferring file over network? what can I use besides ssh?

Well, what I am conserned with things like scp and ssh (I tried scp too) is encryption.
I just moved one file over to a server with scp, and I got about 2.1MB/s... which is about 16.8 Mbps..

I am using Linksys E900, which is wireless-N router. Connection exist between two computers only, nothing else is using that router. However, there is another wireless point outside of our room, which is used for public University network. I don't know how much of effect it would have on our set up.

Both computers have plenty of disk space. One computer (actually development board) uses sd-card for its OS and other disk space.

About 200-300Mbps.. is just something I saw on router's specs.

Last edited by kdar (2012-08-19 03:22:28)

Offline

#5 2012-08-19 04:11:32

fumbles
Member
Registered: 2006-12-22
Posts: 246

Re: Transferring file over network? what can I use besides ssh?

netcat and bbcp.

bbcp can get close to the theoretical limits with a few tweaks. I've managed to get 9.8Mbps with bbcp over 10Gb Ethernet (not on Linux though).

Wireless is never going to be friendly with large transfers. Wireless is a shard half-duplex connection so you best bet is to start the transfer (using whatever mechanism) and just wait longer. Wireless speeds also depend greatly on physical distance.

http://netcat.sourceforge.net
http://www.slac.stanford.edu/~abh/bbcp/

Last edited by fumbles (2012-08-19 04:15:20)

Offline

#6 2012-08-19 05:31:12

fukawi2
Ex-Administratorino
From: .vic.au
Registered: 2007-09-28
Posts: 6,217
Website

Re: Transferring file over network? what can I use besides ssh?

netcat...

At the server console:

$ nc -v -w 30 -p 5600 l- > filename.back

and on the client side:

$ nc -v -w 2 10.0.1.1 5600 < filename

http://oreilly.com/pub/h/1058

Use UDP for even higher throughput, although that may lead to file corruption on the receiving end if packets go missing, especially on larger files.

Offline

#7 2012-08-19 12:58:03

kdar
Member
From: Sweet home Alabama!
Registered: 2009-06-27
Posts: 356

Re: Transferring file over network? what can I use besides ssh?

ah right. I probably will get file corruption with UDP. So netcat is basically a tcp?

Is there away to do netcat transfer with just one command line from client pc (from where I want to send a file). How can I do this transfer if I don't have physical access to server (I am connected to development board with serial cable at the moment, so only have one tty).

Last edited by kdar (2012-08-19 13:00:31)

Offline

#8 2012-08-19 13:58:18

Trent
Member
From: Baltimore, MD (US)
Registered: 2009-04-16
Posts: 990

Re: Transferring file over network? what can I use besides ssh?

kdar wrote:

However, I think ssh is killing my transfer speed. I am getting only 10-20Mbps.. Router should support over 200-300Mbps.

I doubt ssh would have that much effect on the transfer speed, unless you're running on a very slow computer.

Do other things (HTTP downloads, for instance) transfer at this speed? What any arbitrary component supports is a theoretical maximum that rarely if ever gets met, especially over wireless. Wireless throughput depends on everything from the shape and composition of the room you're in to sources of interference that could be kilometers away.

What I'm saying is don't be surprised if using netcat doesn't change your throughput significantly.

Offline

#9 2012-08-20 02:45:52

kdar
Member
From: Sweet home Alabama!
Registered: 2009-06-27
Posts: 356

Re: Transferring file over network? what can I use besides ssh?

You were right... I tried to use netcat and it was almost the same transfer speeds. No different than ssh or scp

I might be getting low speeds since there are several other networks on the same channel (like channel 11).

channel 14 is free, but I will probably have to try to buy a new router that supports this channel number.

Last edited by kdar (2012-08-20 02:46:14)

Offline

#10 2012-08-20 05:06:15

ZekeSulastin
Member
Registered: 2010-09-20
Posts: 266

Re: Transferring file over network? what can I use besides ssh?

You'll probably have a much easier time finding an open channel in the 5 GHz channels than the 2.4 GHz, if you're talking about going to channel 14 as opposed to 1, 6, or 11; assuming you're still in Alabama, it will be ... rather entertaining to get everything in your household using it due to regulations.

Offline

#11 2012-08-20 07:21:32

Awebb
Member
Registered: 2010-05-06
Posts: 6,272

Re: Transferring file over network? what can I use besides ssh?

kdar wrote:

Is there away to do netcat transfer with just one command line from client pc (from where I want to send a file). How can I do this transfer if I don't have physical access to server (I am connected to development board with serial cable at the moment, so only have one tty).

Sure.

ssh user@remotehst "nc -v -w 2 10.0.1.1 5600 < filename" &

You should consider using GNU screen or tmux if you are in such a limited environment. That's what they are made for.

Online

#12 2012-08-20 14:06:59

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: Transferring file over network? what can I use besides ssh?

Well, with an atom n280 ssh/sftp is slow.
For pratical purposes, i use quickserve (aur), to open a webserver and then ssh into the remote box with links to the first to download files.


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

#13 2012-08-20 14:24:10

Trent
Member
From: Baltimore, MD (US)
Registered: 2009-04-16
Posts: 990

Re: Transferring file over network? what can I use besides ssh?

Even if there are no nearby devices using channel 11 (for instance), channel 11 is still susceptible to interference from channels 7 through 13. Channel 14 is the only totally non-overlapping channel but it's not available in North America by regulation (you may still be technically capable of using it but I don't know the legal status of doing so).

In any case, the speeds listed on wireless devices are the purest form of wishful thinking. If you want more predictable transfer speeds, you have to at least wire it up. (Not that wired networking is a guarantee by any means; it's still a crap shoot but weighted more heavily in favor of consistency.)

Offline

#14 2012-08-21 16:23:38

jwele
Member
Registered: 2012-07-03
Posts: 62

Re: Transferring file over network? what can I use besides ssh?

kdar wrote:

You were right... I tried to use netcat and it was almost the same transfer speeds. No different than ssh or scp

I might be getting low speeds since there are several other networks on the same channel (like channel 11).

channel 14 is free, but I will probably have to try to buy a new router that supports this channel number.


I can't speak for any specific program but usually programs use UDP for two main reasons. Speed and Speed.(cited from Beej) Using just UDP is unreliable but on an application level, there can be safeguards in place that make UDP a safe solution so long as sum checking is implemented on an application level.

Offline

#15 2012-08-21 17:23:20

kokoko3k
Member
Registered: 2008-11-14
Posts: 2,390

Re: Transferring file over network? what can I use besides ssh?

...i think that using kernel facilities or hardware error checking is better (more efficient) than rely on userspace tools, and using udp over a wifi network is really a bad idea.
If you trust in your connection stability, you could tweak tcp network parameters by allowing larger windows ( /proc/sys/net/ipv4/) and lowering the total overhead.
http://www.kernel.org/doc/Documentation … sysctl.txt

Last edited by kokoko3k (2012-08-21 17:25:14)


Help me to improve ssh-rdp !
Retroarch User? Try my koko-aio shader !

Offline

Board footer

Powered by FluxBB