You are not logged in.
- Topics: Active | Unanswered
Pages: 1
#1 2022-07-04 10:49:12
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Opening port for bbcp
I'm trying to copy large volumes of data from one Arch box to another on my LAN
bbcp -r -P 2 -v -n /localsource/* 192.168.1.88:/targetpathAfter providing the password for the target server I get the following:
bbcp: Indexing files to be copied...
bbcp: Found 3461 files and 0 links in 376 directories (0 empty) so far...
bbcp: Copying 4514 files and 0 links in 474 directories.
bbcp: Connection refused unable to connect to port localhost:5031
bbcp: Unable to allocate more than 0 of 4 data streams.I'm running a default Arch install, no firewall added (relying on fiber router/modem).
ssh is installed, enabled and functional.
What do I need to do to open the necessary ports for bbcp to do its business?
Last edited by audiomuze (2022-07-08 23:26:40)
Linux user #338966
Offline
#2 2022-07-04 13:59:42
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
Sanity check: is a bbcp instance running on the remote system ('cause something needs to be listening on the other side)?
Offline
#3 2022-07-04 14:33:12
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
Sanity check: is a bbcp instance running on the remote system ('cause something needs to be listening on the other side)?
By default, bbcp uses ssh, as determined by your PATH environmental variable, for authentication on every host that was specified in the source and target specifications.
In order to speed ssh start-up, bbcp disables ssh X11 forwarding (-x), disables the forwarding of the authentication agent connection (-a), and disables the use of rsh when ssh authentication fails (-o).
bbcp executes a copy of itself on the source node as “bbcp SRC” and a mirror copy on the target node as “bbcp SNK”.
Because the commands are well known, you may restrict ssh usage to exactly these commands when a password-less key-file is used to gain access to a host.
The –I option provides a mechanism to specify the location of the identity file should it not reside at the default location.It's installed on both machines. I've added the following to /etc/services on both (this change was made after my first post, and both servers have been restarted. The error message is identical, other than that it now refers to port 60000):
bbcpfirst 60000/tcp
bbcplast 60500/tcp1.8 Dealing With Firewalls (-z and -Z)
bbcp is a peer-to-peer application. Mainly, this means that copies of bbcp on the source and sink nodes appear to be both client as well as server applications .
This may not be possible at some sites because of firewall restrictions. Specifically, some installation may prohibit incoming TCP/IP connections at arbitrary ports.
Normally, bbcp source nodes will connect to their counterpart running on the target node. If the target host prohibits incoming connections, the copy will fail.
However, should the source host allow arbitrary connections, you can specify the –z option.
This option reverses the connection protocol so that the bbcp sink node will always try to connect to its counterpart running on the source host.
When the source and target nodes prohibit arbitrary connections, you will need assistance of an administrator at either node. If the --port option (-Z) is not specified, bbcp checks the /etc/services file for the existence of two services: bbcpfirst and bbcplast.
The bbcpfirst service identifies the starting port number and bbcplast identifies the ending port number that can be used for incoming connections.
When --port option (-Z) is not specified and neither service name can be found, bbcp resorts to using an arbitrary port number.
If the services are found, bbcp restricts its port usage to one of the ports in the indicated range.
Ask the administrator at the source or target nodes to allow a range of well-known port numbers to be used for incoming connections (i.e., allowed to pass through the firewall).
This will require that the administrator register these port numbers in the /etc/services file using the names bbcpfirst and bbcplast (the default names can be changed).
Make sure that at least 8 port numbers exist in the range (more if possible).
If restricted port access is only allowed in the source site, you must specify the –z option when invoking bbcp.More here: https://confluence.slac.stanford.edu/pa … sions=true
Last edited by audiomuze (2022-07-04 14:51:06)
Linux user #338966
Offline
#4 2022-07-04 15:04:08
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
Can you principally ssh into 192.168.1.88 ?
Did you try the "-z" option?
This is on a LAN, right? There's unlikely any port restriction but for a local firewall/iptables.
And does bbcp start on the remote host?
Monitor
netstat -tlEon either system.
Offline
#5 2022-07-04 15:36:05
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
Can you principally ssh into 192.168.1.88 ?
yes, I use it all the time to manage .88
Equally I am able to ssh to .114 from .88
This is on a LAN, right? There's unlikely any port restriction but for a local firewall/iptables.
Correct, LAN. No firewall enabled on either machine.
And does bbcp start on the remote host?
Monitornetstat -tlEon either system.
[x@alib ~]$ ss -tlE
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 0 *:* *:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 0 *:60000 *:*
UNCONN 0 0 192.168.1.88:ssh 192.168.1.114:42860[x@ryzendesktop ~]$ ss -tlE
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 0 *:* *:*
UNCONN 0 0 *:* *:*
UNCONN 0 3 *:56080 [::ffff:192.168.1.88]:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 3 *:36886 [::1]:*
UNCONN 0 0 192.168.1.114:42860 192.168.1.88:ssh
UNCONN 1 0 192.168.1.114%enp9s0:37788 95.216.195.133:http
UNCONN 0 0 [2a0e:3700:2043:f500:3fc0:6f34:d8e1:eb0d]%enp9s0:44746 [2a01:4f9:c010:2636::1]:httpDid you try the "-z" option?
Yes
UNCONN 0 0 *:* *:*
UNCONN 0 0 *:* *:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 0 192.168.1.114:34550 192.168.1.88:ssh
UNCONN 0 0 *:60000 *:* Linux user #338966
Offline
#6 2022-07-04 20:35:26
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
If you didn't change the range on ryzendesktop as well
UNCONN 0 3 *:56080 [::ffff:192.168.1.88]:* is probably it's bbcp instance.
Then there's clearly
UNCONN 0 0 *:60000 *:* on alib which would be its bbcp and there's als a matching ssh connection on both ends
UNCONN 0 0 192.168.1.114:42860 192.168.1.88:ssh
UNCONN 0 0 192.168.1.88:ssh 192.168.1.114:42860On 192.168.1.88 (alib) run
nc -lp 60000and on 192.168.1.114
nmap -p 60000 192.168.1.88This should™ report the port to be open and netcat on 192.168.1.88 to immediately terminate (that's ok - the port should just not show up as filtered)
Offline
#7 2022-07-04 22:36:01
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
If you didn't change the range on ryzendesktop as well
Confirmed both changed. On both systems:
cat /etc/servicesyields (at the end):
bbcpfirst 60000/tcp # bbcp
bbcplast 60500/tcp # bbcp
[x@ryzendesktop ~]$bbcpfirst 60000/tcp # bbcp
bbcplast 60500/tcp # bbcp
[x@alib ~]$ On ryzendesktop:
$ nmap -p 60000 192.168.1.88
Starting Nmap 7.92 ( https://nmap.org ) at 2022-07-04 23:34 BST
Nmap scan report for 192.168.1.88
Host is up (0.00027s latency).
PORT STATE SERVICE
60000/tcp open unknown
Nmap done: 1 IP address (1 host up) scanned in 0.08 secondsOn 192.168.1.88 (alib):
$ ncat -lp 60000
[x@alib ~]$Last edited by audiomuze (2022-07-04 22:39:18)
Linux user #338966
Offline
#8 2022-07-05 07:28:24
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
So the port (range) isn't blocked in any way, but if you had effectively restricted the port range before the previous test, the conneciton on *:56080 doesn't belong to bbcp and in that case bbcp didn't have an open port on ryzendesktop.
When you now compare the "bbcp: Connection refused unable to connect to port localhost:xxxxx" error to the netstat events, does the "xxxxx" port show up anywhere?
Offline
#9 2022-07-07 08:28:17
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
So the port (range) isn't blocked in any way, but if you had effectively restricted the port range before the previous test, the conneciton on *:56080 doesn't belong to bbcp and in that case bbcp didn't have an open port on ryzendesktop.
When you now compare the "bbcp: Connection refused unable to connect to port localhost:xxxxx" error to the netstat events, does the "xxxxx" port show up anywhere?
Regarding ports, other than the additions to /etc/services on both machines everything is as configurted in a default Arch install because my needs are genewraly adequately catered fror being able to ssh from my desktop into other servers on the lan to un updates, install the odd package etc. I'll retun the lot and post the results here. I recall reading something aouut IPv4 and IPV6 in bbcp's rather sparse documentation - will revist that also to see whether there's something I've overlooked.
Linux user #338966
Offline
#10 2022-07-07 20:57:03
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
$ bbcp -r -P 2 -v ./0GWGP/* x@192.168.1.88:/alib/ext4f/zzl/0-CurrentWeek
x@192.168.1.88's password:
bbcp: Host [::ffff:192.168.1.88] redirect connection to localhost.localdomain [::1]
bbcp: Indexing files to be copied...
bbcp: Found 2422 files and 0 links in 230 directories (0 empty) so far...
bbcp: Copying 4514 files and 0 links in 474 directories.
bbcp: Connection refused unable to connect to port localhost:60000
bbcp: Unable to allocate more than 0 of 4 data streams.
bbcp: Accept timed out on port 60000
bbcp: Unable to allocate more than 0 of 4 data streams.[x@ryzendesktop ~]$ ss -tlE
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 0 *:* *:*
UNCONN 0 0 *:* *:*
UNCONN 0 3 *:57588 [::ffff:192.168.1.88]:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 3 *:44144 [::1]:*
UNCONN 0 0 192.168.1.114:43004 192.168.1.88:ssh
UNCONN 0 0 192.168.1.114%enp9s0:55836 95.216.195.133:http
UNCONN 0 0 [2a0e:3700:2043:f500:6544:c2b4:6bb1:cdb1]%enp9s0:52234 [2a01:4f9:c010:2636::1]:http [x@alib ~]$ ss -tlE
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 0 *:* *:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 0 *:60000 *:*
UNCONN 0 0 192.168.1.88:ssh 192.168.1.114:43004 I'm not seeing 60000 on ryzendesktop.
Last edited by audiomuze (2022-07-07 20:58:22)
Linux user #338966
Offline
#11 2022-07-07 21:19:34
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
Check
ss -tEon the source (edit: ie "ryzendesktop")
Last edited by seth (2022-07-07 21:19:58)
Offline
#12 2022-07-08 06:28:58
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
Check
ss -tEon the source (edit: ie "ryzendesktop")
ryzendesktop:
$ ss -tE
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
UNCONN 0 1 [2a0e:3700:2043:f500:3fc0:6f34:d8e1:eb0d]:48296 [2a01:4f8:c2c:b1cf::1]:https
UNCONN 0 0 *:* *:*
UNCONN 0 0 *:* *:*
UNCONN 0 3 *:44138 [::ffff:192.168.1.88]:*
UNCONN 0 0 0.0.0.0:* 0.0.0.0:*
UNCONN 0 3 *:43174 [::1]:*
UNCONN 1 0 192.168.1.114:39156 192.168.1.88:ssh ryzendesktop:
x@192.168.1.88's password:
bbcp: Host [::ffff:192.168.1.88] redirect connection to localhost.localdomain [::1]
bbcp: Indexing files to be copied...
bbcp: Found 4070 files and 0 links in 433 directories (0 empty) so far...
bbcp: Copying 4514 files and 0 links in 474 directories.
bbcp: Connection refused unable to connect to port localhost:60000
bbcp: Unable to allocate more than 0 of 4 data streams.
bbcp: Accept timed out on port 60000
bbcp: Unable to allocate more than 0 of 4 data streams.Linux user #338966
Offline
#13 2022-07-08 07:06:03
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
And broadsword: *Any* port 6000?
ss -aEThough it becomes increasingly clear that bbcp "kinda forgot" to open a local port.
Is there a specific reason why you're not using rsync?
Offline
#14 2022-07-08 18:44:14
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
And broadsword: *Any* port 6000?
ss -aEThough it becomes increasingly clear that bbcp "kinda forgot" to open a local port.
Is there a specific reason why you're not using rsync?
throughput speed - copying/moving terabytes across a network is soul destroying using regular methods.
And would you believe I got it working! I hadn't enabled systemd-resolved on either machine. As soon as I did that it worked. Copied 78GB across at 114MB/s using CAT5E.
So in simple terms the solution was to enable and start systemd-resolved on both machines:
systemctl enable systemd-resolved
systemctl start systemd-resolvedThanks for your help, it is much appreciated.
Last edited by audiomuze (2022-07-08 22:45:49)
Linux user #338966
Offline
#15 2022-07-08 20:01:40
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
I kinda don't see how DNS resolution plays into this, but "yay" anyway.
Sanity check: Does "hostnamectl" show that you've a static hostname?
Offline
#16 2022-07-08 22:45:03
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
Sanity check: Does "hostnamectl" show that you've a static hostname?
it does, on both machines.
I kinda don't see how DNS resolution plays into this, but "yay" anyway.
I think the answer lies in /etc/hosts config on ryzendesktop (hostname not assigned to localhost):
alib:
#<ip-address> <hostname.domain.org> <hostname>
127.0.0.1 localhost.localdomain localhost alib
::1 localhost.localdomain localhost alib
127.0.1.1 alib.localdomain alibryzendesktop:
127.0.0.1 localhost
::1 localhost
127.0.1.1 ryzendesktopSo fixing that should have it work without resorting to systemd-resolved.
Last edited by audiomuze (2022-07-08 22:47:26)
Linux user #338966
Offline
#17 2022-07-08 23:25:58
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
And after all that I spoke too soon. Madew the changes to /etc/hosts, power cycled ryzendesktop and I'm back to square 1.
Last edited by audiomuze (2022-07-08 23:26:26)
Linux user #338966
Offline
#18 2022-07-09 07:41:43
- seth
- Member
- Registered: 2012-09-03
- Posts: 60,805
Re: Opening port for bbcp
Despite resolved??
Offline
#19 2022-07-09 08:46:30
- audiomuze
- Member
- Registered: 2014-03-31
- Posts: 103
Re: Opening port for bbcp
Despite resolved??
It worked once. Now back to what was. Looking at my bash history it seems
bbcp -z -f -r -P 2 --port 60000:60001 -v /source/* x@192.168.1.88:/target/did the trick. Will confirm when I have next batch to move.
Linux user #338966
Offline
Pages: 1