[ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

pepineros · 2022-02-19 19:14:22

Hi all,

The goal
I have a little homelab and a laptop, both running Arch. I want to mount a partition from the homelab on my laptop, so I can see the files on that partition and interact with them, over the network (without transferring the files to the laptop).

What works
I can SSH from client to server by running ssh -p port user@ip (I'm not using port 22).

Using Filezilla, I can access the entire filesystem on the server from my laptop once I enter the appropriate credentials for the user, server, and port.

What doesn't work
Both SCP and SSHFS just hang when I run them, whatever command I try to execute with either of them. I tried leaving it for about an hour and nothing happened.
I've checked that /etc/ssh/sshd_config on the server has
'Subsystem sftp /usr/lib/ssh/sftp-server'
and that /usr/lib/ssh/sftp-server is a valid path.

I tried changing the port back to default and run the commands again without specifying a special port. No change in behaviour from either.
Note that if I use the wrong port when executing SCP, it correctly exits with "connect to host ip port 22: Connection refused".

Output when running SSHFS:

SSHFS version 3.7.2
executing <ssh> <-x> <-a> <-oClearAllForwardings=yes> <-oPort=[port]> <-2> <[user]@[ip]> <-s> <sftp>

Another thing to note is that when I use SSHFS to mount the remote directory on a folder in home, and then try to run ls on home, the terminal just hangs.

The only other change I made to Arch's default sshd_config is to allow Xforwarding, but changing that back to the default did not make a difference to scp either (as expected).

Running scp with -vvv for debugging info gives a lot of output but nothing that indicates something is wrong. The last few lines are:

sending command: scp -v -f /path/to/sourcefile
channel 0: request exec confirm 1
send packet: type 98
channel_input_open_confirmation: channel 0: callback done
channel 0: open confirm rwindow 0 rmax 32768

Everything before that seems to only indicate that a connection is established successfully and I'm authenticated with my SSH key.

I notice that when I SSH to the server with '-vvv', there are a few more lines of output indicating that a packet of type 99 is received back from the server. However I was not able to find any info online about this specific line of output.

I'm at a loss as to what may be causing this. There doesn't seem to be any problem with connection or authentication, but also no hint - as far as I can see - why SCP or SSHFS don't want to play nice.

I'm happy to investigate further, but a hint about where to start would be much appreciated.

Resources I looked at:
Forum post https://bbs.archlinux.org/viewtopic.php?id=27613
forum post https://bbs.archlinux.org/viewtopic.php?id=270555
Wiki https://wiki.archlinux.org/title/SSHFS
Wiki https://wiki.archlinux.org/title/SCP_and_SFTP

Thank you!!

Last edited by pepineros (2022-02-23 21:14:55)

paulkerry · 2022-02-19 21:31:43

It could be a per-user dot file configuration error.
You say you can...
ssh -p port user@ip
so make sure when you do that, you've not added anything to the user dot files for the shell you are using at the "ip" end.
You could also test if the same thing happens with a different or newly created user at the "ip" end - a new user will normally have unedited dot files.

seth · 2022-02-19 21:46:44

Can you "sftp -P port user@ip"?

pepineros · 2022-02-20 06:53:01

paulkerry wrote:

It could be a per-user dot file configuration error. (...) You could also test if the same thing happens with a different or newly created user at the "ip" end - a new user will normally have unedited dot files.

Created a new test user on the server side with no dotfiles whatsoever (just did sudo adduser mytestuser and created a home folder myself, chown-ed it to the new user). I can SSH in with this new user's name and password, but the SCP and SSHFS behaviour is exactly the same.

Should I create a default .bashrc at least for that user, or is no dotfiles at all the way to go?

seth wrote:

Can you "sftp -P port user@ip"?

No, SFTP has the exact same behaviour as SCP and SSHFS, including the same about from adding -vv flags. Which is odd, considering that Filezilla also uses SFTP and did connect without any issues.

I'm stumped!

-thc · 2022-02-20 07:34:29

The default sshd_config only contains 5 active lines:

AuthorizedKeysFile	.ssh/authorized_keys
KbdInteractiveAuthentication no
UsePAM yes
PrintMotd no # pam does that
Subsystem	sftp	/usr/lib/ssh/sftp-server

Please compare it to the sshd_config of your SSH server.

Which authentication method do you use? Keys? Keys only? What kind of keys?

pepineros · 2022-02-20 11:07:45

-thc wrote:

The default sshd_config only contains 5 active lines:

Output of cat /etc/ssh/sshd_config | grep -v ^\# with empty lines removed:

Port 40004
AuthorizedKeysFile	.ssh/authorized_keys
KbdInteractiveAuthentication no
UsePAM yes
PrintMotd no # pam does that
Subsystem	sftp	/usr/lib/ssh/sftp-server

I tested with the Port 40004 line commented out already, but this did not change the behaviour.

-thc wrote:

Which authentication method do you use? Keys? Keys only? What kind of keys?

I use the default size RSA keys for my own user and password for the new testuser that I created earlier. Same behaviour again either way.

Last edited by pepineros (2022-02-20 11:08:18)

seth · 2022-02-20 13:23:45

Silent failure (there's nothing in the server logs?) would usually suggest a packet filter or maybe an MTU issue, but I can't square that w/ the filezilla situation (except if you were maybe using ftps or ftp) - nor does anything but a very elaborative packet filter explain why ssh itself would still work.

Is there nothing in the server logs, maybe if you raise the log level?
https://wiki.archlinux.org/title/OpenSSH#Checklist

-thc · 2022-02-20 17:48:35

pepineros wrote:

I use the default size RSA keys for my own user and password for the new testuser that I created earlier. Same behaviour again either way.

Very strange - I tested sshfs against a newly set up virtual Arch SSH-host with ed25519-keys and it works instantly.

pepineros · 2022-02-23 21:14:06

All, thank you for the very helpful responses! I found that in some cases, the issue is resolved if I turn off the wifi adapter and turn it back on (ip link set wlan0 down/up). However, the issue always resolved itself if I reboot my router and then try again. SCP, SSHFS, and SSH all work flawlessly at that point.

It doesn't seem to be an issue with leases - the DHCP lease time is 4 hours and it fails much more quickly than that after a router reboot. Also both the server and client are on static IP addresses anyway, so DHCP lease time should not be a factor at all.

Either way, it isn't Arch that's causing this behaviour!

Arch Linux

#1 2022-02-19 19:14:22

[ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#2 2022-02-19 21:31:43

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#3 2022-02-19 21:46:44

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#4 2022-02-20 06:53:01

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#5 2022-02-20 07:34:29

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#6 2022-02-20 11:07:45

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#7 2022-02-20 13:23:45

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#8 2022-02-20 17:48:35

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

#9 2022-02-23 21:14:06

Re: [ISOLATED - not Arch] SCP and SSHFS both hang trying to access homelab

Board footer