You are not logged in.

#1 2018-02-22 03:01:46

Salkay
Member
Registered: 2014-05-22
Posts: 356

[Solved]sshd won't accept keys any more

I've previously been able to ssh into my work computer perfectly. Due to a corporate firewall, I maintain an ssh connection from my work client to my home server with the following command.

ssh -vvv home_server -R 45413:localhost:22 -N

Then, from my home server, I ssh into localhost on port 45413. I have password authentication disabled on my work computer's /etc/ssh/sshd_config. Authenticating with keys has worked fine in the past, with my last successful ssh on 15 Feb 2018. However, publickey authorisation now fails. From my home server:

$ ssh -vvv work-system
...
debug3: record_hostkey: found key type ECDSA in file /home/salkay/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys from [localhost]:45413
debug1: Host '[localhost]:45413' is known and matches the ECDSA host key.
debug1: Found key in /home/salkay/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /home/salkay/.ssh/salkay@work.id_rsa (0x16da468), explicit
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/salkay/.ssh/salkay@work.id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey).

Journalctl on my work computer isn't particularly informative.

Feb 22 13:31:48 work-hostname sshd[26289]: Connection closed by authenticating user salkay ::1 port 42182 [preauth]

I can enable password authentication in /etc/ssh/sshd_config (comment out PasswordAuthentication no), and confirm that connecting with a password still works. I've also checked the keys, and they still match (but I've not changed any ssh settings that I'm aware of). I've also checked /etc/ssh/sshd_config on my work computer, and it hasn't changed since it was last working.

My work computer is running Arch, and my home server is running Raspbian. However, I suspect it's an Arch issue, because I can ssh from my home server to other servers fine, using keys. My Raspbian server has only had four packages update since the fail (tomcat/vorbis related), and I keep my work Arch system up-to-date.

Last edited by Salkay (2018-02-27 00:30:40)

Offline

#2 2018-02-22 05:55:50

bulletmark
Member
From: Brisbane, Australia
Registered: 2013-10-22
Posts: 380

Re: [Solved]sshd won't accept keys any more

Just looks like a key error. You say you checked the keys "and they still match" but (with pw authentication enabled) do a ssh-copy-id from your home machine and then edit ~/.ssh/authorized_keys to check and edit out one of the keys.

BTW, that ssh tunnel will be unreliable to maintain. I suggest you run autossh on your work server which exists for exactly this.

Offline

#3 2018-02-22 06:34:31

Salkay
Member
Registered: 2014-05-22
Posts: 356

Re: [Solved]sshd won't accept keys any more

bulletmark wrote:

Just looks like a key error. You say you checked the keys "and they still match" but (with pw authentication enabled) do a ssh-copy-id from your home machine and then edit ~/.ssh/authorized_keys to check and edit out one of the keys.

Thank you for the reply. The strange thing is that I haven't changed anything ssh-related in the two systems. I should have mentioned, but I checked the keys by extracting the public key from the private with ssh-keygen -y -e -f <private key>, then comparing that public key with ~/.ssh/authorized_keys.

However, just in case I stuffed something up, I tested with ssh-copy-id as you suggest. It appended the same key to ~/.ssh/authorized_keys, but I still couldn't log in.

bulletmark wrote:

BTW, that ssh tunnel will be unreliable to maintain. I suggest you run autossh on your work server which exists for exactly this.

I actually have it set as a cronjob to restart if it goes down. I only discovered autossh later, but my cronjob seems to work fairly well.

Offline

#4 2018-02-26 06:55:16

Denharad
Member
Registered: 2015-11-18
Posts: 2

Re: [Solved]sshd won't accept keys any more

Just had this happen on one of my machines. Freaked me out as nothing should have changed recently. Did some sanity checks & found sshd didn't like my home directory having permissions for group or other. Chmod 700 to my home directory & it was working again. I scrubbed the filesystem & got no errors. Is there any reason this would suddenly happen other than a bad actor?

Offline

#5 2018-02-26 08:33:59

Salkay
Member
Registered: 2014-05-22
Posts: 356

Re: [Solved]sshd won't accept keys any more

Denharad wrote:

Just had this happen on one of my machines. Freaked me out as nothing should have changed recently. Did some sanity checks & found sshd didn't like my home directory having permissions for group or other. Chmod 700 to my home directory & it was working again. I scrubbed the filesystem & got no errors. Is there any reason this would suddenly happen other than a bad actor?

Is it the same as my issue? i.e. passwords work by keys don't? And are you doing reverse tunnelling too? Also, are you talking about the home directory on the client or server?

Offline

#6 2018-02-26 20:39:13

Maniaxx
Member
From: .de
Registered: 2014-05-14
Posts: 172

Re: [Solved]sshd won't accept keys any more

Its possible that your key/digest/cipher is too old and got blacklisted in the meantime (by OpenSSH). You would need to explicitly allow it on the server then (if you prefer to continue using it).
It should be mentioned in the server logs. Use debug mode on the server to find out (sshd -dd):

$ man sshd
     -d      Debug mode.  The server sends verbose debug output to standard error, and does not put itself in the
             background.  The server also will not fork and will only process one connection.  This option is only
             intended for debugging for the server.  Multiple -d options increase the debugging level.  Maximum is 3.

sys2064

Offline

#7 2018-02-27 00:30:15

Salkay
Member
Registered: 2014-05-22
Posts: 356

Re: [Solved]sshd won't accept keys any more

Maniaxx wrote:

It should be mentioned in the server logs. Use debug mode on the server to find out (sshd -dd):

Phew! Thank you! This was very useful. After stopping sshd via systemd, I ran it manually with sshd -ddd. I found the following output.

debug1: temporarily_use_uid: 1000/1000 (e=0/0)
debug1: trying public key file /home/salkay/.ssh/authorized_keys
debug1: Could not open authorized keys '/home/salkay/.ssh/authorized_keys': Permission denied

The permissions were fine (700 on the directory, 600 on the file). However, I have an encfs that contains the ~/.ssh directory. This is then symlinked from the decrypted directory to ~/.ssh. I unlock and mount the encfs when I initially boot the system at work. Hence, when I try to log in remotely, this filesystem is already accessible. However, after I moved the .ssh directory from the decrypted directory to ~/.ssh, I could ssh in fine! The problem is that sshd can't read from the (decrypted) encfs.

I discovered that even sudo cannot access a decrypted encfs. I presume that even though sshd is emulating the user, for some reason it's not doing it fully. A workaround is to either mount the encfs with --public (which will require sudo), or modify the default FUSE settings to add allow_other.

I wasn't happy with the former, and the latter was maybe a security issue (?), so instead, I changed the location of the authorized key file in /etc/ssh/sshd_config. This now works fine! Thank you for the tip.

Offline

Board footer

Powered by FluxBB