You are not logged in.

#1 2021-07-16 22:56:05

squatched
Member
Registered: 2014-12-08
Posts: 12

[SOLVED] SSH Ignores Systemd Userspace SSH Agent Identities

I've setup my SSH agent as a Systemd service in user space as described in the wiki and when I try and use any of the identities from that SSH agent, SSH ignores them.

Here's a sample run of setting things up beginning to end:

# Showing the key works. In this example, I'm authenticating against Github.
$ ssh -T -i ~/.ssh/my_id_ed25519 git@github.com
Enter passphrase for key '~/.ssh/my_id_ed25519': 
Hi <username>! You've successfully authenticated, but GitHub does not provide shell access.

# The agent is running (according to systemd) and the environment is setup correctly.
$ systemctl --user status ssh-agent.service
● ssh-agent.service - SSH key agent
     Loaded: loaded (~/.config/systemd/user/ssh-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-07-16 10:31:10 PDT; 4h 54min ago
   Main PID: 1273 (ssh-agent)
      Tasks: 1 (limit: 18755)
     Memory: 988.0K
        CPU: 30ms
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/ssh-agent.service
             └─1273 /usr/bin/ssh-agent -D -a /run/user/1000/ssh-agent.socket

Jul 16 10:31:10 <hostname> ssh-agent[1273]: SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket; export SSH_AUTH_SOCK;
Jul 16 10:31:10 <hostname> ssh-agent[1273]: echo Agent pid 1273;
Jul 16 10:31:10 <hostname> systemd[1262]: Started SSH key agent.
$ echo $SSH_AUTH_SOCK
/run/user/1000/ssh-agent.socket

# Adding the key to the running agent
$ ssh-add -l
The agent has no identities.
$ ssh-add ~/.ssh/my_id_ed25519
Enter passphrase for ~/.ssh/my_id_ed25519: 
Identity added: ~/.ssh/my_id_ed25519 (<pubkey note>)
$ ssh-add -L
ssh-ed25519 <pubkey> <pubkey note>

# Attempting to use the key. Since it's in the agent, I now expect it to no longer ask for the password.
$ ssh -Tv -i ~/.ssh/my_id_ed25519 git@github.com
OpenSSH_8.6p1, OpenSSL 1.1.1k  25 Mar 2021
debug1: Reading configuration data ~/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to github.com [192.30.255.113] port 22.
debug1: Connection established.
debug1: identity file ~/.ssh/my_id_ed25519 type 3
debug1: identity file ~/.ssh/my_id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version babeld-4cec2db4
debug1: compat_banner: no match: babeld-4cec2db4
debug1: Authenticating to github.com:22 as 'git'
debug1: load_hostkeys: fopen ~/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-rsa SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8
debug1: load_hostkeys: fopen ~/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in ~/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<key hash> explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<key hash> explicit
debug1: Server accepts key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<key hash> explicit
Enter passphrase for key '~/.ssh/my_id_ed25519':

I then tried to setup the agent in a local environment and use that (rather than systemd userspace managed ssh agent), and everything works!

$ eval $(ssh-agent)
Agent pid 25949
$ echo $SSH_AUTH_SOCK
/tmp/ssh-XXXXXXmEOgSc/agent.25948
$ ssh-add -l
The agent has no identities.
$ ssh-add ~/.ssh/my_id_ed25519
Enter passphrase for ~/.ssh/my_id_ed25519: 
Identity added: ~/.ssh/my_id_ed25519 (<pubkey note>)
$ ssh-add -l
256 SHA256:<pubkey hash> <pubkey note> (ED25519)
$ ssh -T -i ~/.ssh/my_id_ed25519 git@github.com
Hi <username>! You've successfully authenticated, but GitHub does not provide shell access.

Just on the off chance that there might be a thread to tug on that I've missed, here's a failed attempt at connecting with 3x verbosity:

$ ssh -Tvvv -i ~/.ssh/my_id_ed25519 git@github.com
OpenSSH_8.6p1, OpenSSL 1.1.1k  25 Mar 2021
debug1: Reading configuration data ~/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '~/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '~/.ssh/known_hosts2'
debug2: resolving "github.com" port 22
debug3: ssh_connect_direct: entering
debug1: Connecting to github.com [192.30.255.113] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: Connection established.
debug1: identity file ~/.ssh/my_id_ed25519 type 3
debug1: identity file ~/.ssh/my_id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version babeld-4cec2db4
debug1: compat_banner: no match: babeld-4cec2db4
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to github.com:22 as 'git'
debug3: record_hostkey: found key type RSA in file ~/.ssh/known_hosts:1
debug3: load_hostkeys_file: loaded 1 keys from github.com
debug1: load_hostkeys: fopen ~/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc
debug2: MACs ctos: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none
debug2: compression stoc: none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-rsa SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8
debug3: record_hostkey: found key type RSA in file ~/.ssh/known_hosts:1
debug3: load_hostkeys_file: loaded 1 keys from github.com
debug1: load_hostkeys: fopen ~/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'github.com' is known and matches the RSA host key.
debug1: Found key in ~/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<pubkey hash> explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<pubkey hash> explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: ~/.ssh/my_id_ed25519 ED25519 SHA256:<pubkey hash> explicit
debug3: sign_and_send_pubkey: ED25519 SHA256:<pubkey hash>
debug3: sign_and_send_pubkey: signing using ssh-ed25519 SHA256:<pubkey hash>
Enter passphrase for key '~/.ssh/my_id_ed25519':

Thought maybe the socket had permissive permissions and was being ignored by SSH or something but, I don't think that's the case:

$ ls -lh /run/user/1000/ssh-agent.socket
srw------- 1 <user> <group> 0 Jul 16 15:36 /run/user/1000/ssh-agent.socket

Here's the contents of my systemd unit:

$ systemctl --user cat ssh-agent.service 
# ~/.config/systemd/user/ssh-agent.service
[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
# DISPLAY required for ssh-askpass to work
Environment=DISPLAY=:0
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target

There's no errors from the ssh-agent in journalctl, I changed ssh-add to be run with -d (debug) instead of -D in the systemd unit but that yielded nothing. Journalctl shows the identity being added, and then... nothing...

I'm at a loss... Any other ideas why this setup might not be working?

(Please note that I have scrubbed all output here, so if it seems weird that a "~" is used instead of the full user path, I changed it. I also replaced things like key hashes, names of keys, etc.)

Last edited by squatched (2021-07-17 01:13:41)

Offline

#2 2021-07-17 01:13:16

squatched
Member
Registered: 2014-12-08
Posts: 12

Re: [SOLVED] SSH Ignores Systemd Userspace SSH Agent Identities

Solved the issue! I use firejail and its default profile for ssh includes allowances for access to the default ssh-agent socket location (/tmp/ssh-*). So, I created the following:

$ cat ~/.config/firejail/ssh.local
# Allow a socket located in XDG_RUNTIME_DIR to be accessible.
whitelist ${RUNUSER}/ssh-agent.socket

Offline

Board footer

Powered by FluxBB