[SOLVED] ssh-agent started as user service unable to read in ssh keys

Cbhihe · 2024-02-19 20:45:53

Running kernel 6.7.5.arch1-1, OpenSSH 9.6p1-1, with up to date Gnome on Xorg.

I normally ssh into distant servers having configured an ssh-agent with ssh-keygen generated asymmetric keys, each pair with its pass-phrase.
This morning booting afresh and starting a new Gnome session, ssh-agent did not start automatically as usual, and ssh'ing into distant servers was back to the old username and passwd routine.
So I launched ssh-agent, eval "$(ssh-agent -s)", checked the configuration in ~/.ssh/config, added anew two keys to the agent's cache ssh-add ~/.ssh/{key1,key2}_id_rsa, listed them ssh-add -l and was asked for passphrases each time. I then verified that known_hosts was ok and that each asymmetric key's public keys were adequately saved in the distant servers.

With all the above, the environment variable SSH_AUTH_SOCK was correctly set again and ssh'ing started working as before.

Per Section 4.1 of the wiki, I also included "AddKeysToAgent yes" to ~/.ssh/config.

$ echo "AddKeysToAgent yes" >> ~/.ssh/config
$ cat ~/.ssh/config
Host HOST1 HOST1.blah.com                                                                                            
     Hostname HOST1.blah.com                                                                                        
     IdentityFile ~/.ssh/key1_id_rsa                                                                             
     User MYUSER                                                                                              
     HostKeyAlgorithms +ssh-rsa                                                                                 
     PubkeyAcceptedKeyTypes +ssh-rsa                                                                            
     
Host HOST2 HOST2.bluh.com                                                                                            
     Hostname HOST2.bluh.com 
     IdentityFile ~/.ssh/key2_id_rsa
     User MYUSER
     HostKeyAlgorithms +ssh-rsa
     PubkeyAcceptedKeyTypes +ssh-rsa

AddKeysToAgent  yes

Rebooting wiped out all that... again, and again the ssh-agent did not start...

So I decided to start a systemd user service: "ssh-agent.service"

$ cat ~/.config/systemd/user/ssh-agent.service 
[Unit]
Description=SSH key agent

[Service]
Type=simple
Environment=SSH_AUTH_SOCK=%t/ssh-agent.socket
ExecStart=/usr/bin/ssh-agent -D -a $SSH_AUTH_SOCK

[Install]
WantedBy=default.target

$ echo "SSH_AUTH_SOCK=\"${XDG_RUNTIME_DIR}/ssh-agent.socket\"" >| ~/.config/environment.d/ssh_auth_socket.conf 
$ systemctl --user enable --now ssh-agent
$ sudo reboot now

After rebooting:

$ systemctl --user show-environment | grep SSH
SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket
$
$ systemctl --user status ssh-agent.service
● ssh-agent.service - SSH key agent
     Loaded: loaded (/home/MYUSER/.config/systemd/user/ssh-agent.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-02-19 20:09:25 CET; 18min ago
   Main PID: 7275 (ssh-agent)
      Tasks: 1 (limit: 18786)
     Memory: 924.0K (peak: 1.1M)
        CPU: 16ms
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/ssh-agent.service
             └─7275 /usr/bin/ssh-agent -D -a /run/user/1000/ssh-agent.socket 

Feb 19 20:09:25 HOST systemd[1874]: Started SSH key agent.
Feb 19 20:09:25 HOST ssh-agent[7275]: SSH_AUTH_SOCK=/run/user/1000/ssh-agent.socket; export SSH_AUTH_SOCK;
Feb 19 20:09:25 HOST ssh-agent[7275]: echo Agent pid 7275;

The agent appears to be running. However the ssh-agent's cache is still empty ( ssh-add -l yields nothing).

Following the Gnome Keyring wiki (section 5) I thought a competing ssh-agent might be interfering, e.g. gcr-ssh-agent but neither gcr-ssh-agent.socket nor gcr-ssh-agent.service seem to be enabled and started.

So at this point, I don't get why ssh-agent is not parsing keys and retaining them in cache across reboots. Any pointer welcome.

Last edited by Cbhihe (2024-02-22 11:24:44)

V1del · 2024-02-19 21:11:01

GNOME keyring integration got dropped for being deprecated https://bbs.archlinux.org/viewtopic.php?id=292953 you should be using the gcr services

iPaq · 2024-02-19 22:20:01

This is an awful way to find out after troubleshooting this for half an hour this morning. Was there an announcement?

V1del · 2024-02-20 10:40:21

Not in our channels, but upstream dropped it from being built by default and the package was not updated to potentially reintroduce it, the wiki was ammended relevantly: https://wiki.archlinux.org/title/GNOME/Keyring#SSH_keys

And if this isn't a functionality the packager uses themselves it might have simply slipped past.

Cbhihe · 2024-02-20 18:19:59

iPaq wrote:

This is an awful way to find out after troubleshooting this for half an hour this morning. Was there an announcement?

Others may have seen something on board, but not me, no heads-up whatsoever.

V1del wrote:

GNOME keyring integration got dropped for being deprecated https://bbs.archlinux.org/viewtopic.php?id=292953 you should be using the gcr services

Thanks @V1del for the confirmation.
To activate the Gnome challenge-response agent "gcr-ssh-agent, do I need to enable both units gcr-ssh-agent.socket and gcr-ssh-agent.service in Systemd ? I saw the ammended Wiki on the Gnome keyring as referenced in your reply. But is there a reference for that somewhere so I can read more and learn some?

Last edited by Cbhihe (2024-02-21 07:42:17)

Cbhihe · 2024-02-20 18:48:06

In the end, after not a few crazy hours yesterday, spent NOT figuring out this thing, finally saved by the link referenced by V1del above, the culprit is my 2024.02.18 update of gnome-keyring (1:42.1-3 -> 1:46.1-1). Also see this.

I reversed my setup of the user service ~/.config/systemd/user/ssh-agent.service like so:

$ systemctl --user disable  ssh-agent.service

Finally I enabled and activated the socket that correspond to gcr-ssh-agent, and re-define the $SSH_AUTH_SOCK environment variable:

$ systemctl --user enable --now gcr-ssh-agent.socket
$ echo "SSH_AUTH_SOCK=\${XDG_RUNTIME_DIR}/gcr/ssh" >| ~/.config/environment.d/ssh_auth_socket.conf

But I am not quite sure harcoding SSH_AUTH_SOCK=$XDG_RUNTIME_DIR/gcr/ssh in that way is good practice.
I also did not want to add that env-var permanently to one of my shell dotfiles and some logic to skip that when it's either already set for another purpose.

In particular what happens when that path does not exist, i.e. in absence of a graphical session ? (e.g. SSH'ing to hosts without a GUI).

EDIT:
Escape the "$" sign as shown above, when echoing the "SSH_AUTH_SOCK=${XDG_RUNTIME_DIR}/..." env-var definition into ~/.config/environment.d/ssh_auth_socket.conf. This is necessary to prevent your shell from interpreting the string variable "${XDG_RUNTIME_DIR}" and from pushing the already interpreted value of the string in the ssh_auth_socket.conf file.

In addition to the above code block, execute the commands below, then reboot.

$ systemctl --user enable gnome-keyring-daemon.{service,socket}
$ systemctl --user start gnome-keyring-daemon.{service,socket}

[SOLVED]

Last edited by Cbhihe (2024-02-23 10:59:29)

iPaq · 2024-03-01 21:28:28

It seems the only way to use this new thing is hardcoding variables in a few places. Not too happy about that.

~/.xsessionrc hasn't worked. Even your environment.d file there hasn't worked. Graphical packages which expect to make SSH connections became dead out of the water since this change (For me `virt-manager` and `nemo`) unless launched from a terminal, which is significantly slower and frustrating.

Last edited by iPaq (2024-03-01 21:34:13)

Arch Linux

#1 2024-02-19 20:45:53

[SOLVED] ssh-agent started as user service unable to read in ssh keys

#2 2024-02-19 21:11:01

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

#3 2024-02-19 22:20:01

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

#4 2024-02-20 10:40:21

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

#5 2024-02-20 18:19:59

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

#6 2024-02-20 18:48:06

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

#7 2024-03-01 21:28:28

Re: [SOLVED] ssh-agent started as user service unable to read in ssh keys

Board footer