You are not logged in.

#1 2022-07-05 03:26:25

lhj
Member
Registered: 2013-07-10
Posts: 4

Extremely slow SSH connection

Hello, I apologize if this is a known issue but I couldn't find a solution searching by myself yet.

SSHing from my Arch machine to the raspberry PI (same network, Arch box connected via ethernet cable, PI connected via Wifi) gives me an extremely slow connection.
I experience high latency when typing, and connecting to the PI itself takes a very long time before I get the shell. This happens both with key based authentication and password based authentication.

This is the output of

 ssh -vvv -2 pi@192.168.11.76 
OpenSSH_9.0p1, OpenSSL 1.1.1p  21 Jun 2022
debug1: Reading configuration data /home/ale/.ssh/config
debug1: /home/ale/.ssh/config line 11: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 192.168.11.76 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/ale/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/ale/.ssh/known_hosts2'
debug1: auto-mux: Trying existing master
debug1: Control socket "/home/ale/.ssh/master-pi@192.168.11.76:22" does not exist
debug3: ssh_connect_direct: entering
debug1: Connecting to 192.168.11.76 [192.168.11.76] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: Connection established.
debug1: identity file /home/ale/.ssh/id_rsa type 0
debug1: identity file /home/ale/.ssh/id_rsa-cert type -1
debug1: identity file /home/ale/.ssh/id_ecdsa type -1
debug1: identity file /home/ale/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/ale/.ssh/id_ecdsa_sk type -1
debug1: identity file /home/ale/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /home/ale/.ssh/id_ed25519 type -1
debug1: identity file /home/ale/.ssh/id_ed25519-cert type -1
debug1: identity file /home/ale/.ssh/id_ed25519_sk type -1
debug1: identity file /home/ale/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /home/ale/.ssh/id_xmss type -1
debug1: identity file /home/ale/.ssh/id_xmss-cert type -1
debug1: identity file /home/ale/.ssh/id_dsa type -1
debug1: identity file /home/ale/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5
debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.11.76:22 as 'pi'
debug3: record_hostkey: found key type ED25519 in file /home/ale/.ssh/known_hosts:1
debug3: record_hostkey: found key type ECDSA in file /home/ale/.ssh/known_hosts:2
debug3: load_hostkeys_file: loaded 2 keys from 192.168.11.76
debug1: load_hostkeys: fopen /home/ale/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: have matching best-preference key type ssh-ed25519-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:+jIxHeD7BnOMjM+qZxeLmUyRFLmLmKXAvrYtN7yB/Ao
debug3: record_hostkey: found key type ED25519 in file /home/ale/.ssh/known_hosts:1
debug3: record_hostkey: found key type ECDSA in file /home/ale/.ssh/known_hosts:2
debug3: load_hostkeys_file: loaded 2 keys from 192.168.11.76
debug1: load_hostkeys: fopen /home/ale/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.11.76' is known and matches the ED25519 host key.
debug1: Found key in /home/ale/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: /home/ale/.ssh/id_rsa RSA SHA256:F48ZEMveeImv5RBF3t7UejekNFVnhAZmtdJtk/NqjAg
debug1: Will attempt key: /home/ale/.ssh/id_ecdsa 
debug1: Will attempt key: /home/ale/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /home/ale/.ssh/id_ed25519 
debug1: Will attempt key: /home/ale/.ssh/id_ed25519_sk 
debug1: Will attempt key: /home/ale/.ssh/id_xmss 
debug1: Will attempt key: /home/ale/.ssh/id_dsa 
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/ale/.ssh/id_rsa RSA SHA256:F48ZEMveeImv5RBF3t7UejekNFVnhAZmtdJtk/NqjAg
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /home/ale/.ssh/id_rsa RSA SHA256:F48ZEMveeImv5RBF3t7UejekNFVnhAZmtdJtk/NqjAg
debug3: sign_and_send_pubkey: using publickey with RSA SHA256:F48ZEMveeImv5RBF3t7UejekNFVnhAZmtdJtk/NqjAg
debug3: sign_and_send_pubkey: signing using rsa-sha2-512 SHA256:F48ZEMveeImv5RBF3t7UejekNFVnhAZmtdJtk/NqjAg
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to 192.168.11.76 ([192.168.11.76]:22) using "publickey".
debug1: setting up multiplex master socket
debug3: muxserver_listen: temporary control path /home/ale/.ssh/master-pi@192.168.11.76:22.GD61YMpDzJEeh6Wu
debug2: fd 4 setting O_NONBLOCK
debug3: fd 4 is O_NONBLOCK
debug3: fd 4 is O_NONBLOCK
debug1: channel 0: new [/home/ale/.ssh/master-pi@192.168.11.76:22]
debug3: muxserver_listen: mux listener channel 0 fd 4
debug1: channel 1: new [client-session]
debug3: ssh_session2_open: channel_new: 1
debug2: channel 1: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: id
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: client_input_hostkeys: received RSA key SHA256:5+DWpm5Mo+j57QBAPumniZTqFebCDSMBTEpZ2VuJsio
debug3: client_input_hostkeys: received ECDSA key SHA256:IwWikf0VJSpKkJKNk2TSVSttuMRLfbEJCWbuNe9CpH0
debug3: client_input_hostkeys: received ED25519 key SHA256:+jIxHeD7BnOMjM+qZxeLmUyRFLmLmKXAvrYtN7yB/Ao
debug1: client_input_hostkeys: searching /home/ale/.ssh/known_hosts for 192.168.11.76 / (none)
debug3: hostkeys_foreach: reading file "/home/ale/.ssh/known_hosts"
debug3: hostkeys_find: found ssh-ed25519 key at /home/ale/.ssh/known_hosts:1
debug3: hostkeys_find: found ecdsa-sha2-nistp256 key at /home/ale/.ssh/known_hosts:2
debug1: client_input_hostkeys: searching /home/ale/.ssh/known_hosts2 for 192.168.11.76 / (none)
debug1: client_input_hostkeys: hostkeys file /home/ale/.ssh/known_hosts2 does not exist
debug3: client_input_hostkeys: 3 server keys: 1 new, 1 retained, 1 incomplete match. 0 to remove
debug3: client_input_hostkeys: asking server to prove ownership for 1 keys
debug3: send packet: type 80
debug3: receive packet: type 4
debug1: Remote: /home/pi/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 4
debug1: Remote: /home/pi/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 1: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug2: client_session2_setup: id 1
debug2: channel 1: request pty-req confirm 1
debug3: send packet: type 98
debug2: channel 1: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 1: callback done
debug2: channel 1: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 81
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug3: hostkeys_foreach: reading file "/home/ale/.ssh/known_hosts"
debug3: host_delete: ED25519 key already at /home/ale/.ssh/known_hosts:1
debug3: host_delete: ECDSA key already at /home/ale/.ssh/known_hosts:2
debug1: update_known_hosts: known hosts file /home/ale/.ssh/known_hosts2 does not exist
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 1
debug2: PTY allocation request accepted on channel 1
debug2: channel 1: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 1
debug2: shell request accepted on channel 1
Linux raspberrypi 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Tue Jul  5 04:15:26 2022 from 192.168.11.152

SSH is enabled and the default password for the 'pi' user has not been changed.
This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.

I'm not sure if this helps, but
- If I connect to the PI from my macbook (openssh) the connection is very fast
- If I connect to the PI using plink from my Arch box, connection is very fast

The issue appears only when I use the standard openssh client from my Arch box.

This is my ~/.ssh/config

 Host pcBitbucket bitbucket.org                                                                                                                                                                                                                                                                                                
     HostName bitbucket.org                                                                                                                                                                                                                                                                                                    
     IdentityFile ~/.ssh/<edited>                                                                                                                                                                                                                                                                            
     User <edited>                                                                                                                                                                                                                                                                                                             
                                                                                                                                                                                                                                                     
 Host *                                                                                                                                                                                                                                                                                                                        
 ControlMaster auto                                                                                                                                                                                                                                                                                                            
 ControlPath ~/.ssh/master-%r@%h:%p    

Last edited by lhj (2022-07-05 03:27:11)

Offline

#2 2022-07-05 07:04:31

seth
Member
Registered: 2012-09-03
Posts: 30,912

Re: Extremely slow SSH connection

https://archlinux.org/packages/community/x86_64/mosh/

debug1: Remote: /home/pi/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding

I'd start by removing that.

Is the connection sharing already a mitigational effort (does it happend w/o)?

Offline

#3 2022-07-05 07:39:32

lhj
Member
Registered: 2013-07-10
Posts: 4

Re: Extremely slow SSH connection

Connection is slow with or without connection sharing (I tried setting

ControlMaster no

in my ssh config.

Could you point me in the right direction about modifying key options?
I disabled x11 forwarding for the given key, the output is now:

OpenSSH_9.0p1, OpenSSL 1.1.1p  21 Jun 2022
debug1: Reading configuration data /home/ale/.ssh/config
debug1: /home/ale/.ssh/config line 11: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 192.168.11.76 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/home/ale/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/home/ale/.ssh/known_hosts2'
debug3: ssh_connect_direct: entering
debug1: Connecting to 192.168.11.76 [192.168.11.76] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: Connection established.
debug1: identity file /home/ale/.ssh/id_rsa_raspi type 0
debug1: identity file /home/ale/.ssh/id_rsa_raspi-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.0
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5
debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.11.76:22 as 'pi'
debug3: record_hostkey: found key type ED25519 in file /home/ale/.ssh/known_hosts:1
debug3: record_hostkey: found key type ECDSA in file /home/ale/.ssh/known_hosts:2
debug3: load_hostkeys_file: loaded 2 keys from 192.168.11.76
debug1: load_hostkeys: fopen /home/ale/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: have matching best-preference key type ssh-ed25519-cert-v01@openssh.com, using HostkeyAlgorithms verbatim
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:+jIxHeD7BnOMjM+qZxeLmUyRFLmLmKXAvrYtN7yB/Ao
debug3: record_hostkey: found key type ED25519 in file /home/ale/.ssh/known_hosts:1
debug3: record_hostkey: found key type ECDSA in file /home/ale/.ssh/known_hosts:2
debug3: load_hostkeys_file: loaded 2 keys from 192.168.11.76
debug1: load_hostkeys: fopen /home/ale/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.11.76' is known and matches the ED25519 host key.
debug1: Found key in /home/ale/.ssh/known_hosts:1
debug3: send packet: type 21
debug2: ssh_set_newkeys: mode 1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: ssh_set_newkeys: mode 0
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: /home/ale/.ssh/id_rsa_raspi RSA SHA256:t/cscBtIBt9+hxKbGWamZsxXzQ/G1hZSrXq495JfPUM explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/ale/.ssh/id_rsa_raspi RSA SHA256:t/cscBtIBt9+hxKbGWamZsxXzQ/G1hZSrXq495JfPUM explicit
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: /home/ale/.ssh/id_rsa_raspi RSA SHA256:t/cscBtIBt9+hxKbGWamZsxXzQ/G1hZSrXq495JfPUM explicit
debug3: sign_and_send_pubkey: using publickey with RSA SHA256:t/cscBtIBt9+hxKbGWamZsxXzQ/G1hZSrXq495JfPUM
debug3: sign_and_send_pubkey: signing using rsa-sha2-512 SHA256:t/cscBtIBt9+hxKbGWamZsxXzQ/G1hZSrXq495JfPUM
debug3: send packet: type 50
debug3: receive packet: type 52
Authenticated to 192.168.11.76 ([192.168.11.76]:22) using "publickey".
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: filesystem
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: client_input_hostkeys: received RSA key SHA256:5+DWpm5Mo+j57QBAPumniZTqFebCDSMBTEpZ2VuJsio
debug3: client_input_hostkeys: received ECDSA key SHA256:IwWikf0VJSpKkJKNk2TSVSttuMRLfbEJCWbuNe9CpH0
debug3: client_input_hostkeys: received ED25519 key SHA256:+jIxHeD7BnOMjM+qZxeLmUyRFLmLmKXAvrYtN7yB/Ao
debug1: client_input_hostkeys: searching /home/ale/.ssh/known_hosts for 192.168.11.76 / (none)
debug3: hostkeys_foreach: reading file "/home/ale/.ssh/known_hosts"
debug3: hostkeys_find: found ssh-ed25519 key at /home/ale/.ssh/known_hosts:1
debug3: hostkeys_find: found ecdsa-sha2-nistp256 key at /home/ale/.ssh/known_hosts:2
debug1: client_input_hostkeys: searching /home/ale/.ssh/known_hosts2 for 192.168.11.76 / (none)
debug1: client_input_hostkeys: hostkeys file /home/ale/.ssh/known_hosts2 does not exist
debug3: client_input_hostkeys: 3 server keys: 1 new, 1 retained, 1 incomplete match. 0 to remove
debug3: client_input_hostkeys: asking server to prove ownership for 1 keys
debug3: send packet: type 80
debug3: receive packet: type 4
debug1: Remote: /home/pi/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc
debug3: receive packet: type 4
debug1: Remote: /home/pi/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 81
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug3: hostkeys_foreach: reading file "/home/ale/.ssh/known_hosts"
debug3: host_delete: ED25519 key already at /home/ale/.ssh/known_hosts:1
debug3: host_delete: ECDSA key already at /home/ale/.ssh/known_hosts:2
debug1: update_known_hosts: known hosts file /home/ale/.ssh/known_hosts2 does not exist
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 2097152
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Linux raspberrypi 5.15.32-v8+ #1538 SMP PREEMPT Thu Mar 31 19:40:39 BST 2022 aarch64

The connection is still horribly slow with typing lag.

Re. mosh, I know it's a workaround, but the issue happens with scp/rsync as well, which is what I care the most (file transfer is often very slow, and it takes long to initiate the connection)

I did another test:
- I connected to the PI via ssh from a ubuntu docker container on my Arch box. No issues there, the connection is very fast (as it should be as both machines are on the same lan).
- I connected to the PI via ssh from a archlinux docker container on my Arch box. The connection is very slow, as it is when trying to connect from my Arch box.

It appears there's something wrong in Arch itself (openssh package? some default config? no clue).
It seems the issue is with the default IPQoS settings - there are some notes in the bug tracker https://bugs.archlinux.org/task/67742
Setting

IPQoS lowdelay

in ~/.ssh/config solves the issue

Last edited by lhj (2022-07-05 08:36:18)

Offline

#4 2022-07-05 08:16:03

seth
Member
Registered: 2012-09-03
Posts: 30,912

Re: Extremely slow SSH connection

If you compare a login from your macbook, do these lines differ

debug2: KEX algorithms: sntrup761x25519-sha512@openssh.com,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,sk-ssh-ed25519-cert-v01@openssh.com,sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib

Is the raspi under a lot of CPU pressure during the ssh session?
Have you tried a password login (rather than public key auth)?

Offline

#5 2022-07-05 08:39:08

lhj
Member
Registered: 2013-07-10
Posts: 4

Re: Extremely slow SSH connection

seth wrote:

Is the raspi under a lot of CPU pressure during the ssh session?
Have you tried a password login (rather than public key auth)?

The PI is idle - there's no issue connecting from either ubuntu, or macos, or with plink (putty) from Arch itself.
Also, the issue arises with both password and key based authentication.
Thanks for helping anyway :-)

I'll quote my previous edit to make it easily visible in case someone incurs in the same issue:

lhj wrote:

It seems the issue is with the default IPQoS settings - there are some notes in the bug tracker https://bugs.archlinux.org/task/67742
Setting

IPQoS lowdelay

in ~/.ssh/config solves the issue

Last edited by lhj (2022-07-05 08:43:26)

Offline

#6 2022-07-05 14:35:06

seth
Member
Registered: 2012-09-03
Posts: 30,912

Re: Extremely slow SSH connection

Oh, cool - that's still a thing…

Please always remember to mark resolved threads by editing your initial posts subject - so others will know that there's no task left, but maybe a solution to find.
Thanks.

Offline

Board footer

Powered by FluxBB