You are not logged in.

#1 2014-08-29 16:27:56

bdjnk
Member
Registered: 2009-08-24
Posts: 78

SSH Hangs on WAN, Fine on LAN.

As the title says, ssh connections on the local network are fine. Attempted remote connections hang. I've verified that connection can be made from the remote clients to other ssh servers.

Here's some of (all?) the relevant logs:

When connecting locally, this is the output of sshd -ddd:

debug2: load_server_config: filename /etc/ssh/sshd_config
debug2: load_server_config: done config len = 314
debug2: parse_server_config: config /etc/ssh/sshd_config len 314
debug3: /etc/ssh/sshd_config:41 setting PermitRootLogin no
debug3: /etc/ssh/sshd_config:51 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: /etc/ssh/sshd_config:73 setting ChallengeResponseAuthentication no
debug3: /etc/ssh/sshd_config:94 setting UsePAM yes
debug3: /etc/ssh/sshd_config:102 setting PrintMotd no 
debug3: /etc/ssh/sshd_config:106 setting UsePrivilegeSeparation sandbox		
debug3: /etc/ssh/sshd_config:122 setting Subsystem sftp	/usr/lib/ssh/sftp-server
debug1: sshd version OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_rsa_key" as a RSA1 public key
debug1: private host key: #0 type 1 RSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type DSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_dsa_key" as a RSA1 public key
debug1: private host key: #1 type 2 DSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type ECDSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ecdsa_key" as a RSA1 public key
debug1: private host key: #2 type 3 ECDSA
debug3: Incorrect RSA1 identifier
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ed25519_key" as a RSA1 public key
debug1: private host key: #3 type 4 ED25519
debug1: rexec_argv[0]='/bin/sshd'
debug1: rexec_argv[1]='-ddd'
debug1: rexec_argv[2]='-E'
debug1: rexec_argv[3]='/home/ben/sshCrap'
debug3: oom_adjust_setup
Set /proc/self/oom_score_adj from 0 to -1000
debug2: fd 4 setting O_NONBLOCK
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
debug2: fd 5 setting O_NONBLOCK
debug3: sock_set_v6only: set socket 5 IPV6_V6ONLY
debug1: Bind to port 22 on ::.
Server listening on :: port 22.
debug3: fd 6 is not O_NONBLOCK
debug1: Server will not fork when running in debugging mode.
debug3: send_rexec_state: entering fd = 9 config len 314
debug3: ssh_msg_send: type 0
debug3: send_rexec_state: done
debug1: rexec start in 6 out 6 newsock 6 pipe -1 sock 9
debug3: recv_rexec_state: entering fd = 5
debug3: ssh_msg_recv entering
debug3: recv_rexec_state: done
debug2: parse_server_config: config rexec len 314
debug3: rexec:41 setting PermitRootLogin no
debug3: rexec:51 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: rexec:73 setting ChallengeResponseAuthentication no
debug3: rexec:94 setting UsePAM yes
debug3: rexec:102 setting PrintMotd no 
debug3: rexec:106 setting UsePrivilegeSeparation sandbox		
debug3: rexec:122 setting Subsystem sftp	/usr/lib/ssh/sftp-server
debug1: sshd version OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_rsa_key" as a RSA1 public key
debug1: private host key: #0 type 1 RSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type DSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_dsa_key" as a RSA1 public key
debug1: private host key: #1 type 2 DSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type ECDSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ecdsa_key" as a RSA1 public key
debug1: private host key: #2 type 3 ECDSA
debug3: Incorrect RSA1 identifier
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ed25519_key" as a RSA1 public key
debug1: private host key: #3 type 4 ED25519
debug1: inetd sockets after dupping: 5, 5
Connection from 192.168.0.2 port 39239 on 192.168.0.27 port 22
debug1: Client protocol version 2.0; client software version OpenSSH_6.6
debug1: match: OpenSSH_6.6 pat OpenSSH_6.5*,OpenSSH_6.6* compat 0x14000000
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug2: fd 5 setting O_NONBLOCK
debug3: ssh_sandbox_init: preparing seccomp filter sandbox
debug2: Network child is on pid 1759
debug3: preauth child monitor started
debug3: privsep user:group 99:99 [preauth]
debug1: permanently_set_uid: 99/99 [preauth]
debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth]
debug3: ssh_sandbox_child: attaching seccomp filter program [preauth]
debug2: compat_kex_proposal: original KEX proposal: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug2: Compat: skipping algorithm "curve25519-sha256@libssh.org" [preauth]
debug2: compat_kex_proposal: compat KEX proposal: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug1: list_hostkey_types: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug2: kex_parse_kexinit: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit: first_kex_follows 0  [preauth]
debug2: kex_parse_kexinit: reserved 0  [preauth]
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug2: kex_parse_kexinit: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-ed25519,ssh-rsa,ssh-dss [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit: first_kex_follows 0  [preauth]
debug2: kex_parse_kexinit: reserved 0  [preauth]
debug2: mac_setup: setup hmac-md5-etm@openssh.com [preauth]
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none [preauth]
debug2: mac_setup: setup hmac-md5-etm@openssh.com [preauth]
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none [preauth]
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
debug3: mm_key_sign entering [preauth]
debug3: mm_request_send entering: type 6 [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 6
debug3: mm_answer_sign
debug3: mm_answer_sign: signature 0x7f841e2f56f0(100)
debug3: mm_request_send entering: type 7
debug2: monitor_read: 6 used once, disabling now
debug3: mm_key_sign: waiting for MONITOR_ANS_SIGN [preauth]
debug3: mm_request_receive_expect entering: type 7 [preauth]
debug3: mm_request_receive entering [preauth]
debug2: kex_derive_keys [preauth]
debug2: set_newkeys: mode 1 [preauth]
debug1: SSH2_MSG_NEWKEYS sent [preauth]
debug1: expecting SSH2_MSG_NEWKEYS [preauth]
debug2: set_newkeys: mode 0 [preauth]
debug1: SSH2_MSG_NEWKEYS received [preauth]
debug1: KEX done [preauth]
debug1: userauth-request for user ben service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug3: mm_getpwnamallow entering [preauth]
debug3: mm_request_send entering: type 8 [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 8
debug3: mm_answer_pwnamallow
debug3: Trying to reverse map address 192.168.0.2.
debug2: parse_server_config: config reprocess config len 314
debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1
debug3: mm_request_send entering: type 9
debug2: monitor_read: 8 used once, disabling now
debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth]
debug3: mm_request_receive_expect entering: type 9 [preauth]
debug3: mm_request_receive entering [preauth]
debug2: input_userauth_request: setting up authctxt for ben [preauth]
debug3: mm_start_pam entering [preauth]
debug3: mm_request_send entering: type 100 [preauth]
debug3: mm_inform_authserv entering [preauth]
debug3: mm_request_send entering: type 4 [preauth]
debug2: input_userauth_request: try method none [preauth]
debug3: userauth_finish: failure partial=0 next methods="publickey,password" [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 100
debug1: PAM: initializing for "ben"
debug1: PAM: setting PAM_RHOST to "192.168.0.2"
debug1: PAM: setting PAM_TTY to "ssh"
debug2: monitor_read: 100 used once, disabling now
debug3: mm_request_receive entering
debug3: monitor_read: checking request 4
debug3: mm_answer_authserv: service=ssh-connection, style=
debug2: monitor_read: 4 used once, disabling now
debug1: userauth-request for user ben service ssh-connection method password [preauth]
debug1: attempt 1 failures 0 [preauth]
debug2: input_userauth_request: try method password [preauth]
debug3: mm_auth_password entering [preauth]
debug3: mm_request_send entering: type 12 [preauth]
debug3: mm_auth_password: waiting for MONITOR_ANS_AUTHPASSWORD [preauth]
debug3: mm_request_receive_expect entering: type 13 [preauth]
debug3: mm_request_receive entering [preauth]
debug3: mm_request_receive entering
debug3: monitor_read: checking request 12
debug3: PAM: sshpam_passwd_conv called with 1 messages
debug1: PAM: password authentication accepted for ben
debug3: mm_answer_authpassword: sending result 1
debug3: mm_request_send entering: type 13
debug3: mm_request_receive_expect entering: type 102
debug3: mm_request_receive entering
debug1: do_pam_account: called
debug3: PAM: do_pam_account pam_acct_mgmt = 0 (Success)
debug3: mm_request_send entering: type 103
Accepted password for ben from 192.168.0.2 port 39239 ssh2
debug1: monitor_child_preauth: ben has been authenticated by privileged process
debug3: mm_get_keystate: Waiting for new keys
debug3: mm_request_receive_expect entering: type 26
debug3: mm_request_receive entering
debug3: mm_newkeys_from_blob: 0x7f841e30dd90(138)
debug2: mac_setup: setup hmac-md5-etm@openssh.com
debug3: mm_get_keystate: Waiting for second key
debug3: mm_newkeys_from_blob: 0x7f841e30dd90(138)
debug2: mac_setup: setup hmac-md5-etm@openssh.com
debug3: mm_get_keystate: Getting compression state
debug3: mm_get_keystate: Getting Network I/O buffers
debug3: mm_auth_password: user authenticated [preauth]
debug3: mm_do_pam_account entering [preauth]
debug3: mm_request_send entering: type 102 [preauth]
debug3: mm_request_receive_expect entering: type 103 [preauth]
debug3: mm_request_receive entering [preauth]
debug3: mm_do_pam_account returning 1 [preauth]
debug3: mm_send_keystate: Sending new keys: 0x7f841e2f66a0 0x7f841e2f89b0 [preauth]
debug3: mm_newkeys_to_blob: converting 0x7f841e2f66a0 [preauth]
debug3: mm_newkeys_to_blob: converting 0x7f841e2f89b0 [preauth]
debug3: mm_send_keystate: New keys have been sent [preauth]
debug3: mm_send_keystate: Sending compression state [preauth]
debug3: mm_request_send entering: type 26 [preauth]
debug3: mm_send_keystate: Finished sending state [preauth]
debug1: monitor_read_log: child log fd closed
debug3: mm_share_sync: Share sync
debug3: mm_share_sync: Share sync end
debug3: ssh_sandbox_parent_finish: finished
debug1: PAM: establishing credentials
debug3: PAM: opening session
User child is on pid 1761
debug1: PAM: establishing credentials
debug1: permanently_set_uid: 1001/100
debug2: set_newkeys: mode 0
debug2: set_newkeys: mode 1
debug1: Entering interactive session for SSH2.
debug2: fd 10 setting O_NONBLOCK
debug2: fd 11 setting O_NONBLOCK
debug1: server_init_dispatch_20
debug1: server_input_channel_open: ctype session rchan 0 win 1048576 max 16384
debug1: input_session_request
debug1: channel 0: new [server-session]
debug2: session_new: allocate (allocated 0 max 10)
debug3: session_unused: session id 0 unused
debug1: session_new: session 0
debug1: session_open: channel 0
debug1: session_open: session 0: link with channel 0
debug1: server_input_channel_open: confirm session
debug1: server_input_global_request: rtype no-more-sessions@openssh.com want_reply 0
debug1: server_input_channel_req: channel 0 request x11-req reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req x11-req
debug1: X11 forwarding disabled in server configuration file.
debug1: server_input_channel_req: channel 0 request pty-req reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req pty-req
debug1: Allocating pty.
debug3: mm_request_send entering: type 28
debug3: mm_pty_allocate: waiting for MONITOR_ANS_PTY
debug3: mm_request_receive_expect entering: type 29
debug3: mm_request_receive entering
debug3: mm_request_receive entering
debug3: monitor_read: checking request 28
debug3: mm_answer_pty entering
debug2: session_new: allocate (allocated 0 max 10)
debug3: session_unused: session id 0 unused
debug1: session_new: session 0
debug3: mm_request_send entering: type 29
debug3: mm_answer_pty: tty /dev/pts/0 ptyfd 8
debug1: session_pty_req: session 0 alloc /dev/pts/0
debug1: server_input_channel_req: channel 0 request shell reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req shell
Starting session: shell on pts/0 for ben from 192.168.0.2 port 39239
debug2: fd 5 setting TCP_NODELAY
debug3: packet_set_tos: set IP_TOS 0x10
debug2: channel 0: rfd 14 isatty
debug2: fd 14 setting O_NONBLOCK
debug3: fd 12 is O_NONBLOCK
debug1: Setting controlling tty using TIOCSCTTY.
debug3: Copy environment: MAIL=/var/spool/mail/ben
debug3: Copy environment: XDG_SESSION_ID=c3
debug3: Copy environment: XDG_RUNTIME_DIR=/run/user/1001
debug3: Copy environment: XDG_SEAT=seat0
debug3: Copy environment: XDG_VTNR=2
debug1: Received SIGCHLD.
debug1: session_by_pid: pid 1762
debug1: session_exit_message: session 0 channel 0 pid 1762
debug2: channel 0: request exit-status confirm 0
debug1: session_exit_message: release channel 0
debug2: channel 0: write failed
debug2: channel 0: close_write
debug2: channel 0: send eow
debug2: channel 0: output open -> closed
debug3: mm_request_send entering: type 30
debug2: notify_done: reading
debug2: channel 0: read<=0 rfd 14 len -1
debug2: channel 0: read failed
debug2: channel 0: close_read
debug2: channel 0: input open -> drain
debug2: channel 0: ibuf empty
debug2: channel 0: send eof
debug2: channel 0: input drain -> closed
debug2: channel 0: send close
debug3: channel 0: will not send data after close
debug3: mm_request_receive entering
debug3: monitor_read: checking request 30
debug3: mm_answer_pty_cleanup entering
debug1: session_by_tty: session 0 tty /dev/pts/0
debug3: mm_session_close: session 0 pid 1761
debug3: mm_session_close: tty /dev/pts/0 ptyfd 8
debug1: session_pty_cleanup: session 0 release /dev/pts/0
debug3: session_unused: session id 0 unused
debug2: channel 0: rcvd close
debug3: channel 0: will not send data after close
debug2: channel 0: is dead
debug2: channel 0: gc: notify user
debug1: session_by_channel: session 0 channel 0
debug1: session_close_by_channel: channel 0 child 0
debug1: session_close: session 0 pid 0
debug3: session_unused: session id 0 unused
debug2: channel 0: gc: user detached
debug2: channel 0: is dead
debug2: channel 0: garbage collecting
debug1: channel 0: free: server-session, nchannels 1
debug3: channel 0: status: The following connections are open:
  #0 server-session (t4 r0 i3/0 o3/0 fd -1/-1 cc -1)

Received disconnect from 192.168.0.2: 11: disconnected by user
debug1: do_cleanup
debug3: PAM: sshpam_thread_cleanup entering
debug3: mm_request_receive entering
debug1: do_cleanup
debug1: PAM: cleanup
debug1: PAM: closing session
debug1: PAM: deleting credentials
debug3: PAM: sshpam_thread_cleanup entering

And this is the output of sshd -ddd when connecting remotely:

debug2: load_server_config: filename /etc/ssh/sshd_config
debug2: load_server_config: done config len = 314
debug2: parse_server_config: config /etc/ssh/sshd_config len 314
debug3: /etc/ssh/sshd_config:41 setting PermitRootLogin no
debug3: /etc/ssh/sshd_config:51 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: /etc/ssh/sshd_config:73 setting ChallengeResponseAuthentication no
debug3: /etc/ssh/sshd_config:94 setting UsePAM yes
debug3: /etc/ssh/sshd_config:102 setting PrintMotd no 
debug3: /etc/ssh/sshd_config:106 setting UsePrivilegeSeparation sandbox		
debug3: /etc/ssh/sshd_config:122 setting Subsystem sftp	/usr/lib/ssh/sftp-server
debug1: sshd version OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_rsa_key" as a RSA1 public key
debug1: private host key: #0 type 1 RSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type DSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_dsa_key" as a RSA1 public key
debug1: private host key: #1 type 2 DSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type ECDSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ecdsa_key" as a RSA1 public key
debug1: private host key: #2 type 3 ECDSA
debug3: Incorrect RSA1 identifier
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ed25519_key" as a RSA1 public key
debug1: private host key: #3 type 4 ED25519
debug1: rexec_argv[0]='/bin/sshd'
debug1: rexec_argv[1]='-ddd'
debug1: rexec_argv[2]='-E'
debug1: rexec_argv[3]='/home/ben/sshCrap2'
debug3: oom_adjust_setup
Set /proc/self/oom_score_adj from 0 to -1000
debug2: fd 4 setting O_NONBLOCK
debug1: Bind to port 22 on 0.0.0.0.
Server listening on 0.0.0.0 port 22.
debug2: fd 5 setting O_NONBLOCK
debug3: sock_set_v6only: set socket 5 IPV6_V6ONLY
debug1: Bind to port 22 on ::.
Server listening on :: port 22.
debug3: fd 6 is not O_NONBLOCK
debug1: Server will not fork when running in debugging mode.
debug3: send_rexec_state: entering fd = 9 config len 314
debug3: ssh_msg_send: type 0
debug3: send_rexec_state: done
debug1: rexec start in 6 out 6 newsock 6 pipe -1 sock 9
debug3: recv_rexec_state: entering fd = 5
debug3: ssh_msg_recv entering
debug3: recv_rexec_state: done
debug2: parse_server_config: config rexec len 314
debug3: rexec:41 setting PermitRootLogin no
debug3: rexec:51 setting AuthorizedKeysFile .ssh/authorized_keys
debug3: rexec:73 setting ChallengeResponseAuthentication no
debug3: rexec:94 setting UsePAM yes
debug3: rexec:102 setting PrintMotd no 
debug3: rexec:106 setting UsePrivilegeSeparation sandbox		
debug3: rexec:122 setting Subsystem sftp	/usr/lib/ssh/sftp-server
debug1: sshd version OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type RSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_rsa_key" as a RSA1 public key
debug1: private host key: #0 type 1 RSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type DSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_dsa_key" as a RSA1 public key
debug1: private host key: #1 type 2 DSA
debug3: Incorrect RSA1 identifier
debug1: key_parse_private2: missing begin marker
debug1: read PEM private key done: type ECDSA
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ecdsa_key" as a RSA1 public key
debug1: private host key: #2 type 3 ECDSA
debug3: Incorrect RSA1 identifier
debug3: Incorrect RSA1 identifier
debug3: Could not load "/etc/ssh/ssh_host_ed25519_key" as a RSA1 public key
debug1: private host key: #3 type 4 ED25519
debug1: inetd sockets after dupping: 5, 5
Connection from ExternalIP port 43204 on InternalIP port 22
debug1: Client protocol version 2.0; client software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug2: fd 5 setting O_NONBLOCK
debug3: ssh_sandbox_init: preparing seccomp filter sandbox
debug2: Network child is on pid 1825
debug3: preauth child monitor started
debug3: privsep user:group 99:99 [preauth]
debug1: permanently_set_uid: 99/99 [preauth]
debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth]
debug3: ssh_sandbox_child: attaching seccomp filter program [preauth]
debug1: list_hostkey_types: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit: first_kex_follows 0  [preauth]
debug2: kex_parse_kexinit: reserved 0  [preauth]
debug2: kex_parse_kexinit: curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 [preauth]
debug2: kex_parse_kexinit: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-rsa,ssh-dss [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm@openssh.com,aes256-gcm@openssh.com,chacha20-poly1305@openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: hmac-md5-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-ripemd160-etm@openssh.com,hmac-sha1-96-etm@openssh.com,hmac-md5-96-etm@openssh.com,hmac-md5,hmac-sha1,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib [preauth]
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit:  [preauth]
debug2: kex_parse_kexinit: first_kex_follows 0  [preauth]
debug2: kex_parse_kexinit: reserved 0  [preauth]
debug2: mac_setup: setup hmac-md5-etm@openssh.com [preauth]
debug1: kex: client->server aes128-ctr hmac-md5-etm@openssh.com none [preauth]
debug2: mac_setup: setup hmac-md5-etm@openssh.com [preauth]
debug1: kex: server->client aes128-ctr hmac-md5-etm@openssh.com none [preauth]
debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth]
Connection closed by ExternalIP [preauth]
debug1: do_cleanup [preauth]
debug3: PAM: sshpam_thread_cleanup entering [preauth]
debug3: mm_request_receive entering
debug1: do_cleanup
debug3: PAM: sshpam_thread_cleanup entering
debug1: Killing privsep child 1825

Here is the output of ssh -vvv Username@ExternalIP:

OpenSSH_6.6.1, OpenSSL 1.0.1i 6 Aug 2014
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: ssh_connect: needpriv 0
debug1: Connecting to ExternalIP [ExternalIP] port 22.
debug1: Connection established.
debug1: identity file /home/bdjnk/.ssh/id_rsa type -1
debug1: identity file /home/bdjnk/.ssh/id_rsa-cert type -1
debug1: identity file /home/bdjnk/.ssh/id_dsa type -1
debug1: identity file /home/bdjnk/.ssh/id_dsa-cert type -1
debug1: identity file /home/bdjnk/.ssh/id_ecdsa type -1
debug1: identity file /home/bdjnk/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/bdjnk/.ssh/id_ed25519 type -1
debug1: identity file /home/bdjnk/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.6.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1
debug1: match: OpenSSH_6.6.1 pat OpenSSH_6.6.1* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent

This just hangs, for a looong time. I usually kill it before 20 minutes have passed. One time I left it and it timed out.

Here is the clients' /etc/ssh/ssh_config:

#	$OpenBSD: ssh_config,v 1.28 2013/09/16 11:35:43 sthen Exp $

# This is the ssh client system-wide configuration file.  See
# ssh_config(5) for more information.  This file provides defaults for
# users, and the values can be changed in per-user configuration files
# or on the command line.

# Configuration data is parsed as follows:
#  1. command line options
#  2. user-specific file
#  3. system-wide file
# Any configuration value is only changed the first time it is set.
# Thus, host-specific definitions should be at the beginning of the
# configuration file, and defaults at the end.

# Site-wide defaults for some commonly used options.  For a comprehensive
# list of available options, their meanings and defaults, please see the
# ssh_config(5) man page.

# Host *
#   ForwardAgent no
#   ForwardX11 no
#   RhostsRSAAuthentication no
#   RSAAuthentication yes
#   PasswordAuthentication yes
#   HostbasedAuthentication no
#   GSSAPIAuthentication no
#   GSSAPIDelegateCredentials no
#   BatchMode no
#   CheckHostIP yes
#   AddressFamily any
#   ConnectTimeout 0
#   StrictHostKeyChecking ask
    StrictHostKeyChecking no
#   IdentityFile ~/.ssh/identity
#   IdentityFile ~/.ssh/id_rsa
#   IdentityFile ~/.ssh/id_dsa
#   Port 22
#   Protocol 2,1
#   Cipher 3des
#   Ciphers aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc
#   MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160
#   EscapeChar ~
#   Tunnel no
#   TunnelDevice any:any
#   PermitLocalCommand no
#   VisualHostKey no
#   ProxyCommand ssh -q -W %h:%p gateway.example.com
#   RekeyLimit 1G 1h

And here's the server's /ect/ssh/sshd_config:

#	$OpenBSD: sshd_config,v 1.93 2014/01/10 05:59:19 djm Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

#Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

# The default requires explicit activation of protocol 1
#Protocol 2

# HostKey for protocol version 1
#HostKey /etc/ssh/ssh_host_key
# HostKeys for protocol version 2
#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 1024

# Ciphers and keying
#RekeyLimit default none

# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#RSAAuthentication yes
#PubkeyAuthentication yes

# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile	.ssh/authorized_keys

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to no to disable s/key passwords
ChallengeResponseAuthentication no

# Kerberos options
KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
UsePAM yes

#AllowAgentForwarding yes
#AllowTcpForwarding yes
#GatewayPorts no
#X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
PrintMotd no # pam does that
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
UsePrivilegeSeparation sandbox		# Default for new installations.
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS yes
#PidFile /run/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# override default of no subsystems
Subsystem	sftp	/usr/lib/ssh/sftp-server

# Example of overriding settings on a per-user basis
#Match User anoncvs
#	X11Forwarding no
#	AllowTcpForwarding no
#	PermitTTY no
#	ForceCommand cvs server

Sorry for the massive log and config dump.

I've been messing with this for hours with no progress. Any assistance is greatly appreciated.

Offline

#2 2014-08-31 16:12:04

bdjnk
Member
Registered: 2009-08-24
Posts: 78

Re: SSH Hangs on WAN, Fine on LAN.

I've managed to connect consistently after reading this article and setting my MTU from 1500 to 576 (or even 1472). I'll give it a few days, do a bit more investigating, and hopefully post my findings.

Offline

Board footer

Powered by FluxBB