You are not logged in.

#1 2019-08-08 11:17:35

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Weird SSH connect issue on Wifi (no interactive shell, all else works)

Hi guys,

first of all let me tell you I'm a Unix professional, so I "usually" know what I'm doing. wink

I have this weird issue and I'm not sure whether to attribute it to a HW related issue (proprietary broadcom-wl drivers the cause?) or the configuration.

Please provide your help, I'll provide you with all relevant infos, but feel free to ask me anything, if you believe there's something missing...


[SETUP]

- Ryzen 5 1600, 32 Gb RAM, X370 Taichi, 7x 4Tb NAS HDDs, 1x M.2 SSD (OS), fresh Archlinux Installation via latest ISO (archlinux-2019.08.01-x86_64.iso).
- 1x Ethernet connection
- 1x Wifi Connection via PCIe Card (Asus PCE-AC56, using broadcom-wl-dkms 6.30.223.271-20 driver)
- all packages are latest version, Kernel is linux-lts 4.19.61-1, linux-firmware is 20190717.bf13a71-1
- SSH installed and uses default config (except PermitRootLogin yes and X11Forwarding yes)
- all else is default
- using NetworkManager for network configuration

[NETWORKING INFO]

[root@bignas ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute enp8s0
       valid_lft forever preferred_lft forever
3: wlp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.9/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp11s0
       valid_lft 86065sec preferred_lft 86065sec
[root@bignas ~]# ip route
default via 192.168.1.1 dev wlp11s0 proto dhcp metric 600
192.168.0.0/24 dev enp8s0 proto kernel scope link src 192.168.0.100 metric 100
192.168.1.0/24 dev wlp11s0 proto kernel scope link src 192.168.1.9 metric 600

Internet works normally, can connect to Internet, update packages, everything.

[PROBLEM DESCRIPTION]

I can connect to via ethernet device all fine:

$ ssh root@192.168.0.100
root@192.168.0.100's password:
Last login: Thu Aug  8 12:58:20 2019 from 192.168.0.101
[root@bignas ~]# date
Thu 08 Aug 2019 01:03:53 PM CEST

When I try to do the same on the wireless device, I get stuck at the prompt with no interactive shell until I receive a "connection resetted by IP Port 22" msg. I literally cannot even CTRL-C out of it.

$ ssh root@192.168.1.9
root@192.168.1.9's password:
Connection reset by 192.168.1.9 port 22

Verbose connection attempt:

$ ssh -vvv root@192.168.1.9
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "192.168.1.9" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 192.168.1.9 [192.168.1.9] port 22.
debug1: Connection established.
debug1: identity file /home/myuser/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_rsa-cert type -1
debug1: identity file /home/myuser/.ssh/id_dsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/myuser/.ssh/id_ed25519-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.0
debug1: match: OpenSSH_8.0 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 192.168.1.9:22 as 'root'
debug3: hostkeys_foreach: reading file "/home/myuser/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/myuser/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from 192.168.1.9
debug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com,zlib
debug2: compression stoc: none,zlib@openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib@openssh.com
debug2: compression stoc: none,zlib@openssh.com
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:mSgHJNeMmeDZjMKLhv+GmtTKlxPS9EMftJtn5hS7Jn4
debug3: hostkeys_foreach: reading file "/home/myuser/.ssh/known_hosts"
debug3: record_hostkey: found key type ECDSA in file /home/myuser/.ssh/known_hosts:2
debug3: load_hostkeys: loaded 1 keys from 192.168.1.9
debug1: Host '192.168.1.9' is known and matches the ECDSA host key.
debug1: Found key in /home/myuser/.ssh/known_hosts:2
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 134217728 blocks
debug2: key: /home/myuser/.ssh/id_rsa (0x7fffbcc0efe0)
debug1: Skipping ssh-dss key /home/myuser/.ssh/id_dsa - not in PubkeyAcceptedKeyTypes
debug2: key: /home/myuser/.ssh/id_ecdsa ((nil))
debug2: key: /home/myuser/.ssh/id_ed25519 ((nil))
debug3: send packet: type 5
debug3: receive packet: type 7
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering public key: RSA SHA256:rdvTgy66OhNZl+3m40WbQeE+0ZsP/zIC4LWTYoJnVZQ /home/myuser/.ssh/id_rsa
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /home/myuser/.ssh/id_ecdsa
debug3: no such identity: /home/myuser/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /home/myuser/.ssh/id_ed25519
debug3: no such identity: /home/myuser/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
root@192.168.1.9's password:
debug3: send packet: type 50
debug2: we sent a password packet, wait for reply
debug3: receive packet: type 52
debug1: Authentication succeeded (password).
Authenticated to 192.168.1.9 ([192.168.1.9]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions@openssh.com
debug3: send packet: type 80
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 80
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env GREP_COLOR
debug1: Sending env LC_ALL = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env HOSTTYPE
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env DISPLAY
debug3: Ignored env HISTTIMEFORMAT
debug3: Ignored env EDITOR
debug3: Ignored env WSL_DISTRO_NAME
debug3: Ignored env MACPROOT
debug3: Ignored env USER
debug3: Ignored env PROJECTS
debug3: Ignored env PAGER
debug3: Ignored env PWD
debug3: Ignored env MANPAGER
debug3: Ignored env HOME
debug1: Sending env LC_CTYPE = C.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env NAME
debug3: Ignored env XDG_DATA_DIRS
debug3: Ignored env LOCALE
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug3: Ignored env SHLVL
debug3: Ignored env LANGUAGE
debug3: Ignored env LOGNAME
debug3: Ignored env PATH
debug3: Ignored env HISTSIZE
debug3: Ignored env WSLENV
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: send packet: type 1
Connection reset by 192.168.1.9 port 22

The connection are usually done from my WSL Terminal on Windows or my MobaXTerm, but I also tried to connect to the server from other devices, such as my Smartphone via Terminus, with the same effect though.

What I tried so far:
- deleting the SSH host-keys and re-creating them fresh, no success
- deleting any local kown_hosts from SSH clients, no success
- uninstalling SSH whole and reinstalling with default config, no success

I would like to repeat it works on the ethernet device ONLY, not on the Wifi device, to receive a login / interactive shell.

What's super weird is that on the same wireless device I can't get no normal login, if you instead send to execute any commands with the SSH prompt, the results are sent normally, see here:

$ ssh root@192.168.1.9 "date; echo this works even on wireless"
root@192.168.1.9's password:
Thu Aug  8 13:10:25 CEST 2019
this works even on wireless

So it generally "works", except no interactive login/shell on wireless. To me, this doesn't make any sense and it's the first time I encounter such behavior.

Any ideas?

I'm very grapefruit for useful comments and help.

Thank you guys in advance.

Last edited by Archytect (2019-08-08 11:21:31)

Offline

#2 2019-08-08 11:55:40

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,441
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

If passing date and echo commands work, can you explicitly request a shell:

ssh root@192.168.1.9 /bin/sh

"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#3 2019-08-08 12:14:50

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

One would guess, right?

But apparently not...

$ ssh root@192.168.1.9 "ls -ld /bin/bash"
root@192.168.1.9's password:
-rwxr-xr-x 1 root root 903464 Apr 30 11:13 /bin/bash

$ ssh root@192.168.1.9 "/bin/bash -c 'echo hello'"
root@192.168.1.9's password:
hello

$ ssh root@192.168.1.9 "/bin/bash -c 'echo \$SHELL'"
root@192.168.1.9's password:
/bin/bash

$ ssh root@192.168.1.9 "/bin/bash -c 'echo \$BASH_VERSION'"
root@192.168.1.9's password:
5.0.7(1)-release

$ ssh root@192.168.1.9 "/bin/bash"
root@192.168.1.9's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

I told you it was weird... although to be fair, that's not how I get a shell on the ethernet device either, see here:

$ ssh root@192.168.0.100 "/bin/bash -c 'echo \$BASH_VERSION'"
root@192.168.0.100's password:
5.0.7(1)-release

$ ssh root@192.168.0.100 "/bin/bash"
root@192.168.0.100's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

# However omitting /bin/bash gives a regular shell, see...
$ ssh root@192.168.0.100
root@192.168.0.100's password:
Last login: Thu Aug  8 13:23:25 2019 from 192.168.0.101
[root@bignas ~]# echo $SHELL
/bin/bash
[root@bignas ~]# echo $BASH_VERSION
5.0.7(1)-release
[root@bignas ~]# date
Thu 08 Aug 2019 02:13:49 PM CEST
[root@bignas ~]# exit
logout
Connection to 192.168.0.100 closed.

Offline

#4 2019-08-08 12:31:03

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,441
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

And what about just /bin/sh as requested, or

ssh root@192.168.1.9 /bin/sh -l

"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#5 2019-08-08 13:00:40

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Same results... on both devices btw.

$ ssh root@192.168.0.100 /bin/sh
root@192.168.0.100's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

$ ssh root@192.168.0.100 /bin/sh -l
root@192.168.0.100's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

$ ssh root@192.168.1.9 /bin/sh
root@192.168.1.9's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

$ ssh root@192.168.1.9 /bin/sh -l
root@192.168.1.9's password:
[STUCK IN PROMPT, can exit via CTRL-D]
$

EDIT: Let me provide profiles for evidence & comparison (I didn't touch them, so they should be default):

[root@bignas ~]# cat /etc/profile
# /etc/profile

# Set our umask
umask 022

# Append our default paths
appendpath () {
    case ":$PATH:" in
        *:"$1":*)
            ;;
        *)
            PATH="${PATH:+$PATH:}$1"
    esac
}

appendpath '/usr/local/sbin'
appendpath '/usr/local/bin'
appendpath '/usr/bin'
unset appendpath

export PATH

# Load profiles from /etc/profile.d
if test -d /etc/profile.d/; then
        for profile in /etc/profile.d/*.sh; do
                test -r "$profile" && . "$profile"
        done
        unset profile
fi

# Source global bash config
if test "$PS1" && test "$BASH" && test -z ${POSIXLY_CORRECT+x} && test -r /etc/bash.bashrc; then
        . /etc/bash.bashrc
fi

# Termcap is outdated, old, and crusty, kill it.
unset TERMCAP

# Man is much better than us at figuring this out
unset MANPATH

[root@bignas ~]# cat /etc/bash.bashrc
#
# /etc/bash.bashrc
#

# If not running interactively, don't do anything
[[ $- != *i* ]] && return

[[ $DISPLAY ]] && shopt -s checkwinsize

PS1='[\u@\h \W]\$ '

case ${TERM} in
  xterm*|rxvt*|Eterm|aterm|kterm|gnome*)
    PROMPT_COMMAND=${PROMPT_COMMAND:+$PROMPT_COMMAND; }'printf "\033]0;%s@%s:%s\007" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\~}"'

    ;;
  screen*)
    PROMPT_COMMAND=${PROMPT_COMMAND:+$PROMPT_COMMAND; }'printf "\033_%s@%s:%s\033\\" "${USER}" "${HOSTNAME%%.*}" "${PWD/#$HOME/\~}"'
    ;;
esac

[ -r /usr/share/bash-completion/bash_completion   ] && . /usr/share/bash-completion/bash_completion

[root@bignas ~]# openssl sha224 /etc/profile /etc/bash.bashrc
SHA224(/etc/profile)= bfe19998f259f7217b3f1449831ce9668cfe73ef5679c13cd142da18
SHA224(/etc/bash.bashrc)= 2085c4f9de810fa87d9a4d2a4aefb58a78b7ca8720cdf4c4db0ae589

Last edited by Archytect (2019-08-08 13:03:45)

Offline

#6 2019-08-08 14:21:43

seth
Member
Registered: 2012-09-03
Posts: 49,951

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Can you login as non-root?

Offline

#7 2019-08-08 14:35:46

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

The WiFi interface and Ethernet interface are on different networks.
How are they routed?
How does the network setup on the SSH client you try to connect from look like?
Update: Possibly also a classical MTU problem?

Last edited by schard (2019-08-08 14:38:28)

Online

#8 2019-08-08 14:39:06

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

seth wrote:

Can you login as non-root?

Sure I can, just not on the wireless device.

$ ssh user@192.168.0.100 # ethernet
user@192.168.0.100's password:
Last login: Thu Aug  8 13:23:08 2019 from 192.168.1.10
[user@bignas ~]$ echo hello friend
hello friend
[user@bignas ~]$ logout
Connection to 192.168.0.100 closed.

$ ssh user@192.168.1.9 # wireless
user@192.168.1.9's password:
[NOTHING, not even CTRL-C or CTRL-Z can get you out]

Offline

#9 2019-08-08 14:42:52

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

schard wrote:

The WiFi interface and Ethernet interface are on different networks.
How are they routed?
How does the network setup on the SSH client you try to connect from look like?

So the default route runs on the wireless device.

It's supposed to be that the wireless device is my main administrative network.

The ethernet cable has a direct connection to my workstation with static IPs and no routing, gateway or DNS definitions, just plain old two static IPs talking to each other with no switch, hub or router in between.

The wireless network get's it's IP normally over my Wifi-router and that's where it gets it's internet from.

The routing table is included in the OP above, here is it:

[root@bignas ~]# ip route
default via 192.168.1.1 dev wlp11s0 proto dhcp metric 600
192.168.0.0/24 dev enp8s0 proto kernel scope link src 192.168.0.100 metric 100
192.168.1.0/24 dev wlp11s0 proto kernel scope link src 192.168.1.9 metric 600
schard wrote:

Update: Possibly also a classical MTU problem?

Can you elaborate on that? How can I test this?

Last edited by Archytect (2019-08-08 14:44:07)

Offline

#10 2019-08-08 14:46:07

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Online

#11 2019-08-08 15:13:54

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Ok, this sounds interesting, but I checked and all my devices have MTU 1500 (Server, Client, Router).

"SSH Client"

$ ip addr show dev eth1
8: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ether --:--:--:--:-:--
    inet 192.168.0.101/24 brd 192.168.0.255 scope global dynamic
       valid_lft forever preferred_lft forever
$ ip addr show dev wifi2
38: wifi2: <BROADCAST,MULTICAST,UP> mtu 1500 group default qlen 1
    link/ieee802.11 --:--:--:--:--:--
    inet 192.168.1.10/24 brd 192.168.1.255 scope global dynamic
       valid_lft 69739sec preferred_lft 69739sec

SSH Server:

[root@bignas ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute enp8s0
       valid_lft forever preferred_lft forever
3: wlp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.9/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp11s0
       valid_lft 84860sec preferred_lft 84860sec

Router:

$ ssh admin@192.168.1.1
admin@192.168.1.1's password:
sh: /usr/bin/xauth: not found
admin@RT-AC87U:/tmp/home/root# ip addr show
1: lo: <LOOPBACK,MULTICAST,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.129/24 brd 192.168.0.255 scope global eth0
4: vlan1@eth0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
5: vlan2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
17: eth1: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
18: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br0
    inet xxx.xxx.xxx.xxx/24 brd 169.254.39.255 scope global br0:0

I will still go on and try smaller MTUs on the server's wifi interface and see if it makes a difference, but as of now all wireless devices in my home have MTU 1500.

Offline

#12 2019-08-08 15:16:56

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

I don't see a WiFi interface on the router...

Online

#13 2019-08-08 15:19:51

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

It's this "br0" device, I assume it's a bridged device. 192.168.1.1 is the IP of the router.
The router has Dual band Wifi 2.4Ghz/5Ghz, ... let me try switching to the 2.4Ghz network and see whether that makes a difference.

Last edited by Archytect (2019-08-08 15:21:12)

Offline

#14 2019-08-08 15:20:33

seth
Member
Registered: 2012-09-03
Posts: 49,951

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Let's see whether you tampered w/ access.conf:

pacman -Qkk pam

Offline

#15 2019-08-08 15:21:38

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 1,932
Website

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

br* is normally used for bridges. WiFi interfaces are commonly prefixes wlan* or wlp*. Is the router actually the AP?

Online

#16 2019-08-08 15:22:59

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

seth wrote:

Let's see whether you tampered w/ access.conf:

pacman -Qkk pam
[root@bignas ~]# pacman -Qkk pam
pam: 585 total files, 0 altered files

I setup networking via nmtui, installed Samba, edited smb.conf, sshd_config to allow root logins, that's pretty much it.
Unless I forgot to mention something all else is default Archlinux installation / whatever the packages ship in terms of default config files.

Offline

#17 2019-08-08 15:27:03

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

schard wrote:

br* is normally used for bridges. WiFi interfaces are commonly prefixes wlan* or wlp*. Is the router actually the AP?

Well the router is getting it's IP via the cable-modem (Unitymedia Connectbox with disabled Wifi).

So the cable-modem has an ethernet cable to the internet port of the router, the router receives DHCP IP and has "internet", the router also does the Wifi thing for all wireless devices.

Reason is because the ISP provided cable-modem / router has poor wifi and my router is an ASUS RT-AC87U with excellent reception.

Offline

#18 2019-08-08 16:06:29

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

[UPDATE]

Ok, guys I just burned the latest Ubuntu Desktop LiveCD via rufus and booted it up, installed the broadcom firmware drivers and openssh-server, and tried to connect to the LiveCD environment. With success. 8)

$ rm .ssh/known_hosts
$ ssh ubuntu@192.168.1.9
ubuntu@192.168.1.9's password:
Welcome to Ubuntu 18.10 (GNU/Linux 4.18.0-10-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage



Last login: Thu Aug  8 16:01:55 2019 from 192.168.1.10
ubuntu@ubuntu:~$ ls
Desktop  Documents  Downloads  examples.desktop  Music  Pictures  Public  Templates  Videos
ubuntu@ubuntu:~$ uname -a
Linux ubuntu 4.18.0-10-generic #11-Ubuntu SMP Thu Oct 11 15:13:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ubuntu:~$ dpkg -l | egrep -i broadcom
ii  bcmwl-kernel-source                        6.30.223.271+bdcom-0ubuntu4                amd64        Broadcom 802.11 Linux STA wireless driver source

Boom, I'm in.

This mean, IMHO, two things:

a) it's not an issue/bug with the wireless HW.
b) it's definetely somewhere in Archlinux, be it ssh version or config.

I will try and install ssh from source next and install the arch package, see if I can substanciate my suspicion...

Offline

#19 2019-08-08 19:19:31

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

[UPDATE]

I have installed OpenSSH from source according to the steps described in this site: https://www.tecmint.com/install-openssh … -in-linux/
(Except without --with-selinux, since that would pull in linux-selinux-headers)

Same result.

I realized something now. When I booted the LiveCD I connected to the 2.4Ghz network.

Right now all devices are in the 5Ghz Wifi network.

Apparantly(!!) the login into SSH via the wireless devices don't work from the same network band (2.4Ghz/5Ghz), for whatever reason. I tested with with my laptop at the 2.4Ghz wifi -> I can login. (unbelievable).

It's still weird as to why this is happending to begin with, but at least I can rule out Archlinux specific issues. Still might be HW related.

Offline

#20 2019-08-09 09:29:06

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

[UPDATE]

Ok, so I booted again from Ubuntu Desktop LiveCD and this time I connected to the 5Ghz network. It worked.

So I can say it's not really true that I can't connect from the same bands. But now I'm out of ideas of what the root cause could be.

I feel like this goes back and forth and really doesn't make any sense to me.

$ ssh ubuntu@192.168.1.212
ubuntu@192.168.1.212's password:
Welcome to Ubuntu 18.10 (GNU/Linux 4.18.0-10-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage



Last login: Fri Aug  9 09:24:37 2019 from 192.168.1.10
ubuntu@ubuntu:~$ ls
Desktop  Documents  Downloads  examples.desktop  Music  Pictures  Public  Templates  Videos
ubuntu@ubuntu:~$ date
Fri Aug  9 09:24:54 UTC 2019
ubuntu@ubuntu:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 70:85:c2:71:43:53 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute enp8s0
       valid_lft forever preferred_lft forever
    inet6 fe80::7285:c2ff:fe71:4353/64 scope link
       valid_lft forever preferred_lft forever
3: wlp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 2c:fd:a1:61:36:24 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.212/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp11s0
       valid_lft 86310sec preferred_lft 86310sec
    inet6 fe80::2a77:9cf7:d63a:f720/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
ubuntu@ubuntu:~$ uname -a
Linux ubuntu 4.18.0-10-generic #11-Ubuntu SMP Thu Oct 11 15:13:55 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ubuntu@ubuntu:~$

Offline

#21 2019-08-09 09:55:48

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

[UPDATE]
I tried removing linux-lts 4.19* and replacing it with the current kernel linux 5.2*, no effect.

I just cannot connect to the damn wireless device from my Archlinux installation. It works from LiveCDs (Ubuntu) for whatever reason.

Offline

#22 2019-08-09 13:23:12

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

[UDATE]

In the meantime I tried uninstalling Archlinux and installing Manjaro on the hard disk (not LiveCD booting), still the same effects: can connect to ethernet ssh, but not to ssh via wireless.

Now I installed Ubuntu 18.10 on it and what can I say... I entered the exact same data into nmtui to configure the network and yet there is something different:
a) it WORKS NOW!
b) there is a 169.* IP added that I don't understand where is comes from or what it's purpose is.

See here for evidence:

https://youtu.be/-Q8Ihy_-5m8

Now could somebody please explain to me what is the basic difference between the network setup of the two OS standard-installations? Cuz I don't get it.

from Archlinux:

[root@bignas ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute enp8s0
       valid_lft forever preferred_lft forever
3: wlp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.9/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp11s0
       valid_lft 86065sec preferred_lft 86065sec
[root@bignas ~]# ip route
default via 192.168.1.1 dev wlp11s0 proto dhcp metric 600
192.168.0.0/24 dev enp8s0 proto kernel scope link src 192.168.0.100 metric 100
192.168.1.0/24 dev wlp11s0 proto kernel scope link src 192.168.1.9 metric 600

from Ubuntu:

root@bignas:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp8s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.100/24 brd 192.168.0.255 scope global noprefixroute enp8s0
       valid_lft forever preferred_lft forever
    inet6 fe80::c7af:54ee:ec3:d888/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: wlp11s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.213/24 brd 192.168.1.255 scope global dynamic noprefixroute wlp11s0
       valid_lft 85902sec preferred_lft 85902sec
    inet6 fe80::c64c:8452:6041:ffa4/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
root@bignas:~# ip route
default via 192.168.1.1 dev wlp11s0 proto dhcp metric 600
169.254.0.0/16 dev wlp11s0 scope link metric 1000
192.168.0.0/24 dev enp8s0 proto kernel scope link src 192.168.0.100 metric 100
192.168.1.0/24 dev wlp11s0 proto kernel scope link src 192.168.1.213 metric 600

And if I tried to restore the Archlinux backup I created before installing Ubuntu and just "copied" or duplicated Ubuntu's network config over so I can keep using Archlinux instead of Ubuntu, how would I go about that?

Offline

#23 2019-08-09 13:41:44

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

Let me also provide the NetworkManager config files I extracted from the ROOTFS backup I made and compared it to this new Ubuntu installation's NM files...

$ cat system-connections/wifi.archlinux
[connection]
id=wifi
uuid=f0acdf77-80ad-49cd-a981-d196d17c9c4c
type=wifi
interface-name=wlp11s0
permissions=

[wifi]
mac-address-blacklist=
mode=infrastructure
ssid=wlans_5G

[wifi-security]
key-mgmt=wpa-psk
psk=issecretman

[ipv4]
dns-search=
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=disabled

$ cat system-connections/wifi.ubuntu
[connection]
id=wifi
uuid=79d38ed6-b0f6-42a4-9a7d-1a4b45c3c48e
type=wifi
interface-name=wlp11s0
permissions=

[wifi]
mac-address-blacklist=
mode=infrastructure
ssid=wlans_5G

[wifi-security]
key-mgmt=wpa-psk
psk=issecretman

[ipv4]
dns-search=
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

$ cat system-connections/eth0.archlinux
[connection]
id=eth0
uuid=9bdaab68-b3c6-46ff-958c-7beb41d184ed
type=ethernet
interface-name=enp8s0
permissions=

[ethernet]
mac-address-blacklist=

[ipv4]
address1=192.168.0.100/24
dns-search=
method=manual
never-default=true

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=disabled

$ cat system-connections/eth0.ubuntu
[connection]
id=eth0
uuid=ec7494e4-96e8-305e-8620-90a27e6eff44
type=ethernet
autoconnect-priority=-999
permissions=
timestamp=1565355342

[ethernet]
mac-address=--:--:--:--:--:--
mac-address-blacklist=

[ipv4]
address1=192.168.0.100/24
dns-search=
method=manual
never-default=true

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
method=auto

$ cat NetworkManager.conf
# Configuration file for NetworkManager.
# See "man 5 NetworkManager.conf" for details.

$ cat NetworkManager.conf.ubuntu
[main]
plugins=ifupdown,keyfile

[ifupdown]
managed=false

[device]
wifi.scan-rand-mac-address=no

$ diff system-connections/wifi.*
3c3
< uuid=f0acdf77-80ad-49cd-a981-d196d17c9c4c
---
> uuid=79d38ed6-b0f6-42a4-9a7d-1a4b45c3c48e
24c24
< method=disabled
---
> method=auto

$ diff system-connections/eth0.*
3c3
< uuid=9bdaab68-b3c6-46ff-958c-7beb41d184ed
---
> uuid=ec7494e4-96e8-305e-8620-90a27e6eff44
5c5
< interface-name=enp8s0
---
> autoconnect-priority=-999
6a7
> timestamp=1565355342
8a10
> mac-address=--:--:--:--:--:--
20c22
< method=disabled
---
> method=auto

$ diff NetworkManager.conf*
1,2c1,8
< # Configuration file for NetworkManager.
< # See "man 5 NetworkManager.conf" for details.
---
> [main]
> plugins=ifupdown,keyfile
>
> [ifupdown]
> managed=false
>
> [device]
> wifi.scan-rand-mac-address=no

I mean there are differences, apparently I explicitly disabled/ignored IPV6 configuration, but I cannot imagine that makes such a big difference to allow for this connection "bug".

What do you think?

Last edited by Archytect (2019-08-09 13:42:29)

Offline

#24 2019-08-09 13:50:55

seth
Member
Registered: 2012-09-03
Posts: 49,951

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

b) there is a 169.* IP added that I don't understand where is comes from or what it's purpose is.

18: br0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc htb state UNKNOWN
    link/ether --:--:--:--:--:-- brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.1/24 brd 192.168.1.255 scope global br0
    inet xxx.xxx.xxx.xxx/24 brd 169.254.39.255 scope global br0:0   <<<<<<<<<<================

Afaiu the ISP cable modem has a router as well, so you got a router cascade?
Is the subnet separation deliberate? (It's not "normal" to have wifi and ethernet in different subnets, the carrier is irrelevant for the network layout)

The issue is somewhere in that configuration and the subnet bridging (is what I'm betting your right arm on)

Offline

#25 2019-08-09 14:15:08

Archytect
Member
From: Stuttgart, Germany
Registered: 2011-10-11
Posts: 27

Re: Weird SSH connect issue on Wifi (no interactive shell, all else works)

seth wrote:

Afaiu the ISP cable modem has a router as well, so you got a router cascade?

Correct, the ISP provided cable router has poor configuration options and wifi reception.

I disabled the Wifi of the ISP router, I connected an ethernet cable to my ASUS router into the "internet port".

The ASUS router receives a normal "local" IP address with gateway, dns information etc. like with every other device you plug into your router with cable.

I then configured my ASUS router to share it's "Internet" via Wifi network 192.168.1.0/24.

Offline

Board footer

Powered by FluxBB