Arch Linux

eap

Member

Registered: 2012-02-04

Posts: 5

[SOLVED]Very strange ssh problem: can't connect locally, only remotely

TL;DR: Server at home ONLY rejects ssh connections from my Arch box and ONLY when I direct ssh toward its local address.  Can't figure out why.  VERY FRUSTRATING.

Full version:

Hi everybody.

I've been running Arch for a little over a month and I love it.  However, I have this one, extremely frustrating problem that I can't for the life of me figure out. 

First, let me briefly describe my home network. I have a cable modem connected to a wireless router that has some wired devices attached to it.  One of those devices is my Ubuntu server box and another is my new Arch desktop.  Since I also have a laptop that I take with me everyday, I have DynDNS set up so that I can always ssh into my server when I'm away from home.  I use an ssh config file so that I can just type "ssh homer" or "ssh homel" when I want to connect to my home server from a remote location or locally (HOME Remote, HOME Local).  I only use pubkey authentication, and port forwarding is set up correctly.  Simple enough, right?

With my laptop, it always works.  I can connect to the server using both configurations (local address and DynDNS address) when I'm on the local network (both wireless and ethernet).  It also works when I'm away from home.  It also works with other machines that I use, both locally and remotely.

With my (Arch) desktop, however, ssh is only successful if I use the "homer" alias, ie, when I point ssh toward my external address.  When I try to connect over the LAN, the connection is closed every time.  I get a simple "Connection closed by 192.168.35.72" (the server, of course).  Every time.  No matter what changes I make to my client's config file or the server's sshd_config file, it's always the same situation: I can connect to my external address but not the local one. 

I've looked at logs, run ssh in verbose mode, made new keys, googled my butt off, and searched the forum, yet I can't figure out the problem.  When I run "ssh -vvv homel", I can't find any useful clues.  Here's (the tail end of) what I get:

...
debug1: Host '[192.168.35.72]:32608' is known and matches the RSA host key.
debug1: Found key in /home/eap/.ssh/known_hosts:4
debug2: bits set: 555/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/eap/.ssh/id_rsa (0x718f10)
debug2: key: /home/eap/.ssh/id_dsa ((nil))
debug2: key: /home/eap/.ssh/id_ecdsa ((nil))
Connection closed by 192.168.35.72

Thing is, I've used meld to compare this to a successful connection, and they are all but identical.  All that changes is the number of bits set and the 0x718f10, but those change between successful attempts anyway.  The successful connections just keep going instead of having the connection closed.  This is what is shown after the last line above:

...
debug2: key: /home/eap/.ssh/id_ecdsa ((nil))
debug1: Authentications that can continue: publickey
debug3: start over, passed a different list publickey
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/eap/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg ssh-rsa blen 533
debug2: input_userauth_pk_ok: fp ****************
debug3: sign_and_send_pubkey: RSA ******************
debug1: Authentication succeeded (publickey).
Authenticated to eap.gotdns.org ([**.**.**.**]:32608).
...

I've also set the log level to DEBUG3 in sshd_config, and again the difference between a local and a remote connection is that the local one just stops.  Here's the tail end of the log file for a failed connection (192.168.35.64 is my Arch box):

...
debug1: Client protocol version 2.0; client software version OpenSSH_5.9
debug1: match: OpenSSH_5.9 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3p1 Debian-3ubuntu7
debug2: fd 3 setting O_NONBLOCK
debug2: Network child is on pid 9818
debug3: preauth child monitor started
debug3: mm_request_receive entering
debug3: monitor_read: checking request 0
debug3: mm_answer_moduli: got parameters: 1024 1024 8192
debug3: mm_request_send entering: type 1
debug2: monitor_read: 0 used once, disabling now
debug3: mm_request_receive entering
debug3: monitor_read: checking request 5
debug3: mm_answer_sign
debug3: mm_answer_sign: signature 0x21582f10(271)
debug3: mm_request_send entering: type 6
debug2: monitor_read: 5 used once, disabling now
debug3: mm_request_receive entering
debug3: monitor_read: checking request 7
debug3: mm_answer_pwnamallow
debug3: Trying to reverse map address 192.168.35.64.

A successful connection doesn't say anything about whether it was able to reverse map.  When I google that line, I get people talking about the VerifyReverseMap setting in old sshd_config's, which doesn't seem to be in use anymore.

Am I forgetting something simple?  Is there some weird Arch quirk I don't know about?  Is there a limit to how many devices can use one publickey?  I have no idea what the problem could be.  Arrrrg!

Sorry for such a long post, and sorry if I'm violating etiquette.  Any help would be greatly appreciated!  Thank you for your time, fellow Archers.

Last edited by eap (2012-02-04 20:32:22)

hokasch

Member

Registered: 2007-09-23

Posts: 1,461

Re: [SOLVED]Very strange ssh problem: can't connect locally, only remotely

just some random debugging ideas, sorry..
- try with password login instead of keys
- delete known hosts
- can you "ping archdesktophostname" from the server (and the other way round)?
- any differences in network setup between dekstop and laptop (dhcpd, name servers etc)?

eap

Member

Registered: 2012-02-04

Posts: 5

Re: [SOLVED]Very strange ssh problem: can't connect locally, only remotely

Thanks for the ideas, hokasch!

- I turned on password authentication and hid my client's id_rsa.  I was still able to connect using the external address (and entering my password), but I got the same dropped connection when using the LAN address.

- Deleting known hosts didn't help.  It just asked me to add it again and then promptly dropped the connection.

- I can ping my external address from both the server and the client, and I can ping each other locally using their IP addresses.  With my laptop, I can use an alias locally (eg., servername.local, laptopname.local), but that doesn't work with my Arch box.  What I mean is, no other devices know where to go if I give them archbox.local, and likewise my arch machine doesn't know about servername.local or laptop.local.  I don't really know where the ".local" extension originates.  Could that be the issue?

- Both my laptop and desktop are connected to the same router using dhcp and have addresses reserved for them based on their MAC addresses.  Both use my router as their nameserver (which uses the public Google DNS servers)

Aha!

Thanks again for the suggestions!  Poking around networking stuff gave me an idea: adding a line for my desktop in /etc/hosts on the server.  I'm not sure if I'll consider it a workaround or a solution, but I can log in locally now!  As soon as I comment out that line, the problem returns.  I guess that means the problem was with my server not being able to look up a hostname for my desktop?  Or maybe my desktop wasn't communicating its name to the network?  I'm not sure, but I'm going to mark this thread as solved, even though I would love to hear ideas about why an extra line is necessary in /etc/hosts only for my desktop.

Strike0

Member

From: Germany

Registered: 2011-09-05

Posts: 1,489

Re: [SOLVED]Very strange ssh problem: can't connect locally, only remotely

I guess that means the problem was with my server not being able to look up a hostname for my desktop?  Or maybe my desktop wasn't communicating its name to the network?  I'm not sure, but I'm going to mark this thread as solved, even though I would love to hear ideas about why an extra line is necessary in /etc/hosts only for my desktop.

Great you solved it. Your laptop is running OSX or something else?
The problem was that your router did not advertise the hostnames. You don't have autodiscovery on the clients, so the only point for your clients to get a resolv for "localhomer" is the router and that keeps it for itself.
I am not using it, but have a look at avahi and also here.

eap

Member

Registered: 2012-02-04

Posts: 5

Re: [SOLVED]Very strange ssh problem: can't connect locally, only remotely

Thanks for the help!  Yes, my laptop is running OSX, and avahi is running on the Ubuntu server.  Also, avahi was installed (but not running) on my Arch desktop.  I took at look at the Arch Wiki link you posted, and after simply installing nss-mdns and adding avahi-daemon to the DAEMONS section of my rc.conf, everything works!  After restarting my Arch box (and commenting out the line in my server's /etc/hosts file for trying out avahi) I was able to ssh into my server using the local address, and pinging [archbox].local from there works.  Easy fix  

Looks like this problem is definitely solved, AND I got to learn a little about avahi.  Sweet!  Thanks, everybody!

Last edited by eap (2012-02-05 18:46:06)