possible ssh break-in?

scar · 2014-02-19 16:31:50

Since I use ssh with password authentication to access my home router from anywhere, I've installed fail2ban to protect it.
It worked well, although I had an average of 20 daily break-in attempts from various IP addresses.
But I've got this on my routers sshd log:

febr 17 20:48:03 ROUTER sshd[2571]: Set /proc/self/oom_score_adj to 0
febr 17 20:48:03 ROUTER sshd[2571]: Connection from 108.166.117.141 port 45083 on 91.82.39.2 port 22
febr 17 20:48:05 ROUTER sshd[2571]: reverse mapping checking getaddrinfo for 108-166-117-141.static.cloud-ips.com [108.166.117.141] failed - POSSIBLE BREAK-IN ATTEMPT!
febr 17 20:48:06 ROUTER sshd[2571]: Accepted password for USERNAME from 108.166.117.141 port 45083 ssh2
febr 17 20:48:06 ROUTER sshd[2571]: pam_unix(sshd:session): session opened for user USERNAME by (uid=0)

and that is all...

and according to the systemd-logind log

febr 17 20:48:06 ROUTER systemd-logind[149]: New session 6 of user USERNAME.
febr 17 20:48:06 ROUTER systemd-logind[149]: Removed session 6.

As far as I know, nothing has changed, no files modified or created at that time.
After this one, I have several unsuccessful attempts in my logs too.
Of course, I've changed the password.

Well, the question is, was that really a successful break-in?

Last edited by scar (2014-02-19 16:32:33)

x33a · 2014-02-19 16:41:32

The sshd log does seem to suggest a successful login. But looking at the logind log, the session didn't even last a second.

Were you using a weak password, because I doubt with fail2ban it can be brute forced.

In any case, this looks rather fishy. Maybe it's a vulnerability in the router that was exploited. What router are you using and which firmware?

scar · 2014-02-19 17:31:40

It is an older astaro gw, but I'm using it since 4 years without any major issues. The box runs Arch.
The password was not random, but not really "coherent' one, with lowercase and uppercase letters, numbers and special characters.

Geolocation of the IP address shows Ohio, US - and when i'm googling after" ssh + that ip address":

http://oucsace.cs.ohiou.edu/~tysko/scanattack.2014.01

and that is an interesting one... six time the same IP as on my router.

Last edited by scar (2014-02-19 17:39:42)

ewaller · 2014-02-19 17:53:32

That certainly looks like they got in. Do you use the same passwords on multiple sites? Is it possible that they were able to get their hands on an unsalted hash of your password? That would allow them to perform an off-line brute force attack.

scar · 2014-02-19 18:04:43

I never use the same password on two different places or boxes.
And according to fail2ban that IP was never blocked.
And that was not my attempt to log in.

Imagine my face now...

Do you have any advices (besides changing my password, that I already did) ?

Last edited by scar (2014-02-19 18:09:12)

ewaller · 2014-02-19 18:14:02

You might do a port scan on the Internet side of your router to ensure there are no open ports. Any chance your are exposing a configuration interface, or an FTP server that would allow someone to retrieve a file such as /etc/passwd from your router? If such a file is available, and if the router does not use shadow, then Harry Hacker could obtain your username and and hash of the password. They could then use a brute force dictionary attack generating hashes at random until they find one that matches the hash you (may) have provided them. Once they get a match, they know your user name and password. Robust systems either use a shadow password file , salted passwords, or both. I am not saying this is what happened, but it is a common attack.

scar · 2014-02-19 18:26:33

No way someone could access my /etc/passwd or /etc/shadow remotely.

The only open ports were for ssh, iperf and for the web interface of my router.

I think I will close them all but iperf on the WAN side and set up an openvpn server...

Last edited by scar (2014-02-19 18:37:11)

brebs · 2014-02-19 18:58:55

scar wrote:

port 22

Don't use port 22 for SSH!

Use a random, free port, e.g. 5107. An easy change to improve security.

scar · 2014-02-20 07:32:37

I know, but I've thought with changing my password frequently and using fail2ban I would be better protected.

By the way, I would like to know, what I have to check in this case.
I mean, I've verified if my UNIX users login conditions have changed since then (I had an rsync backup of the box which is a week older than the break-in). I've verified if there are new services running, files created or modified since that date, checked my .bash_history, closed sshd on the WAN side (I will only use openvpn to connect to it), changed the password, checked if there are new users, checked my iptables for suspicious rules. There is nothing.

I think, there is a plethora of documentation about hardening ssh, but what to do in a case like this?
What else do I have to verify?

x33a · 2014-02-20 09:03:11

As I said previously, the session lasted about a second. Assuming it was a successful login, it probably was an automated one.

Since, nothing seems to be changed, I am not sure what to suggest. Maybe see if any new cron jobs popped up, and check for rootkits.

Lekensteyn · 2014-02-20 09:03:36

@scar You can only verify the machine if you took it off-line and booted from trusted media. Otherwise it is possible that a rootkit has been installed that hides its traces.

If possible, disallow password logins (or if you really need them, restrict them to an IP). Then stick to SSH public-key auth only. Put the following in your sshd_config to restrict password-based auth to user scar in your LAN:

PasswordAuthentication no

Match Address 192.168.1.* User scar
        PasswordAuthentication yes

# Set allowed users (scar, root) and limit logins to an address for root
AllowUsers scar root@192.168.2.*

scar · 2014-02-20 09:47:46

Thank you. Off-line check this evening.

scar · 2014-02-21 10:45:16

I've checked the system while off-line with rkhunter, nothing suspicious was found.
What else can I do?

Spider.007 · 2014-02-21 13:45:41

I would re-install all packages with pacman too (something like pacman -S `pacman -Qq` from chroot). You could also use http://archlinux.spider007.net/pacman-f … unowned.sh to find any files that aren't then overwritten by pacman

Last edited by Spider.007 (2014-02-21 13:46:20)

drcouzelis · 2014-02-21 14:43:04

This thread totally freaked me out. So last night I read through the wiki page on SSH keys and disabled username / password logins. I also changed the SSH port from 22 to something else.

...It's funny, I actually had some crazy trouble with my router. I had my router nicely setup to do port forwarding for port 22, right? I then changed it to the new port, I couldn't get SSH to login, so I changed it back to port 22, and I still couldn't login, even though everything was setup exactly the way it was an hour before! I ended up resetting my router to the default factory settings and reconfiguring it (only took about 10 minutes to do) and as I expected port forwarding starting working again as expected. For your information.

Anyway, with only-keys-allowed logins and a sneaky new SSH port hopefully my SSH server is nice and safe now from the badguys.

scar · 2014-02-22 07:27:24

Well, maybe I'm paranoid, but I will zero out my routers CF card today and reinstall the whole thing; forbid ssh on WAN and use it only from a vpn connection. That is it.

Spider.007 · 2014-02-22 15:10:16

Yeah I agree; it's a bit paranoid but lots better then discovering a rootkit a couple of months from now. I think that a successful login would only be registered somewhere to be abused later, but it's not worth the risk. Next time I'd disable pwd authentication entirely and use keys only

scar · 2014-03-03 19:02:25

This is what I did. Thanks for the help, guys.

Arch Linux

#1 2014-02-19 16:31:50

possible ssh break-in?

#2 2014-02-19 16:41:32

Re: possible ssh break-in?

#3 2014-02-19 17:31:40

Re: possible ssh break-in?

#4 2014-02-19 17:53:32

Re: possible ssh break-in?

#5 2014-02-19 18:04:43

Re: possible ssh break-in?

#6 2014-02-19 18:14:02

Re: possible ssh break-in?

#7 2014-02-19 18:26:33

Re: possible ssh break-in?

#8 2014-02-19 18:58:55

Re: possible ssh break-in?

#9 2014-02-20 07:32:37

Re: possible ssh break-in?

#10 2014-02-20 09:03:11

Re: possible ssh break-in?

#11 2014-02-20 09:03:36

Re: possible ssh break-in?

#12 2014-02-20 09:47:46

Re: possible ssh break-in?

#13 2014-02-21 10:45:16

Re: possible ssh break-in?

#14 2014-02-21 13:45:41

Re: possible ssh break-in?

#15 2014-02-21 14:43:04

Re: possible ssh break-in?

#16 2014-02-22 07:27:24

Re: possible ssh break-in?

#17 2014-02-22 15:10:16

Re: possible ssh break-in?

#18 2014-03-03 19:02:25

Re: possible ssh break-in?

Board footer