[Solved] SSH access denied after installing NFS

tylerpnn · 2016-02-24 03:07:32

I am having a very strange issue right now. I have a computer running arch linux that I use as a home server. I usually connect to it via SSH when I need to update things or move files around, but for some reason as of today whenever I type my password into SSH it tells me "Access Denied". I am using Putty as the SSH client. I know 100% for certain that the password is correct, unless someone has hacked my server and changed my password. From what I can tell, though, nothing on the server seems to not be working (my rutorrent client and plex server still seem to be working fine). The last thing I did on it was try to set up NFS (which failed when I realized that windows doesn't have a free NFS client). However I did end up installing the nfs-utils package and libtirpc library. I started up the nfs server but did not enable the service to start on boot with systemd. The problem persists after rebooting.

The reason I am posting here is because the computer has an old graphics card that doesn't have any ports compatible with the monitor I have. Until I can buy a proper converter dongle, I thought I might as well post here and get some feedback.

Last edited by tylerpnn (2016-02-26 02:40:50)

fukawi2 · 2016-02-24 03:14:21

Yeah, we can't access the server either so there's not much we can tell you beyond what you already know. Obviously installing NFS shouldn't have affected SSH in any way; I would guess it's a coincidence. You really need to get on to the console to be able to view logs.

ewaller · 2016-02-24 03:17:38

tylerpnn wrote:

whenever I type my password into SSH it tells me "Access Denied".

Unlikely. Is that an exact quote?

What is in the logs on your server?

Edit: Are you certain you are hitting the Arch Linux box, or are you hitting the Windows machine?

Last edited by ewaller (2016-02-24 03:19:56)

alphaniner · 2016-02-24 04:02:23

@ tylerpnn In lieu of the video dongle, you might be able to learn something if you could connect your server's HDD to another machine, boot to [live] linux, and view the logs.

@ ewaller "Access denied" seems to be an actual PuTTY error. Also, considering the circumstances accessing the logs seems non-trivial

ewaller · 2016-02-24 04:11:18

I do use PuTTY and have not seen that error. If it is configured for ssh keys, I suppose that might be possible. If we are talking about a PAM password that exists on the server (as opposed to a ssh key password which could not be compromised by an attack on the Arch server) I would expect to see "Permission denied, please try again" to be issued by the Arch Linux server. If it is a home server, why is access to the logs non-trivial?

alphaniner · 2016-02-24 04:20:30

Because he apparently can't even access the server locally due to lack of a monitor.

tylerpnn · 2016-02-24 06:41:07

alphaniner wrote:

@ tylerpnn In lieu of the video dongle, you might be able to learn something if you could connect your server's HDD to another machine, boot to [live] linux, and view the logs.
@ ewaller "Access denied" seems to be an actual PuTTY error. Also, considering the circumstances accessing the logs seems non-trivial

Which log am I looking for?

Also another note, the event log in putty says "Password authentication failed" when it prints Access Denied to the console.

EDIT: HOLY SHIT. I looked at the log, and it is unbelievable. Or maybe this happens a lot more often than I thought. The log shows that I my server has been getting login requests from an IP in china for the past 2 weeks.... at a rate of about 1-5 requests per second... I guess I've been brute forced. Lesson learned, ssh keys from now on.

EDIT2: Well now I'm not so sure... the ssh logs also never show anyone else but me actually getting a successful login. And passwd logs do not show the password changing. Still investigating...

EDIT3: So now I've found that no, my password was not changed. My server was being attacked, but none of the attacks have been successful it seems. It turns out that the stuff I did with installing NFS was probably the problem. I'm getting an error with libtirpc, which I installed when trying to get nfs to work. From the log, this is the error every time I try to log in:

Feb 24 02:55:54 arch sshd[557]: PAM unable to dlopen(/usr/lib/security/pam_unix.so): libtirpc.so.1: cannot open shared object file: No such file or directory
Feb 24 02:55:54 arch sshd[557]: PAM adding faulty module: /usr/lib/security/pam_unix.so
Feb 24 02:55:54 arch sshd[557]: PAM unable to dlopen(/usr/lib/security/pam_access.so): libtirpc.so.1: cannot open shared object file: No such file or directory
Feb 24 02:55:54 arch sshd[557]: PAM adding faulty module: /usr/lib/security/pam_access.so
Feb 24 02:55:56 arch sshd[557]: Failed password for ----- from ---.---.---.-- port 50198 ssh2

EDIT4: I tried using the solution on this page: http://archlinuxarm.org/forum/viewtopic.php?f=31&t=9480. I disabled pam for ssh, and now I can ssh into the server! Except now I get an error about a missing module when I try to become root.

~ >> su
su: Module is unknown
~ >> sudo pacman -S pam
sudo: PAM authentication error: Module is unknown

I think pam has become corrupted somehow. I think my only option is to login on the actual console and update PAM (assuming it lets me become root).

Last edited by tylerpnn (2016-02-24 08:41:36)

alphaniner · 2016-02-24 14:22:08

I think my only option is to login on the actual console and update PAM

Sigh. Suckered by partial updaters twice in a week. Shame on me.

ewaller · 2016-02-24 15:12:52

tylerpnn wrote:

EDIT: HOLY SHIT. I looked at the log, and it is unbelievable. Or maybe this happens a lot more often than I thought. The log shows that I my server has been getting login requests from an IP in china for the past 2 weeks.... at a rate of about 1-5 requests per second... I guess I've been brute forced. Lesson learned, ssh keys from now on.

Yeah, that it is about right. ssh keys are a good idea, but if you want to leave the password vector open I suggest:

1. Strong passwords.
2. sshguard, A brute force detector that creates dynamic firewall rules.
3. Enable two factor authentication. I suggest Google Authenticator

I use all three on my system. I still get attacked several times a day, but the attacks last all of about 10 seconds.

ugjka · 2016-02-24 16:23:09

4. Disable root login (PermitRootLogin no)

ayekat · 2016-02-24 16:37:05

5. Consider alternative ports (not security-through-obscurity, but keep-my-logs-clean)

ewaller · 2016-02-24 16:40:16

ugjka wrote:

0. Disable root login (PermitRootLogin no)

FTFY -- How could I possibly forget that one.

Last edited by ewaller (2016-02-24 16:40:40)

tylerpnn · 2016-02-26 02:40:22

Alright, finally fixed the problem. Turns out that back in November the PAM authors accidently released a new version that was linked against the wrong version of libtirpc. Because of that, I was somehow allowed to update libtirpc without also updating pam at the same time. Obviously, this caused a ton of issues with PAM.

After I got the adapter to let me use my monitor with my server, I still wasn't able to log in. Even after just entering my user name it would give me an error saying something like "Incorrect User" before even prompting me for my password. Using an arch installation usb I was able to arch-chroot and update pam and all of the problems were gone.

Arch Linux

#1 2016-02-24 03:07:32

[Solved] SSH access denied after installing NFS

#2 2016-02-24 03:14:21

Re: [Solved] SSH access denied after installing NFS

#3 2016-02-24 03:17:38

Re: [Solved] SSH access denied after installing NFS

#4 2016-02-24 04:02:23

Re: [Solved] SSH access denied after installing NFS

#5 2016-02-24 04:11:18

Re: [Solved] SSH access denied after installing NFS

#6 2016-02-24 04:20:30

Re: [Solved] SSH access denied after installing NFS

#7 2016-02-24 06:41:07

Re: [Solved] SSH access denied after installing NFS

#8 2016-02-24 14:22:08

Re: [Solved] SSH access denied after installing NFS

#9 2016-02-24 15:12:52

Re: [Solved] SSH access denied after installing NFS

#10 2016-02-24 16:23:09

Re: [Solved] SSH access denied after installing NFS

#11 2016-02-24 16:37:05

Re: [Solved] SSH access denied after installing NFS

#12 2016-02-24 16:40:16

Re: [Solved] SSH access denied after installing NFS

#13 2016-02-26 02:40:22

Re: [Solved] SSH access denied after installing NFS

Board footer