Arch Linux

icouto

Member

Registered: 2020-11-12

Posts: 6

[SOLVED] fail2ban not picking up nginx errors in journald log

Nginx logs errors to systemd journald. I've setup a simple http-auth test, and I can see that the errors are being logged without problems:

journalctl -r -p 3 -n 6 -u nginx

-- Logs begin at Fri 2020-11-06 09:54:26 AEDT, end at Fri 2020-11-13 10:57:27 AEDT. --                                                                        
Nov 13 10:56:52 archie-test nginx[7161]: 2020/11/13 10:56:52 [error] 7161#7161: *167 user "hackersix" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"                                                               
Nov 13 10:56:47 archie-test nginx[7161]: 2020/11/13 10:56:47 [error] 7161#7161: *167 user "hackerfive" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"                                                               
Nov 13 10:56:42 archie-test nginx[7161]: 2020/11/13 10:56:42 [error] 7161#7161: *167 user "hackerfour" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"                                                               
Nov 13 10:56:36 archie-test nginx[7161]: 2020/11/13 10:56:36 [error] 7161#7161: *167 user "hackerthree" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"   
Nov 13 10:56:31 archie-test nginx[7161]: 2020/11/13 10:56:31 [error] 7161#7161: *167 user "hackertwo" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"                                                               
Nov 13 10:56:24 archie-test nginx[7161]: 2020/11/13 10:56:24 [error] 7161#7161: *167 user "hackerone" was not found in "/etc/nginx/.htpasswd", client: 192.168.0.2, server: localhost, request: "GET / HTTP/1.1", host: "192.168.0.1"

I've installed fail2ban, and would like to enable the 'nginx-http-auth' jail. Unfortunately, I'm finding it difficult to put together what exactly I'm supposed to do. The docs make it clear that I need to set the 'backend' setting to 'systemd' in 'jail.local'. I've done that, but it doesn't seem enough: fail2ban does not seem to be noticing or registering these login attempts at all:

fail2ban-client status nginx-http-auth

Status for the jail: nginx-http-auth
|- Filter
|  |- Currently failed:	0
|  |- Total failed:	0
|  `- Journal matches:
`- Actions
   |- Currently banned:	0
   |- Total banned:	0
   `- Banned IP list:	

I set the 'loglevel' of fail2ban to 'DEBUG', and saw that it was complaining that the "jail started without journalmatches". I wasn't able to find any examples of what a 'journalmatch' setting should look like for the 'nginx-http-auth' jail, so based only on the info in the man pages, I tried this:

[nginx-http-auth]                                                               
enabled = true                                                                  
backend = systemd                                                               
port    = http,https                                                            
journalmatch = _SYSTEMD_UNIT=nginx.service PRIORITY=3

...but it makes absolutely no difference whatsoever...

So, I'm a bit lost - don't know what to try next. It seems that nginx is logging the errors as it should, but fail2ban is not reading/noticing the log. I feel like there is maybe some silly setting somewhere that I've missed, but I'm stumped - any guidance or suggestions would be greatly appreciated.

Last edited by icouto (2020-11-13 07:43:38)

icouto

Member

Registered: 2020-11-12

Posts: 6

Re: [SOLVED] fail2ban not picking up nginx errors in journald log

I believe I've pinpointed the issue: it's the `failregex` that is automatically loaded by the `nginx-http-auth.conf` filter file. That regex needs a little tweaking, in order to work as expected with systemd-journald logging. The regex seems to expect a full-line match - i.e., that the match will be an entire line, from beginning to end. It seems journald puts some extra info at the beginning - and/or possibly at the end - which throws the regex. All we need to do is to remove the offending parts of the regex. For future reference, here is how I made it work:

1. Make a copy of the original 'nginx-http-auth.conf' filter file:

cp /etc/fail2ban/filter.d/nginx-http-auth.conf /etc/fail2ban/filter.d/nginx-http-auth-journald.conf

2. Open the 'nginx-http-auth-journald.conf' file, and trim the beginning and the end of the declared failregex, so it looks exactly like this:

failregex = \[error\] \d+#\d+: \*\d+ user "(?:[^"]+|.*?)":? (?:password mismatch|was not found in "[^\"]*"), client: <HOST>, server:

3. Open your `jail.local` file, and add a new jail that uses our filter - by adding a new section, like this:

[nginx-http-auth-journald]                                                      
enabled = true    
backend = systemd                                                              
port    = http,https    
journalmatch = _SYSTEMD_UNIT=nginx.service PRIORITY=3 

4. Reload/Restart fail2ban

Seems to work - fail2ban is now catching the http-auth errors logged by nginx, and triggering the ban!