Crond "Stack Smashing" Error

proof-by-accident · 2019-06-18 18:05:49

Hi all, this may not be the right subforum for this, but it appears that the System Admin subforum has been closed down so I figured I'd try my luck here

I run Arch (kernel x86_64 Linux 4.20.12-arch1-1-ARCH) on a Linode VPS to host my personal website, and as a sandbox to screw around and practice networking or sysadmin stuff. I have been using cronie to schedule my SSL certificate renewal for a while without issue, but last week my certificate expired! I dug in to the /var/spool/anacron files and found that anacron hadn't run since April 22. I forced anacron to run using

 anacron -fdn

and it went through just fine. At this point I'm pretty confident the issue isn't with anacron itself. Okay so then what's wrong? I checked

 systemctl status cronie

and got the following:

● cronie.service - Periodic Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/cronie.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2019-02-28 09:56:25 MST; 3 months 18 days ago
 Main PID: 25143 (crond)
    Tasks: 1 (limit: 1164)
   Memory: 2.1M
   CGroup: /system.slice/cronie.service
           └─25143 /usr/bin/crond -n

Jun 18 09:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated
Jun 18 10:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated
Jun 18 10:40:46 localhost systemd[1]: Reloading Periodic Command Scheduler.
Jun 18 10:40:46 localhost systemd[1]: Reloaded Periodic Command Scheduler.
Jun 18 10:41:01 localhost crond[25143]: (CRON) INFO (running with inotify support)
Jun 18 10:42:01 localhost crond[25143]: (root) CAN'T OPEN (/etc/crontab): No such file or directory
Jun 18 11:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated
Jun 18 11:42:01 localhost crond[25143]: (root) CAN'T OPEN (/etc/crontab): No such file or directory
Jun 18 11:43:01 localhost crond[25143]: (root) CAN'T OPEN (/etc/crontab): No such file or directory
Jun 18 12:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated

The "stack smashing problem" seemed promising, so I pulled up

journalctl -u cronie.service

and lo and behold:

Apr 22 13:01:01 localhost CROND[3490]: (root) CMD (run-parts /etc/cron.hourly)
Apr 22 14:01:01 localhost CROND[27096]: (root) CMD (run-parts /etc/cron.hourly)
Apr 22 15:01:01 localhost CROND[18019]: (root) CMD (run-parts /etc/cron.hourly)
Apr 22 16:01:01 localhost CROND[8817]: (root) CMD (run-parts /etc/cron.hourly)
Apr 22 17:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated
Apr 22 18:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated
Apr 22 19:01:01 localhost crond[25143]: *** stack smashing detected ***: <unknown> terminated

It seems like the problem began on the day that anacron stopped working, so my best guess is that since cronie uses cron to run anacron, this stack smashing issue is what's preventing anacron from running on schedule. But now I'm at a total deadend. I've tried reinstalling cronie (hence the "Reloading" stuff up above), but the problem clearly persists. Google is turning up nothing useful for why crond is encountering this problem. Does anyone know what might be causing this, or better yet how I could fix it?

SOLUTION: I removed cronie and switched to dcron instead

Last edited by proof-by-accident (2019-06-18 19:43:50)

qinohe · 2019-06-18 20:15:16

It it not so that there's nothing to be found on this subject 'stack smashing' which leeds to 'stackbuffer overflow'
https://en.wikipedia.org/wiki/Stack_buffer_overflow
https://debian-administration.org/artic … for_Debian
and a still deeper topic with examples:
https://www.exploit-db.com/papers/24085
Well, it was that I found your topic interesting enough to dive into, though,consider me a nitwit on security and coding for certain ;-)
What you have suffered is ' a died canary'. There may or may not have been an attack on that server you host, and there was a buffer overflow. Good for you the process has died because the 'canary died first' giving a signal to process to stop because checked values did not return the correct values back,which, in return killed the process (cronie) because it was not save to continue...
Now you want some advice from someone that's better in coding and security than I am, but at least you have an idea of what could be going on back there... good luck.

proof-by-accident · 2019-06-18 20:34:48

Thanks qinohe! Haha, very true that Google does return stuff on "stack smashing", just not stuff useful to the problem at hand, ie. a simple fix ; )

I really hope there wasn't an attack on my server! I did find this closed issue on the cronie GitHub, which seems like it's a related issue to what I'm experience, but the suggested fix was over my head (and the proposal to adjust LimitSTACK in the service file didn't work for me).

Either way, dcron seems to be working for now, so I'm just gonna consider this one "solved".

qinohe · 2019-06-18 20:54:13

Your welcome, I can not really give a one on one advice,but if it were me this was happening to, I would install snort, and I would audit my system with tools like Lynis Aide and the like .. it could be the underlying problem lies with cronie, but I would like to know that is so, so I'd say deepen yourself into security (if you didn't already do that), read the Debian security handbook etc. etc. etc.

seth · 2019-06-18 21:19:02

Check coredumpctl on whether maybe cronie left behind coredumps - if they point towards load_env, the link you found applies to your issue.

OT, sysadmin was closed because nobody read the stickies…
Also please fix your hostname and ensure to have read the reset of the installation instructions.

proof-by-accident · 2019-06-19 15:43:15

qinohe- thanks for the tips; I'll definitely make sure that my security is locked down!

seth- yeah it did leave a bunch of coredumps, but they don't seem to point to load_env:

TIME                            PID   UID   GID SIG COREFILE  EXE
Mon 2019-04-22 17:01:02 MDT    2760     0     0   6 missing   /usr/bin/crond

Yeah I was reading the history, alas! Sorry but I don't understand what you mean about fixing my hostname.

seth · 2019-06-19 16:03:38

https://wiki.archlinux.org/index.php/Ne … e_hostname - it's part of the network configuration which is part of the installation process.

You'll actually have to look into the coredump, but the one for the particular incident you posted (a crash from April) has been deleted.
"man coredumpctl"

proof-by-accident · 2019-06-19 16:30:48

To clarify- I know what the host_name is, but I don't understand why mine needs to be fixed.

Yes the line I posted was representative of the entire output of coredumpctl; all the coredumps appear to have been deleted.

seth · 2019-06-19 16:37:32

wiki wrote:

A hostname is a unique name created to identify a machine on a network

"localhost" is not an acceptable hostname in a LAN and will get you in trouble if eg. some smart-ass NM
figures it needs to be fixed, because eg. the X11 server resolution runs across the hostname and the NM might change the hostname after the X11 server is up.

If this is the absolute only device in your LAN you probably can get away w/ a generic hostname like localhost - given you're not running in formentioned troubles. Still fix it.

You will still be able to address the localhost as "localhost", even though its hostname is now different.

proof-by-accident · 2019-06-19 16:39:20

I see, thanks!

Arch Linux

#1 2019-06-18 18:05:49

Crond "Stack Smashing" Error

#2 2019-06-18 20:15:16

Re: Crond "Stack Smashing" Error

#3 2019-06-18 20:34:48

Re: Crond "Stack Smashing" Error

#4 2019-06-18 20:54:13

Re: Crond "Stack Smashing" Error

#5 2019-06-18 21:19:02

Re: Crond "Stack Smashing" Error

#6 2019-06-19 15:43:15

Re: Crond "Stack Smashing" Error

#7 2019-06-19 16:03:38

Re: Crond "Stack Smashing" Error

#8 2019-06-19 16:30:48

Re: Crond "Stack Smashing" Error

#9 2019-06-19 16:37:32

Re: Crond "Stack Smashing" Error

#10 2019-06-19 16:39:20

Re: Crond "Stack Smashing" Error

Board footer