yacron - forked from dillon's cron 3.2, includes anacron-like features

Profjim · 2009-08-23 19:17:16

This program was forked from the beautifully minimalist dcron v.3.2. I just needed it to be a bit less minimalist.

I had gotten to the point where I was juggling a combination of dcron, a home-patched version of anacron, my own script to log job output instead of mailing it, my own script to test for various conditions (on A/C? have a network?) before running most jobs, and I still couldn't get the ensemble to do what I now needed. It was getting too baroque. I had only ten or fifteen of my own cronjobs, but they were spread between the root crontab, /etc/anacrontab, some /etc/cron.hourly/* scripts, and where was all the beautiful minimalism now?

Probably I could get fcron to do what I wanted, but it looked so much more complex and heavier than I needed. I just wanted my tools to be a bit more capable.

So, since dcron has such a nice small codebase, I just took it and wrapped in all I needed from anacron... plus the extra capabilities I was hankering for. I've tried to retain the ideals of the original.

This implementation:

* can log to syslog or a file
* can email job output to a specified address, instead of the local user who owns the job
* can replace sendmail with any other script to which all job output should be piped
* allows jobs to be scheduled on the "2nd and last" Mondays of a month
* accepts @daily, @reboot, etc. options in crontabs; @daily jobs don't require the machine to be running at any specific time
* also accepts crontab lines with the format:

* 2-6 * * *  FREQ=2d/1h ID=<job_name> <command>

this will try to run <command> once every 2 days, but only between the hours of 2 and 6 am; if <command> exits with status EAGAIN it will be retried again every 1 hour
* also accepts AFTER=<target_job>/1d assignments: if the specified target_job is scheduled to run within the next 1 day, wait until it has run and exited successfully before running the current command

I welcome any feedback, code reviews, patches, bug reports.

Homepage
Git Repository
AUR Package

john_schaf · 2009-10-14 16:32:35

Sounds good !

You speak about "test for various conditions (on A/C? have a network?)" in the introduction. Is this implemented yet ? how does it work ?

cheers

Profjim · 2009-10-14 17:33:16

It's not internally implemented. The way I do it is, you have a crontab line like:

* * * * *  FREQ=2d/1h ID=<job_name> wrapper-script

wrapper-script tests for the desired condition---for example, checks if ac is available. If it ISN'T, wrapper-script returns exit code 11 (EAGAIN). Then yacron will try calling wrapper-script again in 1 hour. If ac IS available, then wrapper-script should just exec whatever command you want. When yacron sees that that command completed successfully, it will wait 2 days before running this job again.

Gen2ly · 2009-10-14 22:55:42

A couple questions:

* allows jobs to be scheduled on the "2nd and last" Mondays of a month

I'm guessing this is generic right? I'm guessing this could be Thursday, Saturday...

* accepts @daily, @reboot, etc. options in crontabs; @daily jobs don't require the machine to be running at any specific time...

I like this (I always wondered why cron didn't start a cron job if my computer just got booted and a job say wasn't run because it during it's off time), not sure about the nomenclature though.

FREQ=2d/1h

Doesn't seem kiss.

The idea sounds great since it seems to me that crons are becoming a little legacy these days (i.e. not many people working on updating them). I particularly like the ability of writing to log instead of mail or having to be able to choose an email application.

Profjim · 2009-10-14 23:14:29

Gen2ly wrote:

* allows jobs to be scheduled on the "2nd and last" Mondays of a month
I'm guessing this is generic right? I'm guessing this could be Thursday, Saturday...

Yes, right. I don't think this feature is such a big deal. But it seems a little bit useful, and more useful than the interpretation it replaces. (In dcron, a crontab line starting "* * 2,5 Jan Mon" means "run this job on Jan 2, Jan 5, and also on every Monday of January.)

Gen2ly wrote:

* accepts @daily, @reboot, etc. options in crontabs; @daily jobs don't require the machine to be running at any specific time...
I like this (I always wondered why cron didn't start a cron job if my computer just got booted and a job say wasn't run because it during it's off time), not sure about the nomenclature though.

These names are taken verbatim from other, more complex cron daemons like vixie cron. I wouldn't have chosen these exact names myself. The crontab format used here isn't _perfectly_ compatible with other cron daemons, but it is 95% compatible. For the most part, if you took a yacron crontab and used it with a more capable cron daemon like vixie cron, the worst that would happen is some of the extra features of yacron would be ignored. There wouldn't be a syntax error.

Gen2ly wrote:

FREQ=2d/1h
Doesn't seem kiss.

The functionality isn't simple, but is very useful. The idea is that you want a base frequency, and then a retry frequency if the initial attempts fail. For example, if you have a job on your laptop that you want to run once every two days, but it can only run when the network is up, this would let you attempt the job every hour, and then once you "succeeded" (= returned something other than EAGAIN), don't try again for two days.

As to the nomenclature, I agree that isn't so transparent. But I was trying to keep the crontab format reasonably simple, and easy to parse, and such that other cron daemons would just interpret the special yacron features as assignments to environment variables, which would be harmless.

Gen2ly wrote:

The idea sounds great since it seems to me that crons are becoming a little legacy these days (i.e. not many people working on updating them). I particularly like the ability of writing to log instead of mail or having to be able to choose an email application.

I hope it works well for you.

Gen2ly · 2009-10-16 04:17:25

Profjim wrote:

* allows jobs to be scheduled on the "2nd and last" Mondays of a month
Gen2ly wrote:
I'm guessing this is generic right? I'm guessing this could be Thursday, Saturday...
Yes, right. I don't think this feature is such a big deal. But it seems a little bit useful, and more useful than the interpretation it replaces. (In dcron, a crontab line starting "* * 2,5 Jan Mon" means "run this job on Jan 2, Jan 5, and also on every Monday of January.)

I'd been thinking about the implementation of this. While I understand the reason to be compatible, there is a reason I call it legacy, perhaps at times it's better to break things . Perhaps in this case it's not necessary, I actually think this idea could be pretty useful. Say if you wanted to run a script twice a month, I was thinking a better implementation of this would be:

05 00 * * 1,0

This would run the script on the first sunday of the month or for my example before:

05 00 * * 3,1,0

For the first and third weeks of the month. I don't know but it seems more kiss to me.

Profjim wrote:

FREQ=2d/1h
Gen2ly wrote:
Doesn't seem kiss.
The functionality isn't simple, but is very useful. The idea is that you want a base frequency, and then a retry frequency if the initial attempts fail. For example, if you have a job on your laptop that you want to run once every two days, but it can only run when the network is up, this would let you attempt the job every hour, and then once you "succeeded" (= returned something other than EAGAIN), don't try again for two days...

This is where I get into more of my thinking about legacy (i.e when Linux was used largely for servers). How many people who have their computer off actually don't want the script run when the computer starts if a cron job was missed. The default behavior assumes that the compute is on (all the time). Of course, changing this behavior would definitely cause some breakages with compatibility but something I've wanted to seen in cron for quite some time.

I don't know, just spitting out a few ideas here...

Profjim · 2010-01-07 19:55:27

As I posted on the mailing list http://mailman.archlinux.org/pipermail/ … 09981.html, I've now taken over maintainership of dcron and yacron has been merged into dcron 4.0, which is hosted here: http://www.jimpryor.net/linux/dcron.html.

steve___ · 2010-01-08 00:29:22

\o/

ffjia · 2010-01-19 07:22:53

I have installed dcron, 4.3-1, from testing. When it's running with anacrond (yes, I know I have no need for anacron, since dcron has anacron-like feature), it seems dcron and anacrond cannot cope with each other. Everyday I receive two same mails of logwatch in a few minutes...

How to deal with this? Should add "Conflicts With: anacron" in dcron's PKGBUILD file?

Allan · 2010-01-19 07:31:13

Or... you could uninstall anacron...

Profjim · 2010-01-19 13:03:53

Honestly I don't see any need for anacron if one uses the recent versions of dcron, but neither do they conflict. Someone who has complex anacron jobs set up and working might want to use the dcron upgrade but leave anacron in place. There shouldn't be any problem doing that: you'll need to have your crontabs set up to periodically run anacron, just as you always did. Where one would get into trouble is if one had some job scheduled both directly in a crontab and also in your anacrontab. Then both daemons would run that job, and wouldn't pay attention to the fact that the other daemon had also run it. It sounds like that's what you're experiencing. IIRC, the default install off anacron runs all the scripts in the /etc/cron.daily etc. folders. dcron also does this in its crontab for root (do

sudo crontab -l

to see root's crontab). Just disable those jobs for one of the daemons or the other.

gour · 2010-01-23 08:43:16

Profjim wrote:

This implementation:
* can log to syslog or a file
* can email job output to a specified address, instead of the local user who owns the job
* can replace sendmail with any other script to which all job output should be piped

I'm interested for some example how to achieve the above, i.e. logging output of cron job and/or emailing job output to specified address using msmtp?

Sincerely,
Gour

Edit: Ahh, just found out there is '-m' option when invoking crond...:/

Last edited by gour (2010-01-23 10:02:08)

ffjia · 2010-01-27 02:31:10

I was wrong, about receive same logwatch mail. Even after I removed anacron, I could still receive two or three mail from logwatch. It seemd that root user had a crontab:

# root crontab
# DO NOT EDIT THIS FILE MANUALLY! USE crontab -e INSTEAD

# man 1 crontab for acceptable formats:
#    <minute> <hour> <day> <month> <dow> <tags and command>
#    <@freq> <tags and command>

# SYSTEM DAILY/WEEKLY/... FOLDERS
@hourly         ID=sys-hourly   /usr/sbin/run-cron /etc/cron.hourly
@daily          ID=sys-daily    /usr/sbin/run-cron /etc/cron.daily
@weekly         ID=sys-weekly   /usr/sbin/run-cron /etc/cron.weekly
@monthly        ID=sys-monthly  /usr/sbin/run-cron /etc/cron.monthly

And there's /etc/cron.daily/0logwatch file. Does that mean dcron will run jobs under /etc/cron.* twice?

Profjim · 2010-01-27 03:32:49

No, if that's your root crontab, and I can assume the following:
* there are no other user crontabs saying to do anything
with logwatch specifically, or with the /etc/cron.daily folder
* there is no crontab in /etc/cron.d saying anything about logwatch or
/etc/cron.daily
* you didn't only _remove_ anacrond, it's also no longer _running_ (try "pgrep anacrond"---anything show up?)

If all of that's true, then the /etc/cron.daily jobs should only be run once a day.

In one version of dcron, I was getting bug reports that @hourly jobs would be re-run every time crond was started, even if not yet an hour had passed. I never found out whether that was also true for @daily jobs, but perhaps it is. So one explanation of the behavior you're seeing might be: you're restarting crond (perhaps because you're restarting your computer) and you've got a version of dcron with this bug, and so the /etc/cron.daily jobs are all getting run again. They'll be run about 72 minutes after crond starts.

I _think_ that the latest versions of dcron, in [testing] and the dcron-git, fix this bug. Let me know if your problems persist.

ffjia · 2010-01-29 01:40:27

Profjim wrote:

No, if that's your root crontab, and I can assume the following:
* there are no other user crontabs saying to do anything
with logwatch specifically, or with the /etc/cron.daily folder
* there is no crontab in /etc/cron.d saying anything about logwatch or
/etc/cron.daily
* you didn't only _remove_ anacrond, it's also no longer _running_ (try "pgrep anacrond"---anything show up?)
If all of that's true, then the /etc/cron.daily jobs should only be run once a day.

Yep, that's all true...

And after I removed root's crontab, cronjob in /etc/cron.*/ were stopped, including logwatch, updatedb, etc..

Maybe, I'll try dcron-git someday...

According to this:

http://bugs.archlinux.org/task/16085

The arch devs deside to use bcron as an replacement of dcron now?

Allan · 2010-01-29 01:48:44

No, read the closure message. We are using dcron-4.x which is yacron...

ffjia · 2010-01-29 03:55:21

Ok, thanks Allan

After I installed dcron-git, I still got three logwatch mail. The following's from the /var/log/crond.log

Jan 29 10:21:14 southpark crond[2125]: /usr/sbin/crond 4.4 dillon's cron daemon, started with loglevel info
Jan 29 10:22:01 southpark crond[2125]: FILE /var/spool/cron/ffjia USER ffjia PID 2325 fetchmail
Jan 29 10:22:01 southpark crond[2125]: FILE /var/spool/cron/root USER root PID 2326 job sys-hourly
Jan 29 10:22:01 southpark crond[2125]: FILE /var/spool/cron/root USER root PID 2327 job sys-daily
Jan 29 10:22:01 southpark crond[2125]: FILE /var/spool/cron/root USER root PID 2328 job sys-weekly
Jan 29 10:22:02 southpark crond[2338]: mailing fetchmail output for user ffjia
Jan 29 10:32:01 southpark crond[2125]: FILE /var/spool/cron/root USER root PID 5299 job sys-daily
Jan 29 10:42:02 southpark crond[2125]: FILE /var/spool/cron/root USER root PID 7961 job sys-daily
Jan 29 10:46:33 southpark crond[9315]: mailing job sys-daily output for user root
Jan 29 10:47:04 southpark crond[9454]: mailing job sys-daily output for user root
Jan 29 10:48:43 southpark crond[9836]: mailing job sys-daily output for user root

Profjim · 2010-01-29 09:44:44

Hi ffjia, I emailed you asking for some more debugging info.

raymboard · 2010-01-29 15:01:58

Hi.

How do I make a bash script end with exit status EAGAIN ?

JohannesSM64 · 2010-01-29 16:03:53

exit {0,1}

Profjim · 2010-01-29 17:08:58

JohannesSM64 wrote:

exit {0,1}

I'd do "exit 11". What Johannes writes would be equivalent to "exit 0 1" and I don't think that would work. But maybe he knows something that I don't.

raymboard · 2010-01-29 19:06:34

OK, so how can I make an "@daily" job retry every hour if it returns exit status 11?

Profjim · 2010-01-29 19:26:58

Long way:

* * * * *  FREQ=1d/1h ID=<job_name> <command>

Shortcut:

@daily ID=<job_name> <command>

The shortcut has less fine-grained control over how often the retries happen, but it's probably good enough for your purposes. Specifically, a job like @daily will work as though it were FREQ=1d/Xm, where X minutes are 1/20th of a day (so, I think, 72 minutes).

JohannesSM64 · 2010-01-29 19:45:08

Profjim wrote:

JohannesSM64 wrote:
exit {0,1}
I'd do "exit 11". What Johannes writes would be equivalent to "exit 0 1" and I don't think that would work. But maybe he knows something that I don't.

I misunderstood.. Should have read more before replying.

Last edited by JohannesSM64 (2010-01-29 19:47:31)

raymboard · 2010-01-29 21:11:11

Profjim wrote:

Long way:
* * * * *  FREQ=1d/1h ID=<job_name> <command>
Shortcut:
@daily ID=<job_name> <command>
The shortcut has less fine-grained control over how often the retries happen, but it's probably good enough for your purposes. Specifically, a job like @daily will work as though it were FREQ=1d/Xm, where X minutes are 1/20th of a day (so, I think, 72 minutes).

Thanks for making very good program.

Arch Linux

#1 2009-08-23 19:17:16

yacron - forked from dillon's cron 3.2, includes anacron-like features

#2 2009-10-14 16:32:35

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#3 2009-10-14 17:33:16

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#4 2009-10-14 22:55:42

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#5 2009-10-14 23:14:29

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#6 2009-10-16 04:17:25

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#7 2010-01-07 19:55:27

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#8 2010-01-08 00:29:22

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#9 2010-01-19 07:22:53

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#10 2010-01-19 07:31:13

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#11 2010-01-19 13:03:53

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#12 2010-01-23 08:43:16

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#13 2010-01-27 02:31:10

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#14 2010-01-27 03:32:49

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#15 2010-01-29 01:40:27

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#16 2010-01-29 01:48:44

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#17 2010-01-29 03:55:21

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#18 2010-01-29 09:44:44

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#19 2010-01-29 15:01:58

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#20 2010-01-29 16:03:53

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#21 2010-01-29 17:08:58

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#22 2010-01-29 19:06:34

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#23 2010-01-29 19:26:58

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#24 2010-01-29 19:45:08

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#25 2010-01-29 21:11:11

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

Board footer