yacron - forked from dillon's cron 3.2, includes anacron-like features

tri1976 · 2010-01-30 14:11:58

I switched from fcron to dcron (fcron is removed). I don't think the jobs in /etc/cron.* are running, please help me set this up.
Here is the output from "sudo crontab -l"

@hourly ID=sys-hourly /usr/sbin/run-cron /etc/cron.hourly
@daily ID=sys-daily /usr/sbin/run-cron /etc/cron.daily
@weekly ID=sys-weekly /usr/sbin/run-cron /etc/cron.weekly
@monthly ID=sys-monthly /usr/sbin/run-cron /etc/cron.monthly

Here is the output from /var/log/crond.log

Jan 29 19:45:33 optiplex crond[21928]: /usr/sbin/crond 4.4 dillon's cron daemon, started with loglevel info
Jan 29 19:45:33 optiplex crond[21928]: no timestamp found (user root job sys-hourly)
Jan 29 19:45:33 optiplex crond[21928]: no timestamp found (user root job sys-daily)
Jan 29 19:45:33 optiplex crond[21928]: no timestamp found (user root job sys-weekly)
Jan 29 19:45:33 optiplex crond[21928]: no timestamp found (user root job sys-monthly)
Jan 29 19:48:01 optiplex crond[21928]: FILE /var/spool/cron/root USER root PID 21993 job sys-hourly
Jan 29 19:48:02 optiplex crond[21998]: mailing cron output for user root job sys-hourly
Jan 29 19:48:02 optiplex crond[21998]: unable to exec /usr/sbin/sendmail: cron output for user root job sys-hourly to /dev/null
Jan 29 20:48:01 optiplex crond[21928]: FILE /var/spool/cron/root USER root PID 24330 job sys-hourly
Jan 29 20:48:01 optiplex crond[24335]: mailing cron output for user root job sys-hourly
Jan 29 20:48:01 optiplex crond[24335]: unable to exec /usr/sbin/sendmail: cron output for user root job sys-hourly to /dev/null
Jan 29 20:57:01 optiplex crond[21928]: FILE /var/spool/cron/root USER root PID 24937 job sys-daily
Jan 29 20:57:15 optiplex crond[25129]: mailing cron output for user root job sys-daily
Jan 29 20:57:15 optiplex crond[25129]: unable to exec /usr/sbin/sendmail: cron output for user root job sys-daily to /dev/null

I don't use sendmail, how do I redirect error/output to a log file?

patroclo7 · 2010-01-30 14:29:32

tri1976 wrote:

I switched from fcron to dcron (fcron is removed)

Fcron is actually in the AUR and many people are using it (for example to receive mails only when there is an error with the erroronlymail option, or to disable logging for specific jobs etc.). This does not answer your problem (sorry) but your post gave the impression that fcron is generally unavailable in archlinux or deprecated for some reason.

Profjim · 2010-01-30 15:21:31

Hi tri11976, the jobs in /etc/cron.daily and so on are running, that's what the lines in your cron log like:

tri1976 wrote:

Jan 29 19:48:01 optiplex crond[21928]: FILE /var/spool/cron/root USER root PID 21993 job sys-hourly
Jan 29 19:48:02 optiplex crond[21998]: mailing cron output for user root job sys-hourly
Jan 29 19:48:02 optiplex crond[21998]: unable to exec /usr/sbin/sendmail: cron output for user root job sys-hourly to /dev/null

mean. The jobs are running, producing some error putput, and crond is trying to mail that output to you. But failing.

tri1976 wrote:

I don't use sendmail, how do I redirect error/output to a log file?

Make a script something like this:

#!/bin/bash
{
    echo
    cat
    echo "---------------------------------------- "
} >> /var/log/cron-output.log

Name it something like /usr/local/bin/cron-output, do a chmod +x on it, and then
edit your /etc/rc.d/crond script like this:

@@ -10,7 +10,7 @@ start)
 
    # defaults to using syslog, and sendmail-ing cron output to local user
    # to mail output to remote address instead, add "-m user@host"
-   if [[ -z $PID ]] && /usr/sbin/crond -S -l info; then
+   if [[ -z $PID ]] && /usr/sbin/crond -S -l info -M /usr/local/bin/cron-output; then
 
        PID=$(pidof -o %PPID /usr/sbin/crond)
        echo "$PID" > /var/run/crond.pid

You'll probably also want to add "etc/rc.d/crond" (note no initial backslash) to the end of your "NoUpgrade =" line in /etc/pacman.conf. That will tell pacman to preserve the changes you've made to /etc/rc.d/crond, rather than overwriting them everytime there's an upgrade.

Alternatively, you could make a link from /usr/sbin/sendmail to /usr/local/bin/cron-output, but I wouldn't really recommend that. It'd be fine as far as dcron is concerned, but it might mislead other programs.

steve___ · 2010-01-30 16:05:47

Could the -M setting be handled as a variable in the crontab itself? Or can dcron have a cofing file (eg $HOME/.dcronrc)?

Profjim · 2010-01-30 16:18:40

steve___ wrote:

Could the -M setting be handled as a variable in the crontab itself?

Not as a global, once-per-crontab setting. That would require massive changes to the crontab parsing.

However, you can do this on a job-by-job basis. Just replace:

* * * * * cronjob args

with:

* * * * * cronjob args > /usr/local/bin/cron-output 2>&1

Or can dcron have a cofing file (eg $HOME/.dcronrc)?

I'm not now warm on that idea. It's quite simple to just adjust the command-line invocation of dcron and tell your package manager not to overwrite your changes. But I'll remember that you've asked for it and maybe someday I'll see enough reason to implement this.

ataraxia · 2010-01-30 23:21:47

Jim, I saw something interesting with manually-scheduled jobs. I forced a run of a job scheduled with @hourly by writing it into the cron.update file. This worked fine, but the next run of the job after the manual one was still based on the time of the last automatic run.

To be clear, I got this:

Jan 29 16:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 4217 job sys-hourly
Jan 29 17:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 5080 job sys-hourly
Jan 29 18:30:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6654 job sys-hourly <-- (manual run)
Jan 29 18:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6711 job sys-hourly
Jan 29 19:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6954 job sys-hourly

and what I expected was more like this:

Jan 29 16:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 4217 job sys-hourly
Jan 29 17:48:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 5080 job sys-hourly
Jan 29 18:30:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6654 job sys-hourly <-- (manual run)
Jan 29 19:30:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6711 job sys-hourly
Jan 29 20:30:01 amaranth crond[4043]: FILE /var/spool/cron/root USER root PID 6954 job sys-hourly

Is this the expected behavior, or a bug? Maybe it would be wiser to delay the re-checking of timestamp files until after any jobs get run.

Profjim · 2010-01-31 00:48:26

ataraxia wrote:

Jim, I saw something interesting with manually-scheduled jobs. I forced a run of a job scheduled with @hourly by writing it into the cron.update file. This worked fine, but the next run of the job after the manual one was still based on the time of the last automatic run.

Thanks, that is interesting.

Is this the expected behavior, or a bug? Maybe it would be wiser to delay the re-checking of timestamp files until after any jobs get run.

Honestly, I wrote the scheduling logic months ago and I don't remember. I think the time of the next run is getting stored in the data structure after your 17:48 run, and then your 18:30 manual run doesn't clear that because 18:30 + 1 hour is > 18:48. I agree with you it's more natural to expect your second behavior. I'm not sure whether to be concerned about this, though, as this seems an unusual corner case.

However, I will look again at the scheduling logic when I get a chance to confirm this diagnosis. I can't rule out that this is the manifestation of some bug. If it's an easy tweak to get the behavior you're expecting, I'll do that.

Thanks for bringing it to my attention.

Profjim · 2010-01-31 15:11:59

Profjim wrote:

However, I will look again at the scheduling logic when I get a chance to confirm this diagnosis. I can't rule out that this is the manifestation of some bug. If it's an easy tweak to get the behavior you're expecting, I'll do that.

OK, checked the code and yes it's more or less like I said. Basically @daily jobs are scheduled to run all the time but suppressed for a period of one day following their last run. The field line->cl_NotUntil keeps track of when a job should be suppressed until. This field is getting updated by the normal scheduler, in TestJobs, and isn't touched by the bit of code (in CheckUpdates) which picks up cron.update requests to manually schedule a job.

It doesn't seem like it would be very difficult to have the CheckUpdates code also update the cl_NotUntil field, however it would take some care to do it right and to test it to be sure I haven't broken something in the process. I can't do that right away I'm afraid. Would this be very useful to you, or is it just a curiosity? That'll affect its priority.

(Patches and/or testing volunteers are always welcome.)

ataraxia · 2010-01-31 15:57:54

Profjim wrote:

It doesn't seem like it would be very difficult to have the CheckUpdates code also update the cl_NotUntil field, however it would take some care to do it right and to test it to be sure I haven't broken something in the process. I can't do that right away I'm afraid. Would this be very useful to you, or is it just a curiosity? That'll affect its priority.

It's no big deal to me. I don't manually schedule things very often, and none of my jobs cause me any harm if they run too often once in a while.

orbisvicis · 2010-01-31 23:13:31

what is the difference between a minute and a month in the Frequency?

for example:
# run no more than once a month, and only (once) between the 1st and the 10th of every month
* * 1-10 * * ID=cull_thumbnails FREQ=1mo ...

Profjim · 2010-01-31 23:24:22

Sorry, ran out of letters. FREQ only understands 'm' (minutes), 'h' (hours), 'd' (days), and 'w' (weeks). You could use FREQ=30d. (That's what @monthly uses, by the way---we don't strictly honor once/month. And @yearly is 365d.)

orbisvicis · 2010-02-01 00:47:31

Thanks, I'll do that
btw, what do you mean "not strictly honoring once/month"
? not stricly following 28/29/30/31 etc days depending on the month
? if the script fails, retry every "default" time period, as opposed to FREQ=30d/30d

probably reading too much into the man page. "# run whenever it's between 2-4 am, and at least one day (1d) has elapsed since this job ran **successfully**"

Last edited by orbisvicis (2010-02-01 01:03:08)

orbisvicis · 2010-02-01 01:10:00

Oh, is it also possible that the FREQ and the days get out of sync? Perhaps 2 years later that cron job will no longer run?

Edit: Nevermind, that wouldn't happen in this case (though if more months were shorter than 30 days maybe the cronjob would skip a month), and probably wouldnn't in any other cases.

Last edited by orbisvicis (2010-02-01 01:18:00)

Profjim · 2010-02-01 01:15:00

orbisvicis wrote:

Thanks, I'll do that
btw, what do you mean "not strictly honoring once/month"
? not stricly following 28/29/30/31 etc days depending on the month

That's what I mean. When you say "monthly" it always interprets that as exactly 30 days.

? if the script fails, retry every "default" time period, as opposed to FREQ=30d/30d

If I remember rightly (perhaps I don't), @monthly translates to FREQ=30d/Xm, where X is the number of minutes in 30 days, divided by 20.

Similarly, @daily translates to FREQ=1d/Xm, where X is the number of minutes in one day, divided by 20. So @daily and so on also will try again if the job returns with exit code 11.

probably reading too much into the man page. "# run whenever it's between 2-4 am, and at least one day (1d) has elapsed since this job ran **successfully**"

Whoops, I say that? That's an error. I remember I did keep screwing that up. The try-again will only happen if the job returns with exit code 11. If the job succeeds with exit code 0, or fails with any other exit code besides 11, crond will say, ok now this job has run, I won't try it again for another day (if it's @daily), and so on.

I'll fix the manpage in the next release, thanks for catching that.

Profjim · 2010-02-01 01:22:39

orbisvicis wrote:

Oh, is it also possible that the FREQ and the days get out of sync? Perhaps 2 years later that cron job will no longer run?

Well, it treats 1 day as just 24*60*60 seconds, so after 60 leap seconds, which would be about 90 years, it will have slipped 1 minute. I don't think we need to worry about this.

It's only @monthly and @yearly where the program's interpretation differs to any appreciable degree from the correct values. And even then, crond will try to run any job as soon as it can _after_ what it understands "one month" or "one year" to be. So any jobs should still get run, just not perhaps at the exact times a human would expect.

Last edited by Profjim (2010-02-01 01:25:03)

orbisvicis · 2010-02-01 01:24:41

orbisvicis wrote:

"# run whenever it's between 2-4 am, and at least one day (1d) has elapsed since this job ran **successfully**"

well, I added the *asterisks* to emphasize my interpretation

Last edited by orbisvicis (2010-02-01 01:26:02)

ataraxia · 2010-02-01 15:48:41

Jim, could we have an /etc/conf.d/crond file to go with the /etc/rc.d/crond, so we can use a variable like $CROND_ARGS to set extra command-line arguments? This would remove the need to add /etc/rc.d/crond to NoUpgrade - instead, the conf.d file would just go into the backup array. I have the feeling the Arch devs don't want to do this on their own, as it would deviate from upstream, but if you provided it, I think Paul would take it with no problem.

steve___ · 2010-02-01 18:06:18

I agree. Expecting users to edit /etc/rc.d/FILE is not standard.

Profjim · 2010-02-01 18:31:21

Yeah, this is a nice idea. When I expressed reluctance about a settings file, I was thinking of something that the crond binary would need to interpret. It seems perfectly reasonable to me to have the /etc/rc.d/crond script source a settings file for arguments to supply to the daemon. I don't even think this is up to me, but to Paul who's the Arch packager. But I'll suggest it to him for the next version. (I distribute a sample /etc/rc.d script in the source tarball, but it's only intended as an example.)

ataraxia · 2010-02-01 20:23:30

Profjim wrote:

I don't even think this is up to me, but to Paul who's the Arch packager. But I'll suggest it to him for the next version. (I distribute a sample /etc/rc.d script in the source tarball, but it's only intended as an example.)

Paul distributes your sample rc.d script, sample root crontab, and sample logrotate config verbatim. That's primarily why I think it'll go over best if the conf.d file, and the change to the rc.d script to accommodate it, come from you, as I get the impression that he wants to stay as close to your upstream as possible.

Profjim · 2010-02-02 02:16:18

ataraxia wrote:

Profjim wrote:
I don't even think this is up to me, but to Paul who's the Arch packager. But I'll suggest it to him for the next version. (I distribute a sample /etc/rc.d script in the source tarball, but it's only intended as an example.)
Paul distributes your sample rc.d script, sample root crontab, and sample logrotate config verbatim. That's primarily why I think it'll go over best if the conf.d file, and the change to the rc.d script to accommodate it, come from you, as I get the impression that he wants to stay as close to your upstream as possible.

There's a chicken-and-egg here. I distribute those samples because that's what Arch has been using all along. Wherever the initiative comes from, I'll probably distribute as samples whatever we end up distributing in Arch.

raymboard · 2010-02-03 16:04:11

Hi.

How to make a script run on Monday, Teusday, Wednesday, Thursday and Friday at nine o clock?

raymboard · 2010-02-03 16:20:52

I think this will work:

0 9 * * mon-fri command

zivziv · 2010-02-05 06:19:38

dcron 4.4 executes cron.daily twice a day sometimes:

$ sudo grep "sys-daily" /var/log/crond.log

Feb  1 00:01:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 16564 job sys-daily
Feb  1 00:02:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 17801 job sys-daily
Feb  2 00:02:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 12413 job sys-daily
Feb  3 00:02:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 30449 job sys-daily
Feb  4 00:02:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 11783 job sys-daily
Feb  4 00:03:01 ziv crond[3433]: FILE /var/spool/cron/root USER root PID 13849 job sys-daily

$ sudo crontab -l

# root crontab
# DO NOT EDIT THIS FILE MANUALLY! USE crontab -e INSTEAD

# man 1 crontab for acceptable formats:
#    <minute> <hour> <day> <month> <dow> <tags and command>
#    <@freq> <tags and command>

# SYSTEM DAILY/WEEKLY/... FOLDERS
@hourly         ID=sys-hourly   /usr/sbin/run-cron /etc/cron.hourly
@daily          ID=sys-daily    /usr/sbin/run-cron /etc/cron.daily
@weekly         ID=sys-weekly   /usr/sbin/run-cron /etc/cron.weekly
@monthly        ID=sys-monthly  /usr/sbin/run-cron /etc/cron.monthly

$ sudo cat /var/spool/cronstamps/root.sys-daily

2010-02-04 00:02

Last edited by zivziv (2010-02-05 06:23:00)

Profjim · 2010-02-05 12:18:24

zivziv wrote:

dcron 4.4 executes cron.daily twice a day sometimes:

Hi thanks, I identified and fixed one bug that might be causing this. See
http://bbs.archlinux.org/viewtopic.php? … 47#p701047. The fix is in dcron-git now and will be in the next release.

However, I am still surprised by the actual crond.log snippet you posted. There may well be another issue involved here.

If you have the time to follow the instructions at http://bbs.archlinux.org/viewtopic.php? … 77#p698877 and send me the debugging info, that would help me determine why you're seeing the behavior you are.

Thanks.

Arch Linux

#26 2010-01-30 14:11:58

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#27 2010-01-30 14:29:32

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#28 2010-01-30 15:21:31

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#29 2010-01-30 16:05:47

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#30 2010-01-30 16:18:40

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#31 2010-01-30 23:21:47

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#32 2010-01-31 00:48:26

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#33 2010-01-31 15:11:59

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#34 2010-01-31 15:57:54

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#35 2010-01-31 23:13:31

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#36 2010-01-31 23:24:22

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#37 2010-02-01 00:47:31

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#38 2010-02-01 01:10:00

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#39 2010-02-01 01:15:00

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#40 2010-02-01 01:22:39

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#41 2010-02-01 01:24:41

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#42 2010-02-01 15:48:41

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#43 2010-02-01 18:06:18

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#44 2010-02-01 18:31:21

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#45 2010-02-01 20:23:30

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#46 2010-02-02 02:16:18

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#47 2010-02-03 16:04:11

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#48 2010-02-03 16:20:52

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#49 2010-02-05 06:19:38

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

#50 2010-02-05 12:18:24

Re: yacron - forked from dillon's cron 3.2, includes anacron-like features

Board footer