Cronwhip - no more missed cronjobs

Lockheed · 2013-11-28 23:42:50

I don't know what you mean. That's the full output, except the last line:

$ 0.status cronwhip
cronwhip.service - Run missed cronjobs with cronwhip.
   Loaded: loaded (/usr/lib/systemd/system/cronwhip.service; enabled)
   Active: failed (Result: exit-code) since Thu 2013-11-28 23:52:30 CET; 48min ago
  Process: 3576 ExecStart=/usr/bin/cronwhip (code=exited, status=1/FAILURE)
 Main PID: 3576 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/cronwhip.service

And there is crontab for root, even though it does not seem to run tasks which are in it.

Xyne · 2013-11-29 00:02:57

I don't have a command named "0.status" but it looks like it's an alias to "systemctl status". Run that command as root and it will print out the last lines of the log, which should include the error output from cronwhip when it failed. You can also check the output directly with "journalctl -xn".

Normally systemctl prints an error message when it fails with instructions to check the aforementioned output. This should be done as root if you have not made the journal accessible to your non-root user.

Lockheed · 2013-11-29 06:09:54

journalctl -xn produces nothing cron-related.

Here is the output of systemctl status with su, which for some reason is different than with sudo:

cronwhip.service - Run missed cronjobs with cronwhip.
   Loaded: loaded (/usr/lib/systemd/system/cronwhip.service; enabled)
   Active: failed (Result: exit-code) since Fri 2013-11-29 06:55:45 CET; 12min ago
  Process: 506 ExecStart=/usr/bin/cronwhip (code=exited, status=1/FAILURE)
 Main PID: 506 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/cronwhip.service

Nov 29 06:55:45 panzor cronwhip[506]: error: invalid interval [2013-11-28 23:49, 2013-11-28 23:49]
Nov 29 06:55:45 panzor systemd[1]: cronwhip.service: main process exited, code=exited, status=1/FAILURE
Nov 29 06:55:45 panzor systemd[1]: Failed to start Run missed cronjobs with cronwhip..
Nov 29 06:55:45 panzor systemd[1]: Unit cronwhip.service entered failed state.

Xyne · 2013-11-30 17:34:14

Please post root's crontab and the output of the following run as root in a terminal:

cronwhip -v --show-missed
cronwhip -vn

Lockheed · 2013-11-30 19:25:12

# root crontab
# DO NOT EDIT THIS FILE MANUALLY! USE crontab -e INSTEAD

# man 1 crontab for acceptable formats:
#    <minute> <hour> <day> <month> <dow> <tags and command>
#    <@freq> <tags and command>

# SYSTEM DAILY/WEEKLY/... FOLDERS
*/1 * * * sh /home/juha/.scripts/empty_pacman_cache.bash
@hourly         ID=sys-hourly   /usr/bin/run-cron /etc/cron.hourly
@daily          ID=sys-daily    /usr/bin/run-cron /etc/cron.daily
@weekly         ID=sys-weekly   /usr/bin/run-cron /etc/cron.weekly
@monthly        ID=sys-monthly  /usr/bin/run-cron /etc/cron.monthly

$ cronwhip -v --show-missed
error: invalid literal for int() with base 10: 'sh'
[$ cronwhip -vn
/usr/bin/modprobed-db store &> /dev/null
error: invalid literal for int() with base 10: 'sh'

Xyne · 2013-12-01 17:56:56

Lockheed wrote:

*/1 * * * sh /home/juha/.scripts/empty_pacman_cache.bash

It looks like your crontab is missing a field. Each entry should consist of 5 time fields and one command field. You only have 4 time fields in the quoted line, so the "sh" is interpretted as the 5th.

Refer to the following for syntax:
man 5 crontab
https://en.wikipedia.org/wiki/Cron
http://www.thegeekstuff.com/2009/06/15- … -examples/

Lockheed · 2013-12-02 00:55:49

Well, that was embarrassing. Thanks for pointing out my oversight.

Lockheed · 2013-12-03 00:30:32

Looks, like it wasn't causing this problem. The problem still exists, it just goes away during some reboots.

Xyne · 2013-12-03 23:27:35

Lockheed wrote:

Looks, like it wasn't causing this problem. The problem still exists, it just goes away during some reboots.

What's the error message? What's the output from the previous commands?

Lockheed · 2013-12-04 13:34:25

$ cronwhip -v --show-missed
error: invalid interval [2013-12-04 10:11, 2013-12-04 10:11]

cronwhip.service - Run missed cronjobs with cronwhip.
   Loaded: loaded (/usr/lib/systemd/system/cronwhip.service; enabled)
   Active: failed (Result: exit-code) since Wed 2013-12-04 10:12:12 CET; 4h 21min ago
  Process: 327 ExecStart=/usr/bin/cronwhip (code=exited, status=1/FAILURE)
 Main PID: 327 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/cronwhip.service

Xyne · 2013-12-04 17:47:19

Post the output again, along with the most recent shutdown and reboot times reported by "last -x".

If the interval really is 0 then there is nothing to do. If there is something wrong with your hardware clock then you can create your own service file, set it to run 5 minutes after the system starts up (or however long it takes the clock to sync), and use "--end now" to make sure that it uses the sync'd time.

Lockheed · 2013-12-11 10:03:22

cronwhip -v --show-missed
cronwhip -vn

return nothing

$ last -x
juha     pts/1        :0.0             Wed Dec 11 11:00   still logged in
juha     pts/1        :0.0             Wed Dec 11 10:54 - 10:56  (00:01)
root     pts/5        :0.0             Wed Dec 11 10:49 - 10:53  (00:04)
juha     pts/4        :0.0             Wed Dec 11 10:48 - 10:54  (00:05)
root     pts/4        :0.0             Wed Dec 11 10:42 - 10:48  (00:05)
root     pts/4        :0.0             Wed Dec 11 10:42 - 10:42  (00:00)
root     pts/4        :0.0             Wed Dec 11 10:34 - 10:40  (00:05)
juha     pts/3        :0.0             Wed Dec 11 10:33 - 10:54  (00:20)
juha     pts/2        :0.0             Wed Dec 11 10:29 - 10:54  (00:24)
root     pts/2        :0.0             Wed Dec 11 10:27 - 10:27  (00:00)
root     pts/2        :0.0             Wed Dec 11 10:27 - 10:27  (00:00)
root     pts/2        :0.0             Wed Dec 11 10:27 - 10:27  (00:00)
root     pts/2        :0.0             Wed Dec 11 10:27 - 10:27  (00:00)
root     pts/2        :0.0             Wed Dec 11 10:27 - 10:27  (00:00)
juha     pts/1        :0               Wed Dec 11 10:26 - 10:53  (00:27)
juha     pts/1        :0.0             Tue Dec 10 15:18 - 16:29  (01:10)
juha     pts/1        :0.0             Tue Dec 10 13:56 - 13:56  (00:00)
juha     pts/1        :0.0             Tue Dec 10 12:40 - 13:02  (00:22)
juha     pts/0        :0               Tue Dec 10 11:47   still logged in
juha     tty1                          Tue Dec 10 11:47   still logged in
reboot   system boot  3.12.1-pf        Tue Dec 10 11:47   still running
shutdown system down  3.12.1-pf        Tue Dec 10 11:18 - 11:47  (00:29)
juha     pts/1        :0.0             Tue Dec 10 11:15 - 11:15  (00:00)
juha     pts/1        :0.0             Tue Dec 10 10:30 - 10:34  (00:04)
juha     pts/1        :0.0             Tue Dec 10 09:14 - 10:22  (01:07)
juha     pts/1        :0.0             Tue Dec 10 08:55 - 09:12  (00:16)
juha     pts/1        :0.0             Tue Dec 10 00:46 - 00:46  (00:00)
juha     pts/0        :0               Tue Dec 10 00:46 - 11:16  (10:30)
juha     tty1                          Tue Dec 10 00:45 - down   (10:32)
reboot   system boot  3.12.1-pf        Tue Dec 10 00:45 - 11:18  (10:32)
juha     pts/1        :0               Mon Dec  9 21:43 - 21:43  (00:00)
juha     pts/0        :0               Mon Dec  9 21:34 - 00:43  (03:09)
juha     tty1                          Mon Dec  9 21:34 - crash  (03:11)
reboot   system boot  3.12.1-pf        Mon Dec  9 21:34 - 11:18  (13:43)
shutdown system down  3.12.1-pf        Mon Dec  9 21:33 - 21:34  (00:00)
root     pts/2        :0.0             Mon Dec  9 21:32 - 21:32  (00:00)
root     pts/2        :0.0             Mon Dec  9 21:32 - 21:32  (00:00)

Slower shutdowns started just two days ago when I put in an old HDD that I need to extract data from.

Xyne · 2013-12-11 17:35:50

Lockheed wrote:

shutdown system down  3.12.1-pf        Tue Dec 10 11:18 - 11:47  (00:29)

If it returns nothing then there were no (detectable) missed jobs in that interval.

Lockheed · 2013-12-12 22:55:12

Ok, I got something more this time:

$ cronwhip -v --show-missed
error: invalid literal for int() with base 10: 'rm'
$ cronwhip -vn
/usr/bin/modprobed-db store &> /dev/null
sh ./.scripts/emptytrash.bash
error: invalid literal for int() with base 10: 'rm'

and root:

# cronwhip -vn
hostsblock
# cronwhip -vn --show-missed
2013-12-12 23:49 hostsblock
2013-12-12 23:50 hostsblock

Last edited by Lockheed (2013-12-12 22:56:37)

Xyne · 2013-12-12 23:07:34

Lockheed wrote:

$ cronwhip -v --show-missed
error: invalid literal for int() with base 10: 'rm'

And the crontab shows...?

If it's the same error as above, then you already have the solution.

Lockheed wrote:

# cronwhip -vn
hostsblock
# cronwhip -vn --show-missed
2013-12-12 23:49 hostsblock
2013-12-12 23:50 hostsblock

It seems to be working.

Lockheed · 2013-12-13 06:17:31

No, it must be something else, cause I already corrected timings everywhere:

*/1 * * * *   /usr/bin/modprobed-db store &> /dev/null
*/10 * * * * sh ./.scripts/emptytrash.bash
0 * * * * sh ./.scripts/emptymozilla.bash
30 * * * * sh ./.scripts/emptychrome.bash
0 * * * * sh ./.scripts/emptychrome2.bash
30 * * * * sh ./.scripts/emptychromeFS.bash
0 * * * * sh ./.scripts/emptythumbs.bash
30 * * * * sh ./.scripts/emptyopera.bash
0 * * * * sh ./.scripts/emptyopera2.bash
30 * * * * sh ./.scripts/emptyopera3.bash
0 * * * * sh ./.scripts/emptycache.bash
30 * * * * sh ./.scripts/emptyscreenshots.bash
*/5 * * * rm -r -f /mnt/Disk_D/.Trash-1*
15 * * * * rm -rf /home/juha/.opera-backup-crashrecovery-*
15 * * * * rm -rf /home/juha/.config/google-chrome-backup-crashrecovery-*
#0 0 */3 * * SpiderOak
@monthly sh ./.scripts/.ipfilter
#50 22 * * * SpiderOak --headless
#30 21 * * * killall SpiderOak

Xyne · 2013-12-13 22:51:53

Lockheed wrote:

No, it must be something else, cause I already corrected timings everywhere:
*/5 * * * rm -r -f /mnt/Disk_D/.Trash-1*
15 * * * * rm -rf /home/juha/.opera-backup-crashrecovery-*
15 * * * * rm -rf /home/juha/.config/google-chrome-backup-crashrecovery-*

Ffs, the output told you that there was something wrong with the crontab and it even told you that it was with a command that starts with "rm". Look at the first rm command that you posted. There is clearly a field missing.

I don't have the patience to walk you through this over and over again. Cronwhip works as intended afaict. Any errors are almost certainly due to your configuration. Reread the previous posts and make an effort to resolve the problem yourself. Your carelessness alone has dragged this on far longer than it should have been, and forcing me to fish for info with every post has only added to the tedium.

I hope you get it working as you like, but at this point I think I'm done trying to help you.

Lockheed · 2013-12-13 22:57:42

Yeah, I missed that. Don't really know how it happened. Sorry about that. I appreciate your help and apologies for being thoughtless.

Lockheed · 2014-01-06 13:06:44

I still have this problem. I checked the syntax and it looked fine, so I started hashing out lines one by one in root and user crontabs, to see which one was causing that.
I hashed them all out, and I still get the same error:

# systemctl status cronwhip
cronwhip.service - Run missed cronjobs with cronwhip.
   Loaded: loaded (/usr/lib/systemd/system/cronwhip.service; enabled)
   Active: failed (Result: exit-code) since Mon 2014-01-06 14:01:52 CET; 3min 47s ago
  Process: 386 ExecStart=/usr/bin/cronwhip (code=exited, status=1/FAILURE)
 Main PID: 386 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/cronwhip.service

Jan 06 14:01:52 panzor cronwhip[386]: error: invalid interval [2014-01-06 12:58, 2014-01-06 12:58]

I am sorry if I'm being thick, but I really don't get it.

Xyne · 2014-01-06 21:53:47

Cronwhip runs missed cronjobs in a given interval. By default it tries to use the interval of the last shutdown reported by last -x to run jobs that would have been run when the system was off. For whatever reason, the interval detected on your machine is 0 (same start and end times in the last line). An empty interval makes no sense so it reports and error.

I do not know why it fails to detect the interval on your machine. last -x reports shutdown times recorded in /var/log/wtmp (and numerically suffixed rotated files, e.g. /var/log/wtmp.1). If those are missing or last -x fails to report the last shutdown time then it won't work. Look for shutdown intervals in the output of last -x in a console. The lines begin with

shutdown system down

followed by the kernel version, the time the system was shut down, the time the system was rebooted, and the elapsed interval. Post the output if you are unsure.

You can run the following script to check the interval detected by cronwhip:

#!/usr/bin/env python3
from cronwhip import *

start, end = get_last_shutdown_interval_from_wtmp()
print(time.strftime(TIME_DISPLAY_FMT, start))
print(time.strftime(TIME_DISPLAY_FMT, end))

If the interval reported there differs from the interval reported by last -x then there is a bug in cronwhip.

I've just had an idea. If the output of last -x is localized (i.e. not in English) then the output parser will fail. If that is the case, then please try this script and let me know if it reports the correct interval:

#!/usr/bin/env python3

import locale

locale.setlocale(locale.LC_ALL, 'C')

from cronwhip import *

start, end = get_last_shutdown_interval_from_wtmp()
print(time.strftime(TIME_DISPLAY_FMT, start))
print(time.strftime(TIME_DISPLAY_FMT, end))

Pre-emptive strike: to run the scripts, save them as text files named "test1.py" and "test2.py", then run them with "python3 test1.py" and "python3 test2.py".

Lockheed · 2014-01-07 09:35:16

This is how it looks:

reboot   system boot  3.12.3-pf        Tue Jan  7 10:26   still running
shutdown system down  3.12.3-pf        Tue Jan  7 10:25 - 10:26  (00:00)

and this is cronwhip error for this instance:

# systemctl status cronwhip
cronwhip.service - Run missed cronjobs with cronwhip.
   Loaded: loaded (/usr/lib/systemd/system/cronwhip.service; enabled)
   Active: failed (Result: exit-code) since Tue 2014-01-07 10:26:27 CET; 2min 2s ago
  Process: 383 ExecStart=/usr/bin/cronwhip (code=exited, status=1/FAILURE)
 Main PID: 383 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/cronwhip.service

Jan 07 10:26:27 panzor cronwhip[383]: error: invalid interval [2014-01-07 10:25, 2014-01-07 10:25]
# cronwhip -v --show-missed
error: invalid interval [2014-01-07 10:25, 2014-01-07 10:25]

$ ./cronwhipscript 
2014-01-07 10:25
2014-01-07 10:25

( I just made the script executable and run them as above )

Xyne · 2014-01-08 20:22:58

Lockheed wrote:

This is how it looks:

reboot   system boot  3.12.3-pf        Tue Jan  7 10:26   still running
shutdown system down  3.12.3-pf        Tue Jan  7 10:25 - 10:26  (00:00)

So, according to that, the system was down for 0 minutes. If there is no shutdown interval, there is nothing to run. What do you expect cronwhip to do? Do you want it to accept null intervals without throwing an error?

Lockheed · 2014-01-08 20:49:58

Which means that if the machine has a quick reboot, and shutsdown and starts within the same minute, cron will throw out an error. Error, that will, however, not affect its proper operation.

Xyne · 2014-01-08 21:33:53

I have updated cronwhip to display a warning instead of exiting with an error when the interval is invalid. It will be up on my site and the AUR in a few minutes.

Lockheed · 2014-01-08 21:35:18

Ok, thank you for your work.

Arch Linux

#76 2013-11-28 23:42:50

Re: Cronwhip - no more missed cronjobs

#77 2013-11-29 00:02:57

Re: Cronwhip - no more missed cronjobs

#78 2013-11-29 06:09:54

Re: Cronwhip - no more missed cronjobs

#79 2013-11-30 17:34:14

Re: Cronwhip - no more missed cronjobs

#80 2013-11-30 19:25:12

Re: Cronwhip - no more missed cronjobs

#81 2013-12-01 17:56:56

Re: Cronwhip - no more missed cronjobs

#82 2013-12-02 00:55:49

Re: Cronwhip - no more missed cronjobs

#83 2013-12-03 00:30:32

Re: Cronwhip - no more missed cronjobs

#84 2013-12-03 23:27:35

Re: Cronwhip - no more missed cronjobs

#85 2013-12-04 13:34:25

Re: Cronwhip - no more missed cronjobs

#86 2013-12-04 17:47:19

Re: Cronwhip - no more missed cronjobs

#87 2013-12-11 10:03:22

Re: Cronwhip - no more missed cronjobs

#88 2013-12-11 17:35:50

Re: Cronwhip - no more missed cronjobs

#89 2013-12-12 22:55:12

Re: Cronwhip - no more missed cronjobs

#90 2013-12-12 23:07:34

Re: Cronwhip - no more missed cronjobs

#91 2013-12-13 06:17:31

Re: Cronwhip - no more missed cronjobs

#92 2013-12-13 22:51:53

Re: Cronwhip - no more missed cronjobs

#93 2013-12-13 22:57:42

Re: Cronwhip - no more missed cronjobs

#94 2014-01-06 13:06:44

Re: Cronwhip - no more missed cronjobs

#95 2014-01-06 21:53:47

Re: Cronwhip - no more missed cronjobs

#96 2014-01-07 09:35:16

Re: Cronwhip - no more missed cronjobs

#97 2014-01-08 20:22:58

Re: Cronwhip - no more missed cronjobs

#98 2014-01-08 20:49:58

Re: Cronwhip - no more missed cronjobs

#99 2014-01-08 21:33:53

Re: Cronwhip - no more missed cronjobs

#100 2014-01-08 21:35:18

Re: Cronwhip - no more missed cronjobs

Board footer