hwclock constantly resets itself

Altay_H · 2010-02-14 20:25:22

I've been having an issue with my hwclock time. It advances forward correctly, but randomly resets itself to a seemingly arbitrary time. It's not slow or fast, it just randomly jumps backward.

I ran a script overnight that uses hwclock and date to output the time every 30 minutes. The output is below. Notice that both times are perfectly in sync for the first half hour, then the BIOS clock time jumps backwards.

Sat 13 Feb 2010 09:10:17 PM EST  -0.569935 seconds
Sat Feb 13 21:10:17 EST 2010

Sat 13 Feb 2010 09:40:18 PM EST  -0.985086 seconds
Sat Feb 13 21:40:18 EST 2010

Sat 13 Feb 2010 09:20:13 PM EST  -0.005284 seconds
Sat Feb 13 22:10:18 EST 2010

Sat 13 Feb 2010 09:50:13 PM EST  -0.048741 seconds
Sat Feb 13 22:40:18 EST 2010

Sat 13 Feb 2010 09:16:00 PM EST  -0.069590 seconds
Sat Feb 13 23:10:18 EST 2010

Sat 13 Feb 2010 09:46:00 PM EST  -0.019067 seconds
Sat Feb 13 23:40:18 EST 2010

Sat 13 Feb 2010 09:11:50 PM EST  -0.969612 seconds
Sun Feb 14 00:10:19 EST 2010

Sat 13 Feb 2010 09:41:50 PM EST  -0.058574 seconds
Sun Feb 14 00:40:19 EST 2010

Sat 13 Feb 2010 09:07:38 PM EST  -0.028973 seconds
Sun Feb 14 01:10:19 EST 2010

Sat 13 Feb 2010 09:37:38 PM EST  -0.043397 seconds
Sun Feb 14 01:40:19 EST 2010

Sat 13 Feb 2010 09:03:26 PM EST  -0.984740 seconds
Sun Feb 14 02:10:20 EST 2010

Sat 13 Feb 2010 09:33:26 PM EST  -0.099995 seconds
Sun Feb 14 02:40:20 EST 2010

Sat 13 Feb 2010 08:59:16 PM EST  -0.971924 seconds
Sun Feb 14 03:10:21 EST 2010

Sat 13 Feb 2010 09:29:17 PM EST  -0.995066 seconds
Sun Feb 14 03:40:22 EST 2010

Sat 13 Feb 2010 08:55:05 PM EST  -0.009355 seconds
Sun Feb 14 04:10:22 EST 2010

Sat 13 Feb 2010 09:25:05 PM EST  -0.067611 seconds
Sun Feb 14 04:40:22 EST 2010

Sat 13 Feb 2010 08:50:53 PM EST  -0.029792 seconds
Sun Feb 14 05:10:22 EST 2010

Sat 13 Feb 2010 09:20:54 PM EST  -0.990515 seconds
Sun Feb 14 05:40:23 EST 2010

Sat 13 Feb 2010 08:46:42 PM EST  -0.015870 seconds
Sun Feb 14 06:10:23 EST 2010

Sat 13 Feb 2010 09:16:42 PM EST  -0.083583 seconds
Sun Feb 14 06:40:23 EST 2010

Sat 13 Feb 2010 08:42:30 PM EST  -0.009090 seconds
Sun Feb 14 07:10:23 EST 2010

Sat 13 Feb 2010 09:12:30 PM EST  -0.023033 seconds
Sun Feb 14 07:40:23 EST 2010

Sat 13 Feb 2010 08:38:19 PM EST  -0.970880 seconds
Sun Feb 14 08:10:24 EST 2010

Sat 13 Feb 2010 09:08:19 PM EST  -0.108766 seconds
Sun Feb 14 08:40:25 EST 2010

Sat 13 Feb 2010 08:34:07 PM EST  -0.043792 seconds
Sun Feb 14 09:10:25 EST 2010

Sat 13 Feb 2010 09:04:08 PM EST  -0.985132 seconds
Sun Feb 14 09:40:26 EST 2010

Sat 13 Feb 2010 08:29:56 PM EST  -0.030727 seconds
Sun Feb 14 10:10:26 EST 2010

Sat 13 Feb 2010 08:59:57 PM EST  -0.513275 seconds
Sun Feb 14 10:40:27 EST 2010

Sat 13 Feb 2010 08:25:45 PM EST  -0.036977 seconds
Sun Feb 14 11:10:27 EST 2010

Sat 13 Feb 2010 08:55:45 PM EST  -0.033182 seconds
Sun Feb 14 11:40:27 EST 2010

Sat 13 Feb 2010 08:21:34 PM EST  -0.038648 seconds
Sun Feb 14 12:10:27 EST 2010

Sat 13 Feb 2010 08:51:34 PM EST  -0.009488 seconds
Sun Feb 14 12:40:27 EST 2010

Sat 13 Feb 2010 08:17:22 PM EST  -0.034714 seconds
Sun Feb 14 13:10:27 EST 2010

Sat 13 Feb 2010 08:47:22 PM EST  -0.106170 seconds
Sun Feb 14 13:40:27 EST 2010

Sat 13 Feb 2010 08:13:11 PM EST  -0.936454 seconds
Sun Feb 14 14:10:28 EST 2010

Sat 13 Feb 2010 08:43:12 PM EST  -0.976617 seconds
Sun Feb 14 14:40:29 EST 2010

Has anyone else ever had this problem? Any ideas?

ewaller · 2010-02-15 02:19:11

I have not seen the problem, but a hardware problem springs to my mind.
How does your system establish the time of day at boot? Do you depend on the hardware clock or do you get the time from an external source (NTP, GPS, etc..)
Is the hardware clock correct at boot time? (Cold boot? Warm Boot?)
Once it is off, does the hardware clock ever come up with the correct time unless it is reset?

Altay_H · 2010-02-15 04:43:31

My system initially uses the hardware clock to set the time. One minute after booting [to ensure a network connection] openntpd starts up and corrects the system time, but leaves the hardware clock time unaffected. I'll investigate further, but I haven't noticed any problem with the hardware clock when the system is off. The hardware clock is only off significantly when the system remains on for a long time, not off.

If you look closely at the output of my script you'll notice that the hardware clock seems to be resetting about once every hour, and appears to be gradually working its way further into the past. I'll leave my computer off all day tomorrow and see how accurately the hardware clock keeps time on its own.

Altay_H · 2010-02-16 04:57:22

My hardware clock was accurate today after leaving my computer off all day; however when I booted up Arch linux the hardware clock time was reset back to this morning. Here's what's happening:
1. While the computer is off the hardware clock is accurate
2. When Arch boots up, the hardware clock is reset to an incorrect time and the system time is set to the same incorrect time
3. One minute after booting up openntpd starts up and corrects the system time, but leaves the hardware clock incorrect
4. Once every hour or so the hardware clock is reset to an incorrect time

I'm not entirely sure about step 4 since the reseting of the hardware clock is somewhat arbitrary. I'll try to get more accurate results tonight. Also, I've noticed that the time to which the hardware clock is reset roughly corresponds to the last time the system was shut down. I'll need to experiment further to verify this.

Any ideas as to what might cause Arch to continue "correcting" the hardware clock time?

Raqua · 2010-02-16 07:00:35

I am having very similar issue. I did not check it the way as OP while system running, nut that appears to be fine. My clock seems to randomly leap backwards with reboots. I have Ubuntu and Arch on the same machine. Ubuntu seems to correct the clock problem, so I disabled its ntp daemon start. Still, when booting to Ubuntu, all is fine, bios time is ok when system starts, but when I boot to Arch, time is off.
I already changed motherboard battery, it is new.

Altay_H · 2010-02-16 14:55:37

Raqua wrote:

My clock seems to randomly leap backwards with reboots. I have Ubuntu and Arch on the same machine. Ubuntu seems to correct the clock problem, so I disabled its ntp daemon start. Still, when booting to Ubuntu, all is fine, bios time is ok when system starts, but when I boot to Arch, time is off.

Do you have your HARWARECLOCK in rc.conf set to localtime or UTC? If you're dual-booting you'll probably want to set it to UTC. Ubuntu uses UTC by default, and if Arch is using localtime it'll cause problems.

I re-ran my script overnight, this time having it output every minute. I've trimmed the output so only the relevant portions remain. Initially the hardware clock is jumping back a little more than an hour every hour or so. Over the course of six hours it worked its way 30 minutes back in time.

Tue 16 Feb 2010 12:21:07 AM EST  -0.334209 seconds
Tue Feb 16 00:21:07 EST 2010
[snip]
Tue 16 Feb 2010 12:48:33 AM EST  -0.003652 seconds
Tue Feb 16 00:48:32 EST 2010

Tue 16 Feb 2010 12:18:22 AM EST  -0.004375 seconds
Tue Feb 16 00:49:32 EST 2010
[snip]
Tue 16 Feb 2010 01:17:44 AM EST  -0.004384 seconds
Tue Feb 16 01:48:54 EST 2010

Tue 16 Feb 2010 12:14:33 AM EST  -0.004463 seconds
Tue Feb 16 01:49:54 EST 2010
[snip]
Tue 16 Feb 2010 01:12:56 AM EST  -0.004148 seconds
Tue Feb 16 02:48:18 EST 2010

Tue 16 Feb 2010 12:09:44 AM EST  -0.004364 seconds
Tue Feb 16 02:49:18 EST 2010
[snip]
Tue 16 Feb 2010 01:09:06 AM EST  -0.974110 seconds
Tue Feb 16 03:48:40 EST 2010

Tue 16 Feb 2010 12:05:55 AM EST  -0.974144 seconds
Tue Feb 16 03:49:41 EST 2010
[snip]
Tue 16 Feb 2010 01:05:11 AM EST  -0.004451 seconds
Tue Feb 16 04:48:57 EST 2010

Tue 16 Feb 2010 12:02:00 AM EST  -0.004459 seconds
Tue Feb 16 04:49:57 EST 2010
[snip]
Tue 16 Feb 2010 01:00:40 AM EST  -0.974195 seconds
Tue Feb 16 05:48:37 EST 2010

Mon 15 Feb 2010 11:57:28 PM EST  -0.974220 seconds
Tue Feb 16 05:49:38 EST 2010
[snip]
Tue 16 Feb 2010 12:56:46 AM EST  -0.974234 seconds
Tue Feb 16 06:48:56 EST 2010

Mon 15 Feb 2010 11:53:36 PM EST  -0.974230 seconds
Tue Feb 16 06:49:57 EST 2010

I'm still open to any ideas.

Raqua · 2010-02-16 15:03:12

Honestly, I do not remember, what do I have set, but I guess it is UTC.
However, I have had this setup running fine for more than 6 months, this problem started just recently.

I run a check like you have, but my clock seems to be fine while computer is booted. They reset on reboot.

Altay_H · 2010-03-19 00:24:17

I suspected that openntpd might be causing the problem, so I removed it from my rc.local to prevent it from starting up. Unfortunately the only difference this made was that now neither of my clocks had the right time, so I added it back into rc.local. This must be a problem with the software that controls the hardware clock. It's either hwclock itself, or some other low-level program that has the right to mess with the system clock every hour.

I ran my script overnight again, and got some more detailed results:
- The hardware clock sets itself back 63 minutes every 60 minutes.

Here's my data:

Wed 17 Feb 2010 05:08:27 AM EST  -0.873978 seconds
Wed Feb 17 23:30:15 EST 2010
[snip]
Wed 17 Feb 2010 06:07:25 AM EST  -0.964210 seconds
Thu Feb 18 00:29:13 EST 2010

Wed 17 Feb 2010 05:04:14 AM EST  -0.964265 seconds
Thu Feb 18 00:30:14 EST 2010
[snip]
Wed 17 Feb 2010 06:03:12 AM EST  -0.994805 seconds
Thu Feb 18 01:29:12 EST 2010

Wed 17 Feb 2010 05:00:01 AM EST  -0.994798 seconds
Thu Feb 18 01:30:13 EST 2010
[snip]
Wed 17 Feb 2010 05:58:59 AM EST  -0.994801 seconds
Thu Feb 18 02:29:11 EST 2010

Wed 17 Feb 2010 04:55:49 AM EST  -0.994812 seconds
Thu Feb 18 02:30:12 EST 2010
[snip]
Wed 17 Feb 2010 05:54:47 AM EST  -0.994850 seconds
Thu Feb 18 03:29:11 EST 2010

Wed 17 Feb 2010 04:51:36 AM EST  -0.994123 seconds
Thu Feb 18 03:30:12 EST 2010
[snip]
Wed 17 Feb 2010 05:50:34 AM EST  -0.990383 seconds
Thu Feb 18 04:29:10 EST 2010

Wed 17 Feb 2010 04:47:23 AM EST  -0.964350 seconds
Thu Feb 18 04:30:11 EST 2010
[snip]
Wed 17 Feb 2010 05:46:21 AM EST  -0.994849 seconds
Thu Feb 18 05:29:09 EST 2010

Wed 17 Feb 2010 04:43:10 AM EST  -0.994834 seconds
Thu Feb 18 05:30:10 EST 2010
[snip]
Wed 17 Feb 2010 05:42:08 AM EST  -0.994854 seconds
Thu Feb 18 06:29:08 EST 2010

Wed 17 Feb 2010 04:38:57 AM EST  -0.994899 seconds
Thu Feb 18 06:30:09 EST 2010

Does anyone have any idea what could be doing this?

It's an incredibly hackish workaround, but I suppose I could try putting "hwclock -u --systohc" in an hourly cron job as root. I'll post a follow-up if I give it a try.

EDIT:
Since this is an hourly problem, it occurred to me that something in root's hourly cron could be the culprit. The only thing there is adjtime, which I believe is part of openntpd's setup. I searched the wiki for adjtime, which led me to discover that I have HARDWARECLOCK="UTC" in rc.conf. I tried changing it to HARDWARECLOCK="ntp" and it looks like it may have fixed the problem. I'll mark this thread as [SOLVED] when I'm sure it's fixed.

Last edited by Altay_H (2010-03-19 00:30:30)

thestinger · 2010-03-19 03:21:27

there's a cronjob called adjtime in /etc/cron.hourly that controls the hwclock - setting HARDWARECLOCK to ntp just stops it from doing anything

openNTPD doesn't deal with clock drift so you can use the ntp daemon if you want that functionality

theapodan · 2010-03-19 03:28:58

This wiki page mentions your exact problem, read here:

http://wiki.archlinux.org/index.php/Time#Time_Skew

In particular this section:

An Arch Linux cron job script adjusts the hardware clock according to the recorded drift every hour. If you are seeing large changes to your hardware clock, it is likely because a large drift is recorded in /var/lib/hwclock/adjtime. This can happen if you have set the hardware clock time incorrectly, or your time standard is not syncronized with a Windows or Mac OS install. To fix this remove /var/lib/hwclock/adjtime, set the correct hardware clock and system clock time, and check if your time standard is correct.

Altay_H · 2010-03-19 03:46:57

thestinger wrote:

openNTPD doesn't deal with clock drift so you can use the ntp daemon if you want that functionality

Doesn't openNTPD synchronize the system clock with a server? Wouldn't any drift be fixed with each synchronization?

theapodan wrote:

This wiki page mentions your exact problem, read here:
http://wiki.archlinux.org/index.php/Time#Time_Skew

I don't understand why adjtime would continue to set the clock back an hour every hour. Shouldn't it stop adjusting the time significantly once it sets the time to what it thinks is correct? And surely it's not possible to drift more than an hour every hour unless time is running in reverse.

Moreover, why would adjtime set the time incorrectly at all if it knows the hardware clock is using UTC? The hardware clock is set to the correct UTC time, so how does openNTPD cause adjtime to think the hardware clock is incorrect?

Thank you for your help.

thestinger · 2010-03-19 17:27:20

clock drift is the skew of the hardware clock itself - hwclock is supposed to calculate the drift and then set the clock back or forwards every hour to compensate.

if it calculated the drift completely wrong (like if you boot into windows and it changes the hwclock from UTC to localtime) then it will screw up the clock every hour

openNTPD just syncs the time - ntpd also calculates drift between syncs in order to compensate for drift as it happens (or after a reboot)

Arch Linux

#1 2010-02-14 20:25:22

hwclock constantly resets itself

#2 2010-02-15 02:19:11

Re: hwclock constantly resets itself

#3 2010-02-15 04:43:31

Re: hwclock constantly resets itself

#4 2010-02-16 04:57:22

Re: hwclock constantly resets itself

#5 2010-02-16 07:00:35

Re: hwclock constantly resets itself

#6 2010-02-16 14:55:37

Re: hwclock constantly resets itself

#7 2010-02-16 15:03:12

Re: hwclock constantly resets itself

#8 2010-03-19 00:24:17

Re: hwclock constantly resets itself

#9 2010-03-19 03:21:27

Re: hwclock constantly resets itself

#10 2010-03-19 03:28:58

Re: hwclock constantly resets itself

#11 2010-03-19 03:46:57

Re: hwclock constantly resets itself

#12 2010-03-19 17:27:20

Re: hwclock constantly resets itself

Board footer