Cron jobs consistently missed after system clock steps backwards due to NTP slewing

Environment

Ubuntu Server 22.04 systems using chrony (chronyd) to sync time to an internal NTP server. Servers run multiple cron jobs scheduled at precise minutes (system crontab and root's crontab). Recently observed during a network outage when clock update slewed time back 4–6 minutes.

Symptoms

Daily maintenance cron jobs scheduled for specific times did not run for the affected hosts when the system clock was slewed backwards by several minutes. The jobs are not executed later (jobs are not backfilled) and there are no cron entries indicating retries.

Observations

Syslog shows chronyd correcting clock by slewing rather than stepping; timestamp of the slewing event precedes the expected cron invocation time. Cron's logs show no attempt to run the jobs at the original scheduled times, and manual invocation runs succeed. systemd timers with Persistent=true were not in use for those tasks. The problem occurred consistently on multiple hosts during the same NTP event.

Already attempted

Restarted the cron daemon, manually ran the missed jobs, changed chrony Configuration to prefer stepping for corrections > 1s using makestep in a test host, and converted one job to a systemd timer with Persistent=true which ran fine after the next time correction.

Question

Why does cron skip jobs when the system clock moves backwards, and what reversible configuration changes should I make to ensure scheduled jobs run reliably across time corrections without risking large sudden system time steps?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 1w ago• Trouble ID

Most likely causes

Cron's scheduling logic evaluates 'next runtime' against the adjusted wall clock and doesn't backfill jobs when time moves backward 90%

Large slewing operations change the wall clock across the scheduled instant so cron does not see the job slot as expired/runnable 85%

Use of cron for jobs that must not be skipped across time corrections rather than using persistent systemd timers or at-job tracking 80%

Summary

Traditional cron uses wall-clock time to determine when jobs should run; when the kernel clock moves backwards, cron's internal next-run calculation can result in the scheduled time being in the past relative to the adjusted clock and thus skipped. systemd timers offer better guarantees (Persistent=true) because they record missed runs and trigger on boot. For time corrections, choosing between slewing and stepping affects whether tasks miss runs: slewing slowly shifts time and can still produce missed events depending on timing. Ensuring jobs are resilient to clock adjustments is the safer approach.

Suggested Fixes

Diagnostic step 1

Convert critical scheduled tasks to systemd timers with Persistent=true so missed runs due to clock jumps are executed when the system stabilizes.

Diagnostic step 2

Configure chrony to step the clock for large corrections during maintenance windows (use makestep for corrections larger than a threshold) or set acceptable precision limits to reduce large slews during production hours.

Diagnostic step 3

For cron-managed jobs that must not be skipped, add an idempotent wrapper that checks if expected outputs exist for a recent interval and runs the job if missing (this preserves reversibility and avoids changing system time behavior).