Ubuntu VMs jump 60–120 seconds ahead after live migration; systemd‑timesyncd reports large step adjustments

Environment

Ubuntu 22.04 LTS VMs running on KVM/QEMU hosts managed by libvirt; live migrations performed between hosts with chrony on hosts but VMs configured to use systemd‑timesyncd (default) and no guest chrony/ntpd installed.

Symptoms

Shortly after a live migration, the VM's clock jumps forward by 60–120 seconds. Applications relying on time (cron jobs, TLS certificate checks) experience failures or run prematurely. Jump happens only after migrations, not during normal reboot.

Observations

journalctl shows systemd‑timesyncd logging 'Clock step was XXs' and then adjusting time. The VM's hwclock (rtc) remains at the pre‑migration value. Host clocks are synchronized with chrony and show no similar jumps. Disabling systemd‑timesyncd on the guest prevents the jump; installing chrony in the guest also prevents the problem. No host‑level live migration failures or guest panic recorded.

Already attempted

Restarted systemd‑timesyncd service, installed chrony as an experiment on a test VM (stopped timesyncd and installed chrony — solved jumps), adjusted timesyncd configuration to fewer peers (no effect), ensured host and guest do not share RTC misconfiguration. Did not modify hypervisor kernel or QEMU args in production.

Question

Why does systemd‑timesyncd step the clock forward after live migration, what safely reversible configuration changes will prevent it, and what long‑term solution is recommended for virtualized Ubuntu guests?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 1w ago• Trouble ID

Most likely causes

Guest lacks a robust time sync daemon (chrony/ntpd) that tolerates live migration 95%

systemd‑timesyncd's default step threshold triggers an immediate clock step after measuring migration offset 90%

Paravirtual clock (kvmclock) and host clocks slightly out of sync at migration boundary causing guest clock delta 80%

Summary

Live migration can introduce discontinuities between the guest's virtual clock sources (kvmclock, paravirtualized timers) and the host timebase. systemd‑timesyncd is conservative about correcting monotonic time but will step the system clock when the measured offset exceeds its threshold; this often happens after migration because the guest's time source is reset or the host reports a slightly advanced time. Installing a proper NTP client like chrony inside the guest avoids abrupt steps because chrony can slewrate large offsets gradually or step only when safe. The underlying cause is time source handover during live migration combined with systemd‑timesyncd's step policy.

Suggested Fixes

Diagnostic step 1

For immediate mitigation, stop systemd‑timesyncd on problematic guests and install and configure chrony to manage time inside the guest; chrony handles slewing and large offsets more gracefully in virtualized environments.

Diagnostic step 2

If cron job timing is critical, configure cron to use monotonic timers where possible or add guard logic to avoid double execution after migrations.

Diagnostic step 3

Collect vm and host journal entries around migration times, and compare /proc/driver/rtc and dmesg before and after migration to see which time sources changed; preserve logs for virtualization/admin escalation if necessary.