1
Diagnostic step 1
Check system logs (kern.log, dmesg) on master for OOM killer or process signals at the time of BGSAVE termination (look for 'Out of memory' or SIGKILL)
FULL AI ANALYSIS
Redis replica stops applying updates after RDB mismatch during background save (BGSAVE)
Generated Oct 3, 2026, 12:35:03 AM
Analysis summary
A checksum mismatch on the RDB indicates the file written by the master's child process (spawned by BGSAVE) was truncated or corrupted before completion. Common causes include the BGSAVE child being terminated by signals (OOM-killer), interference by filesystem-level actions (snapshots, backup tools) operating on the same file, or disk I/O errors during the write. When the master creates a truncated dump.rdb, replicas attempting to do a full sync will detect the mismatch and fail to load the RDB, requiring manual intervention. The correlation with LVM snapshot activity suggests filesystem-level operations may be conflicting with Redis' child writes.
Possible causes
Concurrent filesystem operations (LVM snapshots, backup agents) interfering with the atomic rename pattern Redis uses for dump.rdb producing incomplete/truncated file
95%BGSAVE child killed by OOM-killer or by administrator/system script during heavy memory/disk activity
60%Disk or filesystem errors causing partially written dump.rdb (e.g., mounted with options that interact poorly with atomic replacements)
40%Insufficient ulimits or system-level process limits causing child termination during heavy snapshot write
15%Suggested fixes
2
Diagnostic step 2
Schedule LVM snapshots or backup jobs to avoid overlapping with Redis BGSAVE: either pause snapshots during backup window or configure snapshots to operate on a different file/volume; test BGSAVE repeatedly while snapshot disabled to confirm correlation
3
Diagnostic step 3
Enable Redis RDB and AOF coexistence temporarily (appendonly yes) as a mitigation so the replica can replay AOF if RDB load fails; note AOF has different performance characteristics—test in non-production first
4
Diagnostic step 4
Increase Redis child write reliability: ensure stable disk with journal enabled (avoid mounting with 'nodelalloc' if that causes issues), raise ulimits for Redis process, and ensure adequate free memory to avoid OOM during snapshot
5
Diagnostic step 5
If filesystem-level interference suspected, move Redis persistence to a dedicated block device/partition that snapshots do not touch, or coordinate snapshot timing with Redis maintainers/ops team; if frequent corruption persists, escalate to storage team and plan for controlled failover testing before making persistent changes.