Docker containers see config file permission change after editor 'atomic save' — bind-mounted file temporarily unreadable until container restart

Environment

Ubuntu 22.04 LTS host running Docker Engine 24.0.2, containers running nginx:stable with a host bind mount /etc/nginx/conf.d/site.conf -> /etc/nginx/conf.d/site.conf inside container. Host edits done via VS Code's default save (atomic write) plugin which writes to a temp file then renames.

Symptoms

After saving the configuration file from the host editor, nginx inside the container reports 'open() "/etc/nginx/conf.d/site.conf" failed (2: No such file or directory)' and serves a default page until the container is restarted or the file is re-chmod/chown from within the container. The host file exists with correct content and permissions immediately after save. Container sees a different inode; some processes show 'file not found' until manual in-container operations are performed.

Observations

On the host, ls -li /etc/nginx/conf.d/site.conf shows inode A before edit, and inode B after save (atomic rename). Inside container, /etc/nginx/conf.d/site.conf may still reference inode A but the file path points to inode B on the host FS; nginx opened old FD and failed because it accessed via opened path? strace inside container shows attempts to open path and receiving ENOENT intermittently. Mount is a bind mount from ext4 host into overlayFS used by Docker; journalctl shows kernel warnings about 'overlayfs: revalidate: lower inode changed' around the time of edit. Restarting nginx inside container sometimes fixes, but not reliably without restarting the container.

Already attempted

Replaced editor save behavior to non-atomic (disabled atomic saves in VS Code) — this reduced occurrences but did not eliminate the issue. Tried using 'docker cp' to copy file into container which works consistently. Upgraded Docker Engine to latest stable, restart docker daemon; issue still reproducible. Verified host permissions and SELinux/AppArmor not enforcing (AppArmor in complain mode).

Question

Why does an atomic rename on the host break a bind-mounted file inside an overlay-backed container, and what reliable, reversible changes to host editor, Docker mount flags, or container configuration avoid breaking processes that expect the file to be continuously present?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 1w ago• Trouble ID

Most likely causes

Atomic replace (rename) creates a new inode on the host; overlayfs/backing filesystem semantics cause the container view to be temporarily inconsistent, yielding ENOENT for some processes. 95%

Container process opens file path while overlayfs is updating layers; race between rename and mount namespace view update produces transient file-not-found errors. 90%

Docker bind mount with overlay/overlay2 has known edge cases with host-side atomic replacements leading to container-side caching/inode mismatch. 85%

Summary

Atomic editor saves (write-to-temp + rename) replace the inode under the bind mount; with overlayfs or Docker's use of different mount namespaces, the container may observe inconsistent filesystem state until VFS and overlay refresh occur. Some containerized processes may have race conditions where they try to open the file at a moment when the path exists but lower/upper layer behavior causes ENOENT. Kernel messages about inode changes suggest overlay/upper-layer invalidation is involved. Disabling atomic saves mitigated but didn't fully eliminate; robust fixes involve ensuring in-place writes or using mechanisms that update config via volume semantics containers expect (e.g., copy into container, use docker config/secrets, or use non-overlay mounts).

Suggested Fixes

Diagnostic step 1

Avoid atomic replace for files that are bind-mounted into containers: configure your editor to write in-place (disable atomic save) or use editor plugins that update file contents without renaming. This is reversible by re-enabling atomic saves.

Diagnostic step 2

Use Docker-managed mechanisms for configuration delivery: mount a host directory and update files in-place, use docker config (for swarm) or copy updated files into the container via 'docker cp' followed by a SIGHUP to service process. These approaches avoid replacing the inode under the container's view.

Diagnostic step 3

If in-place updates are not possible, trigger a graceful reload inside the container after host file change (e.g., watch the host file with inotify and run 'nginx -s reload' or 'systemctl reload' in the container) so the process re-opens the new file. Implement a small supervisor script inside the container to handle this; keep changes reversible.