1
Diagnostic step 1
Collect diagnostic logs before making changes: capture dmesg, journalctl -u docker, and the exact overlayfs kernel messages (dmesg -T | grep overlay) to document the failure.
FULL AI ANALYSIS
After a kernel upgrade Docker containers get 'permission denied' writing to bind-mounted volumes (overlay2/overlayfs change)
Generated Oct 3, 2026, 12:35:03 AM
Analysis summary
A kernel update introduced a change in overlayfs or in user namespace / shiftfs handling that makes Docker's overlay2 with bind-mounted host directories fail to map UIDs correctly or enforce different permission checks. The fact that --privileged allows writes and that reverting the kernel fixes the problem points to a kernel-side regression/behavioral change rather than an application bug. Another possibility is a change in required mount options for overlayfs, or that user namespace remapping (userns) now requires shiftfs/explicit mapping which the current Docker version doesn't provide in your configuration.
Possible causes
Kernel regression or behavior change in overlayfs requiring different handling for user namespace remapping (shiftfs) leading to UID/GID mismatch enforcement.
88%A change in how the kernel enforces DAC/permissions for overlay-mounted bind mounts (stricter checks introduced).
85%Docker storage driver incompatibility with the new kernel (overlay2 expecting an older overlayfs ABI).
70%AppArmor or other LSM enforced differently under the new kernel (less likely given audit logs), or a mount option (e.g., noexec/nodev) implicitly applied to bind mounts.
10%Suggested fixes
2
Diagnostic step 2
Test with a minimal reproduce case: run an official lightweight image (alpine) and bind mount a new host directory to reproduce the EACCES; keep the test directory separate from production data.
3
Diagnostic step 3
If a regression is suspected, temporarily boot the previous working kernel to restore production and avoid data loss while performing deeper diagnostics.
4
Diagnostic step 4
Try running the container with --privileged in a controlled test to confirm the kernel/permission interaction; do not leave production containers privileged long-term.
5
Diagnostic step 5
Test Docker with an alternate storage driver (vfs) in a non-production environment to confirm that overlay2/overlayfs is the culprit. vfs is slower but useful diagnostically and reversible. If diagnostics point to a kernel regression (matching dmesg traces and behavior), open a bug with the kernel maintainers and Docker with your dmesg output, kernel version, Docker version, and a minimal reproduction case. For immediate mitigation, run the host on the last-working kernel and schedule coordinated upgrades: either upgrade Docker to a version that supports the newer kernel's overlayfs behavior or hold kernel updates until a fix is available. Escalate to your OS vendor if using a vendor-supplied kernel.