1
Diagnostic step 1
Re-enable TCP MSS clamping on the IPsec endpoint(s) to mitigate immediately (reversible change).
FULL AI ANALYSIS
Site-to-site IPsec tunnel stalls large TCP transfers after disabling MSS clamp on edge firewall
Generated Oct 13, 2026, 9:45:52 AM
Analysis summary
This is a classic Path MTU Discovery (PMTUD) issue combined with IPsec/NAT encapsulation reducing the effective path MTU. Encapsulated packets exceed the actual path MTU; because DF is set, routers need to return ICMP 'fragmentation needed' messages to allow endpoints to reduce segment size. The edge/firewall was dropping those ICMP messages, so PMTUD failed and TCP flows transmitted packets that never got through (black hole). MSS clamping works around this by adjusting the TCP handshake MSS to a safe value, preventing large segments that would trigger fragmentation. NAT/encapsulation overhead and ICMP filtering are the main contributors. Hardware offload/acceleration bugs on some firewalls also sometimes mishandle DF packets inside IPsec tunnels and can mimic this behavior.
Possible causes
Firewall globally blocks ICMP Type 3 Code 4 (Fragmentation needed), preventing PMTUD from working.
100%MSS/TCP window clamping disabled on edge device; endpoints negotiate too-large MSS during TCP handshake.
95%IPsec + NAT overhead reduces effective path MTU but endpoints still send full-size segments (DF set).
90%Hardware acceleration or buggy packet offload on firewall drops or mangles ICMP or large DF packets inside IPsec ESP packets.
5%Suggested fixes
2
Diagnostic step 2
Temporarily allow ICMP Type 3 Code 4 through the relevant firewall policies between the two sites and re-test PMTUD behavior; remove overly broad ICMP drops rather than allowing all ICMP.
3
Diagnostic step 3
Use tracepath or ping -M do -s to probe PMTU and verify endpoints can receive ICMP Fragmentation Needed messages end-to-end.
4
Diagnostic step 4
If using NAT over IPsec, account for encapsulation overhead: either set MSS clamp to 1450 (or lower depending on measured overhead) or lower MTU on tunnel endpoints only (avoid changing every host).
5
Diagnostic step 5
Disable hardware offload or bypass acceleration on the firewall temporarily and re-test to rule out NIC/ASIC fragmentation bugs; if confirmed, plan firmware/driver updates with vendor guidance before re-enabling offload in production. If the firewall is closed-source or you suspect a firmware bug, collect packet captures (inside and outside the tunnel) and escalate to vendor support with capture timestamps and config snippets.