IPv6-only backend reachable but client connections intermittently reset by NAT64 gateway

Environment

On-prem edge router running OpenWrt 22.03 acting as NAT64 (tayga) gateway for IPv4-only clients; backend services are native IPv6 on Debian 12 servers (nginx, TLS). Clients are multiple residential ISPs that are IPv4-only.

Symptoms

Clients on IPv4-only ISPs intermittently see TCP RSTs when connecting to backend services via the NAT64 address (64:ff9b::/96). Short requests (small HTTP GETs) usually succeed; larger responses or long-lived connections frequently get torn down mid-transfer with a RST from the NAT64 gateway. The problem is intermittent and affects different clients at different times.

Observations

tcpdump on the NAT64 gateway shows the NAT64 translating IPv6 backend SYN-ACKs to IPv4 responses but in many failed cases the gateway later sends a TCP RST toward the IPv4 client. conntrack table shows many entries in TIME_WAIT and some entries being evicted near the timeouts. Firewall rules allow established connections. Reducing backend response size occasionally avoids the reset. No obvious kernel oopses in dmesg.

Already attempted

Restarted tayga and the edge router, increased net.netfilter.nf_conntrack_max, temporarily disabled firewall rules, disabled TCP timeouts tuning, disabled TCP offload (ethtool -K) on the gateway, and forced smaller MTU between gateway and backend. None of these eliminated RSTs; frequency decreased when MTU was reduced.

Question

What is causing the NAT64 gateway to issue TCP RSTs mid-connection, and which diagnostic steps and tunables should I apply to prevent resets while preserving IPv6 connectivity for IPv4-only clients?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 1w ago• Trouble ID

Most likely causes

PMTU / fragmentation issues (ICMPv6 'Packet Too Big' blocked between backend and gateway) causing retransmission behavior that leads gateway to reset connections 90%

conntrack table evictions or timeouts causing the gateway to no longer recognize an established session and closing it with RST 65%

a NAT64 implementation bug or race condition in tayga or kernel translation code under certain packet sizes 45%

Summary

Intermittent RSTs from a NAT64 gateway are commonly due to state tracking issues (conntrack evictions or timeouts), packet MTU/fragmentation problems leading to connection failure (especially if ICMP v6 'Packet Too Big' messages are blocked), or a bug/limitation in the NAT64 implementation (race when mapping block devices or handling large packets). TCP offload or checksum offload can interact poorly with translation and fragmentation. The fact that reducing MTU reduced resets points to an MTU/fragmentation-related path or PMTU discovery failure, but conntrack pressure and kernel-level implementation bugs remain plausible.

Suggested Fixes

Diagnostic step 1

Capture synchronized packet traces (tcpdump) on the IPv4 client-facing interface of the NAT64 gateway and on the IPv6 backend interface for the same flow when a failure occurs; include packet lengths, MSS, and observed TCP flags.

Diagnostic step 2

Monitor conntrack usage (conntrack -L / -S or /proc/net/nf_conntrack) while reproducing; increase nf_conntrack_max further and enable conntrack UDP/TCP timeout tuning to see if evictions correlate with failures.

Diagnostic step 3

Temporarily disable TSO/GSO/TSO6 on the gateway interfaces (ethtool -K <if> tso off gso off gro off) to rule out offload-induced corruption.