PostgreSQL primary WAL sender CPU spikes and replication lag after enabling frequent TLS renegotiation on streaming replication

Environment

PostgreSQL 13.7 primary on Debian 11 (8-core VM), three synchronous replicas on separate hosts. Streaming replication over TLS (server certs) through a TCP load balancer that enforces SSL session renegotiation every 60s. wal_level = replica, max_wal_senders = 10, synchronous_commit = on.

Symptoms

After configuring the load balancer for aggressive TLS renegotiation, WAL sender (walsender) CPU usage spikes from <5% to 80–100% during renegotiation events and secondaries lag by several seconds to minutes. Replication slot reports 'streaming' but standby replay falls behind. When the load balancer's renegotiation interval is increased or disabled, CPU and lag return to normal.

Observations

GDB/strace on the walsender processes during spikes shows repeated SSL handshake operations and blocking reads/writes. PostgreSQL logs include repeated 'terminating walsender due to administrative action' messages correlated with load balancer's SSL reconnects. Network captures show TCP sessions closed and new TLS handshakes every 60s. No network packet loss outside the deliberate renegotiation. CPU profiling indicates most time spent inside OpenSSL library crypto operations on the primary host.

Already attempted

Increased max_wal_senders and tuned wal_sender_timeout (no measurable improvement). Upgraded OpenSSL to latest patch in distro (no change). Temporarily bypassed the load balancer (direct IP TLS) — issue disappears. Tried enabling session resumption on the load balancer but sessions still invalidated by its policy.

Question

Why does frequent TLS renegotiation cause WAL sender CPU spikes and replication lag, and what safe configuration changes on server, load balancer, or PostgreSQL can mitigate the problem without removing TLS protection?

Post timeline

COMMUNITY RESPONSES

Verified community troubleshooting

0 replies

No answers yet..

Showing 0 of 0 community responses

AI DIAGNOSTICS

Incident analysis

Updated 2d ago• Trouble ID

Most likely causes

Load balancer configured to force full TLS renegotiation frequently, preventing session resumption and causing crypto CPU surge. 95%

Walsender processes are single-threaded per connection and block on handshake crypto, so many renegotiations saturate CPU cores. 90%

Intermediate device or policy forcing connection close instead of renegotiation, causing repeated new handshakes. 85%

Summary

TLS renegotiation forces repeated full handshakes which are CPU-expensive, especially on servers performing crypto for each walsender. PostgreSQL's walsender uses blocking SSL operations and is not optimized for frequent full handshake churn; constant teardown/reconnect also disrupts the streaming replication pipeline, causing lag. Session resumption (TLS session tickets or TLS 1.3 PSK) can avoid full handshakes, but must be supported and correctly configured by the load balancer and server. Offloading TLS or allowing long-lived TLS sessions reduces CPU cost and stabilizes streaming.

Suggested Fixes

Diagnostic step 1

Disable aggressive TLS renegotiation on the load balancer; prefer long-lived TLS sessions with session tickets or TLS1.3 PSK resumption.

Diagnostic step 2

Enable and verify TLS session resumption (session tickets) on both PostgreSQL server OpenSSL configuration and the load balancer. Test that a resumed session avoids a full handshake.

Diagnostic step 3

If load balancer cannot support session resumption, consider TLS offload to a dedicated proxy with proper resumption support or use a TCP passthrough and terminate TLS on the database hosts with lower churn.