1
Diagnostic step 1
Immediate mitigation: reduce guest MTU to 1500 in a controlled fashion (short downtime) to restore connectivity under current firmware if jumbo is not essential, or move a subset of services to a host with previous firmware.
FULL AI ANALYSIS
KVM guests lose all network after host NIC firmware update under sustained I/O — MTU resets and guests show 1500-only traffic
Generated Oct 14, 2026, 9:39:02 AM
Analysis summary
Symptoms (guest MTU collapse to 1500, driver warnings in dmesg, issue triggered under sustained I/O) point strongly to a host NIC firmware or driver regression affecting offloads (TSO/GSO/GRO) or bridge/bridge-netdev interactions. Possible that driver now advertises offload capabilities inconsistently under load or that firmware enforces MTU constraints per PF/VF. Guest-to-guest working indicates vhost/virtio path internal to host remains functional, while external offload/MTU negotiation to the wire fails. Reverting firmware or using driver workarounds is likely required.
Possible causes
NIC firmware/driver regression breaking offload handling (TSO/GSO/GRO) or MTU propagation to bridge/vhost, triggered under high I/O
85%Host bridge or bond configuration edge-case incompatibility with new driver (e.g., VLAN/jumbo handling bug)
55%IOMMU or VF reset behavior changed causing transient detach of vhost-net devices under load
30%Underlying switch or L2 path rejecting jumbo frames after host changed negotiated settings (less likely because host still shows MTU 9000)
20%Suggested fixes
2
Diagnostic step 2
Collect extended dmesg, ethtool -k, ethtool -S, and bridge/link stats during reproduction; capture the exact driver version and firmware build. Note the warning strings for vendor TAC.
3
Diagnostic step 3
On a single test host, try booting with the previous driver module (if available) or rollback firmware in a maintenance window to confirm regression. If rollback not feasible, add kernel module options to disable specific offloads (ethtool -K) to see if disabling TSO/GSO/GRO restores guest MTU propagation.
4
Diagnostic step 4
Test creating a new simple bridge and attaching a test guest to it to isolate whether bond/bridge config interacts with driver change.
5
Diagnostic step 5
If vendor firmware/driver bug suspected (high likelihood), gather logs and escalate to NIC vendor support with reproduction steps and captured outputs. Avoid in-place mass firmware rollbacks without vendor guidance; follow vendor-prescribed roll-back procedure and backups.