Troubleshooting Tip: 'Invalid ESP packet detected (replayed packet)'. VPN error

Description

This article describes how to understand and troubleshoot the 'Invalid ESP packet detected (replayed packet)' error.

Scope

FortiOS.

Solution

The error:
Invalid ESP packet detected (replayed packet).

Reason:
A sequence number that monotonically increases is assigned to each encrypted packet by IPsec to provide anti-replay protection against an attacker. The receiving IPsec endpoint keeps track of which packets it has already processed when it uses these numbers and a sliding window of acceptable sequence numbers.

When incoming IPsec traffic is received on FortiGate with sequence number already received, this packet is marked as a duplicate and dropped.

When the IPsec SA life is too long, or the volume of traffic is high, it is possible to see the same ESP sequence number once the ESP sequence number in 32 bits been utilized and start again from 1.

In order to verify, duplicate sequence number ESP packet capture can be performed.

To decrypt the ESP packet refer to:
Technical Tip: Decrypt ESP packets

Solution:

Option 1: Disable anti-replay protection on VPN tunnel:

config vpn ipsec phase2-interface
    edit <phase2-name>
        set replay <enable | disable>
end

For more information about anti-replay protection refer to:

Technical Tip: Explaining IPsec Anti-replay and preventing packet drops

Option 2: Reduce IPsec phase2 lifetime, for new SA creation:

For keylifeseconds:

config vpn ipsec phase2-interface
    edit <phase2-name>
        set keylifeseconds <time in seconds>
end

For keylifekbs:

config vpn ipsec phase2-interface
    edit <phase2-name>
        set keylife-type
        set keylifekbs <integer>
    end

Make sure the same settings are configured on both local and peer IPsec Gateway.

Note: In environments using ECMP (Equal-Cost Multi-Path), SD-WAN load balancing, or asymmetric forwarding, ESP packets may arrive out of sequence. If packet reordering exceeds the configured anti-replay window, FortiGate may log replay packet events even though the tunnel remains operational.

Replay packet logs alone should not be treated as evidence of packet loss. It is recommended to validate tunnel counters, application behavior, and packet ordering before considering any changes to anti-replay settings.