FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
fwilliams
Staff
Staff
Article Id 230343
Description

This article describes how to understand and troubleshoot 'HMAC validation failed' error. 

Scope FortiGate v6.4, v7.0 and v7.2.
Solution

The Error:

Invalid ESP packet detected (HMAC validation failed).

 

Reason:

The reason of  this error on FortiGate is that the MAC calculated by FortiGate does not match the one inside the ESP packet received from the VPN peer.

 

The cause of this error can reside in 2 places:

1) In the Kernel (software).

2 )In the Hardware (NPU).

 

In case it resides in the kernel, it will be visible in:

 

Event logs:

Similar messages will be visible in the log:

 

status=esp_error error_num=Invalid ESP packet detected (HMAC validation failed). spi=fe2b6574 seq=0000ba0d

 

- IKE debug:

Similar messages will be visible in IKE debug:

 

ike 1:My_VPN: invalid ESP 1 (HMAC) SPI fe2c9989 seq 0000c10f 9  x.x.x.x->y.y.y.y

 

- The RX error (rxe) counter:

 Use ' diag netlink interface list My_VPN' on the IPSec interface (phase 1), repeat the command couple of times, and the rxe counter will increase.

 

# diag netlink interface list My_VPN

if=My_VPN family=00 type=768 index=15 mtu=1500 link=0 master=0

ref=15 state=off start fw_flags=0 flags=up p2p run noarp multicast

Qdisc=noqueue

stat: rxp=0 txp=0 rxb=0 txb=0 rxe=125 txe=0 rxd=0 txd=0 mc=0 collision=0

 

It can reside in hardware (NPU):

In this case, the error will not be visible in the first 3 places mentioned (Event log, IKE debug, and rxe counter).

It is only possible to see the error in the NPU DCE counter (non-zero sub-engine drop count).

 

To check NPU DCE counter, use:

 

# diag npu np6xlite dce 0   <-----Nnp6xlite platform.

# diag npu np6 dce 0               <----- Np6 platform.

 

 

Possible causes:

1) This error can be caused by a misconfiguration on any of the peers (local FortiGate or remote FortiGate if both ends are FortiGates, or 3rd party device if peering with 3rd party). 

2) It can be because the ESP packet was altered.

- Wireshark can be used to check the HMAC of the ESP packet sent and receive on the FortiGate.

3) It can be an issue with the NPU offloading.

 

Fixes:

1) Ensure that the proposal matches (on local and remote).

2) Ensure IPsec SA Encrypt/Decrypt algorithm and HMAC are supported by the unit’s NPU.

3) Ensure the interface bound to IPSec phase 1 is on the NPU’s port list.

 

Check with:

 

# diag npu np6 port-list | grep <port_name>

# diag npu np6xlite port-list | grep <port_name>

 

4) Try to disable hardware offload under phase1:

 

# config vpn ipsec phase1-interface

    edit <p1_name>

        set npu-offload disable

    end

 

5) CPx offload can be disabled if needed:

 

# config system global

    set ipsec-asic-offload disable

end

 

6) Disable anti-reply under phase 2:

 

# config vpn ipsec phase2-interface

    edit <p2_name>

        set replay disable

end

 

7) Disable IPsec-inbound-cache:

 

# config system npu

    set ipsec-inbound-cache disable

end

Contributors