Technical Tip: IPSec site-to-site VPN tunnel’s phase 2 is down due to ‘error calculating auth information’

[srajapratap]

Description	This article describes an IPSec scenario with error ‘error calculating auth information’ in IKE debug logs In this scenario, the IPsec tunnel is configured between FortiGate and FortiGate/non-Fortinet peer, with appropriate phase1 and phase2 configuration on respective nodes, the phase 2 remains down. In IKE debug logs, it can be seen that phase1 negotiation is successful, in phase 2, the negotiation stops when the responder is unable to process the authentication message sent by the initiator. To generate IKE debug logs, see Troubleshooting Tip: IPsec Tunnel (debugging IKE). Responder logs: ike 1:To_Initiator:107663: sent IKE msg (INFORMATIONAL):<FortiGate/responder IP>:500-><Peer/initiator IP>:500, len=76, id=01a434efd1ff1bef/b6c552c4dc25fe05 ike 1: To_Initiator: schedule auto-negotiate ike 1: To_Initiator: flushed ike 1: To_Initiator:107664: processed INITIAL-CONTACT ike 1: To_Initiator:107664: error calculating auth information After the responder fails to calculate authentication information, the initiator or peer will try to send authentication information periodically and phase2 will remain down. ike 1: comes <Peer/initiator IP>:500-><FortiGate/responder IP>:500,ifindex=7.... ike 1: IKEv2 exchange=AUTH id=44425e1f361f44cf/e27c22935352beff:00000001 len=284 ike 1: in <Hash> ike 1: To_Initiator:107665: detected retransmit ike shrank heap by 159744 bytes ike 1: comes <FortiGate/responder IP>:500-> <Peer/initiator IP>:500,ifindex=7.... ike 1: IKEv2 exchange=AUTH id=44425e1f361f44cf/e27c22935352beff:00000001 len=284 ike 1: in <Hash> ike 1: To_Initiator:107665: detected retransmit After the responder fails to calculate authentication information, the initiator will try to send authentication information periodically. This error message can appear when either using IKEv1 or IKEv2.
Scope	FortiGate 7.0 and above.
Solution	Make sure that the P1 interface had localid-type set to ‘auto’ instead of ‘key-id’. In the previous configuration, localid-type is set as keyed but no string value is assigned to the ‘set localid <IP string>’ command. Either users can unset the localid option or manually set it to ‘auto’. This will ensure that the phase2 is successfully negotiated. Before: config vpn ipsec phase1-interface     edit "To_Initiator"         set interface "port1"         set ike-version 2         set keylife 28800         set peertype any         set net-device disable         set proposal aes256-sha1         set localid-type keyid    <---- set localid-type auto.         set dhgrp 14         set nattraversal disable         set remote-gw <FortiGate/responder IP>         set psksecret ENC     next end After: config vpn ipsec phase1-interface     edit "To_Initiator"          set interface "port1"          set ike-version 2          set keylife 28800          set peertype any          set net-device disable          set proposal aes256-sha1          set localid-type auto          set dhgrp 14          set nattraversal disable          set remote-gw <FortiGate/responder IP>          set psksecret ENC     next end Related documents: Troubleshooting Tip: IPsec Tunnel (debugging IKE) Technical Tip: Use of PeerID and LocalID in IPsec VPN between two FortiGates Technical Tip: How to use local-ID type IP address other than the IP addresses configured in the int... IPsec Phase 1 configuration Troubleshooting Tip: Troubleshooting IPsec site-to-site tunnel connectivity