Technical Tip: IPSec site-to-site VPN tunnel’s phase 2 is down due to ‘error calculating auth information’

srajapratap · ‎09-14-2022

Description

This article describes an IPSec scenario with error ‘error calculating auth information’ in IKE debug logs

In this scenario, the IPsec tunnel is configured between FortiGate and FortiGate/non-Fortinet peer, with appropriate phase1 and phase2 configuration on respective nodes, the phase 2 remains down.

In IKE debug logs, it can be seen that phase1 negotiation is successful, in phase 2, the negotiation stops when the responder is unable to process the authentication message sent by the initiator.

Below KB article can be used to generate IKE debug logs:
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-IPSEC-Tunnel-debugging-IKE/ta-p/1900...

Responder logs:

ike 1:To_Initiator:107663: sent IKE msg (INFORMATIONAL):<FortiGate/responder IP>:500-><Peer/initiator IP>:500, len=76, id=01a434efd1ff1bef/b6c552c4dc25fe05
ike 1: To_Initiator: schedule auto-negotiate
ike 1: To_Initiator: flushed
ike 1: To_Initiator:107664: processed INITIAL-CONTACT
ike 1: To_Initiator:107664: error calculating auth information

After the responder fails to calculate authentication information, the initiator or peer will try to send authentication information periodically and phase2 will remain down.

ike 1: comes <Peer/initiator IP>:500-><FortiGate/responder IP>:500,ifindex=7....
ike 1: IKEv2 exchange=AUTH id=44425e1f361f44cf/e27c22935352beff:00000001 len=284
ike 1: in <Hash>
ike 1: To_Initiator:107665: detected retransmit
ike shrank heap by 159744 bytes
ike 1: comes <FortiGate/responder IP>:500-> <Peer/initiator IP>:500,ifindex=7....
ike 1: IKEv2 exchange=AUTH id=44425e1f361f44cf/e27c22935352beff:00000001 len=284
ike 1: in <Hash>
ike 1: To_Initiator:107665: detected retransmit

After the responder fails to calculate authentication information, the initiator will try to send authentication information periodically.

This error message can appear when either using IKEv1 or IKEv2.

Scope

FortiGate 7.0 and above

Solution

Make sure that the P1 interface had localid-type set to ‘auto’ instead of ‘key-id’. In the previous configuration, localid-type is set as keyed but no string value is assigned to the ‘set localid <IP string>’ command. Either users can unset the localid option or manually set it to ‘auto’. This will ensure that the phase2 is successfully negotiated.

Before:

# config vpn ipsec phase1-interface
      edit "To_Initiator"
        set interface "port1"
        set ike-version 2
        set keylife 28800
        set peertype any
        set net-device disable
        set proposal aes256-sha1
        set localid-type keyid    <----set localid-type auto
        set dhgrp 14
        set nattraversal disable
        set remote-gw <FortiGate/responder IP>
        set psksecret ENC
      next
   end

After:

# config vpn ipsec phase1-interface
      edit "To_Initiator"
         set interface "port1"
         set ike-version 2
         set keylife 28800
         set peertype any
         set net-device disable
         set proposal aes256-sha1
         set localid-type auto
       set dhgrp 14
       set nattraversal disable
       set remote-gw <FortiGate/responder IP>
       set psksecret ENC
      next
   end