Troubleshooting Tip: IPsec VPN Phase 1 Process - Aggressive Mode

Jackie_T · ‎09-11-2019

Description

This article describes the process through which IPsec VPN is established in Phase 1 - aggressive mode with some example from Wireshark.

Useful links:
Fortinet Documentation.

Solution

The IPsec VPN communications build up with 2-step negotiation:
Phase1: Authenticates and/or encrypt the peers.
Phase2 (Quick mode): Negotiates the algorithm and agree on which traffic will be sent across the VPN.

In this KB, the focus will be on Phase1 aggressive mode. Aggressive mode usually used for remote access VPN or if one or both peers have dynamic external IP addresses.

IKEv1 aggressive mode only requires three messages to establish the security association.

Network Topology:

FortiClient (Remote VPN) ----------- L3 Network --------- LAB FortiGate
[192.168.242.57] [Port1(WAN): 10.47.2.72]

The 3 messages will be explained one by one here below:

MESSAGE 1:
The first message will be from initiator (192.168.242.57) to responder (10.47.2.72).

The first packet always has Responder SPI with 0 value.
In this 1st message, the security associations attributes, DH nonces and the identification (in clear text) is available. This is the difference with main mode, as main mode will have identification info encrypted.

MESSAGE 2:
In this message, the responder will generate the DH shared key and send some messages to the initiator so that it can calculate the DH shared key.

It also calculates a hash that is used for authentication. This message also negotiates whether NAT-Traversal is needed.

MESSAGE 3:
The last message from initiator is a hash that is used for authentication.
If there is NAT-traversal being used, this message will be communicate using port UDP 4500. In this example, NAT-traversal is not in use, and hence port UDP 500 is used.

Now, the VPN Phase 1 is up. And the process will go to Phase2.

Related articles: