Technical Tip: Configuring more than one Main-Mode Pre-Shared Key (PSK) dialup IPSec phase1 on a FortiGate

rmetzger · ‎11-25-2009

Description

This article explains a limitation inherent to IKE when trying to configure multiple IPSec phase1 in Pre-Shared Key (PSK) mode.

If you configure more than one 'dialup' phase1 in Main Mode PSK on the same FortiGate port (IP), then the same phase1 will always be matched by ALL dialers. The error will show up in the third Main Mode round (authentication) as per the example below :

2009-11-04 16:00:00 device_id=FGTxxx log_id=0101023003 type=event subtype=ipsec pri=error loc_ip=aa.bb.cc.dd loc_port=500 rem_ip=ee.ff.gg.hh rem_port=500 out_if="wan1" vpn_tunnel="branchOffice" cookies=asd2345sdf4sdf345 action=negotiate status=negotiate_error msg="Negotiate SA Error: probable pre-shared secret mismatch"

When multiple 'dialup' Main Mode PSK phase1 exist, IKE has no way to know which phase1 must be matched upon receipt of the first main mode packet :

The source IP (dialer's IP) cannot be used as a distinguisher (any source IP is allowed for dialup phase1).
The destination IP (FortiGate's IP) cannot be used as a distinguisher (we have assumed that all dialup phase1 were bound to the same FortiGate IP).
There is no payload in the first Main Mode packet, which would make it possible to select a specific phase1.

Consequently, the first phase1 in the IKE internal list is always matched.
This is not a FortiOS limitation.

The PSK itself is part of the phase1 root key (SKEYID) calculation.
This root key is then derived into two keys (SKEYID_e, SKEYID_a), which are used for the phase1 encryption and authentication.
Phase1 keys are therefore dependent on the PSK.

When the phase1 initiator sends the MI3 message (fifth Main Mode message), it is protected with its locally calculated SKEYID_e/SKEYID_a. If the phase1 which was chosen by the responder, is not the correct one, then SKEYID_e and SKEYID_a on the responder are different from those on the initiator therefore leading to a failure ('probable PSK mismatch' in the 'debug ike') because the responder is unable to decipher the initiator's packet.

Scope

All FortiOS.

Solution

To work around this Main Mode PSK limitation for multiple dial-up connections :

Use aggressive mode with ID (ID is sent in clear text in the first Aggressive Mode message and can therefore be used by the responder to select the correct phase1).
Use Main Mode with RSASIG.

Main Mode RSASIG does not suffer from this limitation and still offers ID protection :
Upon receipt of the first Main Mode message, the IKE responder picks up the first dialup Main Mode RSASIG phase1 in its list. If it is not the correct phase1, it is not an issue.

When MI3 message will reach the IKE responder, it will be able to authenticate and to decipher it because SKEYID_a/e keys do not depend on the phase1 specification (the root SKEYID is based on payloads that were exchanged between each peer (nonces) and on the Diffie-Hellman private shared key that was locally computed by each peer).

Once the MI3 message is decrypted, the IKE responder finally knows the ID of the initiator (which by default is the DN of the initiator certificate) and can therefore 'fallback' to the correct phase1 if needed.

Related articles:

Troubleshooting Tip: Troubleshooting IPsec Site-to-Site Tunnel Connectivity

Technical Tip: How to configure VPN Site to Site between FortiGates (Using VPN Setup Wizard)

Troubleshooting Tip: IPsec VPNs tunnels

Technical Tip: Setting multiple DNS server for IPSec dial-up VPN

Technical Tip: NAT-traversal comparison between site-to-site and dial-up” dynamic” tunnels

Technical Tip: FortiGate Hub with multiple IPSec Dial-up phase1 using IKEv2 and PSK authentication

Technical Tip : How to configure multiple VPN tunnels from the same ISP to the same remote peer ISP.

Technical Tip: IPSec dial-up full tunnel with FortiClient

Technical Tip: Differences between Aggressive and Main mode in IPSec VPN configurations

Technical Note: Dynamic routing (BGP) over IPsec tunnel

Technical Tip: OSPF with IPSec VPN for network redundancy