|
IPSec Protocol Basics.
IPSec VPN tunnels maintain their connection through several underlying mechanisms. Understanding these mechanisms is essential for accurately diagnosing disconnection issues.
IPSec VPN consists of two main phases:
- Step 1 IKE (UDP500/UDP4500(NAT-T)): Establishes a secure channel between peers and authenticates them. Over this connection, the security parameters for Step 2 are also determined. This connection determines if a peer is up or not.
- Step 2 IPsec(ESP IP protocol 50, NAT-T ESP over 4500): Using the security parameters negotiated over the IKE connection, this second connection carries the secured, tunneled data.
Both phases have lifetimes, after which renegotiation is required. Improper handling of this renegotiation process can result in disconnections from the tunnel.
Common Causes of IPSec VPN Disconnections.
- Dead Peer Detection (DPD).
DPD is a mechanism that detects when a VPN peer is no longer responsive. When a tunnel becomes idle (no traffic passing through), DPD begins sending "Are You There?" probes to verify if the peer is still active.
How DPD works:
- The FortiGate sends R_U_THERE messages when no traffic is detected for the DPD idle timeout period.
- If no response is received after a configured number of retries, the tunnel is declared dead and torn down.
Impact on Disconnections:
When DPD is set too aggressively (short retry intervals, low retry counts), normal network fluctuations can trigger unnecessary tunnel teardowns. This is especially problematic for users on non-dedicated internet connections like residential broadband, where packet loss is more common.
For more information on DPD configuration and troubleshooting, review the following KB article: Technical Tip: Configuring DPD (dead peer detection) on IPsec VPN .
-
Keepalive Mechanisms.
Keepalive messages maintain the tunnel's active state by generating periodic traffic. Unlike DPD, which checks if a peer is alive, keepalives prevent the tunnel from going idle. Without keepalives, a tunnel that experiences no user traffic may be susceptible to disconnection due to DPD or other timeout mechanisms. The following is a snapshot of the configuration relevant to the keepalive.
config vpn ipsec phase2-interface
edit "phase2-name"
set keepalive enable
next
end
-
IP protocol issues (IPv4/IPv6).
A common but often overlooked cause of disconnection is the presence of dual IP protocol stacks. When a client has both IPv4 and IPv6 enabled, but the VPN is configured primarily for IPv4, protocol conflicts can occur.
This can manifest as:
- Tunnel establishment over IPv6, but intended for IPv4 traffic.
- Peer identification issues where the client attempts to use an IPv6 address as its peer ID.
- Routing conflicts between IPv4 and IPv6 traffic.
To verify if this is causing issues, the following article can help.
Technical Tip: FortiClient IPsec dialup tunnel keep disconnecting
-
Network Path Issues.
Network path problems are often the root cause of intermittent disconnections, especially when they only affect specific users or locations.
Common network path issues include:
- ISP traffic shaping or rate-limiting of IPSec protocol traffic (UDP ports 500/4500).
- Intermittent packet loss on residential internet connections.
- NAT device timeouts that close translation entries for idle connections.
- Firewalls or routers along the path drop the ESP packet.
These issues need to be reviewed with the ISP, as they fall outside the IPsec peers.
-
Traffic Idle Timeouts.
Many network devices maintain connection state tables and will remove entries for connections that remain idle for too long. When this happens, the tunnel may appear to be up but cannot pass traffic, leading to a disconnect when traffic resumes.
Idle timeout issues can be mitigated by:
- Enabling keepalives to generate periodic traffic.
- Configuring appropriate DPD settings.
- Implementing application-level keepalives for critical services.
-
Rekey Operations.
IPSec tunnels rely on security associations (SAs) with defined lifetimes, which must be rekeyed upon expiration.
Two primary rekey methods exist:
- Soft rekey: New SAs are established before the old ones expire, allowing for a seamless transition.
- Hard rekey: Old SAs are deleted before new ones are established, causing a brief disconnection.
Several software issues or design challenges throughout the years have been seen on different vendors, including FortiClient, and were later fixed, which prevented the correct rekey, such as:
- Break the Tunnel before rekey, causing momentary disconnections.
- Using incorrect parameters in the REKEY messages, such as using the wrong Message-ID.
To troubleshoot these issues, the network administrator can capture and decrypt UDP network traffic using a sniffer to observe the IKE exchanges. For more details on this technique, the following article provides additional information: Technical Tip: How to decrypt IPSec Phase-2 (ISAKMP) packets IKEv2
-
Hardware Offloading Issues.
Modern FortiGate devices use specialized hardware (NPU - Network Processing Units) to accelerate IPSec traffic processing. While this generally improves performance, it can sometimes cause issues with specific traffic patterns or when combined with other features.
To determine if NPU offloading is causing disconnection issues, temporarily disable it for the problematic tunnel:
config vpn ipsec phase1-interface
edit "tunnel-name"
set npu-offload disable
next
end
-
DDoS Policy.
If the DDoS policy on the WAN interface used by the IPsec VPN tunnel has aggressive threshold values, then the FortiGate clears the VPN connection. The same can be verified from the logs. under: Log & Report -> Security Events -> Logs, select Anomaly in the drop-down) and filter for the user's public IP address.
Systematic Troubleshooting Methodology.
Troubleshooting intermittent disconnections requires a methodical approach:
- Identify the Disconnection Pattern.
- Are disconnections happening at regular intervals? This suggests a timeout or rekey issue.
- Do the disconnects occur only during periods of inactivity? This suggests DPD or keepalive issues.
- Do the disconnects affect all users or just those on specific networks? This suggests a path or ISP issues.
- Do the disconnects correlate with specific actions or times of day? This suggests traffic-related triggers.
- Collect and analyze diagnostic information.
- Apply targeted fixes.
-
MTU/Fragmentation issues.
If a VPN tunnel encounters packets larger than the path MTU, fragmentation may fail. This is common when NAT devices, firewalls, or intermediate routers drop oversized packets or fragments. The DF (Don’t Fragment) bit in IP headers can prevent fragmentation, and default FortiGate behavior (set honor-df enable) respects this bit.
Impact:
Large packets may be dropped, resulting in intermittent connectivity, application errors or apparent tunnel disconnections.
Mitigation:
-
Lower the MTU on the tunnel interface (typical: 1400–1420 bytes) or the parent WAN interface.
-
Configure TCP MSS clamping for TCP traffic to prevent oversize packets.
-
Enable Path MTU Discovery (PMTUD) so endpoints can adjust packet size dynamically.
-
Use pre-encapsulation fragmentation if intermediate devices drop fragments:
config vpn ipsec phase1-interface
edit <tunnel-name>
set ip-fragmentation pre-encapsulation
set npu-offload disable # required for NP6/NP6XL
next
end
Note:
- Consider set honor-df disable only if endpoints cannot receive ICMP (fragmentation needed) messages.
- Test with diagnose vpn tunnel list and packet captures to verify successful fragmentation.
To learn more about MTU/fragmentation issues, please check this link: Technical Tip: IP Packet fragmentation over IPSec tunnel interface explained
-
Certificate or authentication issues.
IPsec tunnels using certificates or PSKs can fail due to:
-
Expired or untrusted certificates.
-
Mismatched pre-shared keys (PSK).
-
Certificate Revocation List (CRL) check failures.
-
Misconfigured certificate authentication settings (e.g., EAP-cert or signature mismatch).
Impact:
-
The tunnel may fail during Phase 1 or authenticate and then drop immediately after rekey.
-
Intermittent disconnects if CRL validation intermittently fails.
-
Authentication errors logged, such as 'certificate validation before EAP failed' or failure to verify peer certificates.
Mitigation:
Related articles:
Technical Tip: How to troubleshoot Intermittent IPsec Dial up VPN disconnection
Troubleshooting Tip: How to Troubleshoot the issue of Intermittent packet drops over SSL VPN
Troubleshooting Tip: Site-to-Site IPsec VPN Intermittent Connection due to phase 2 mismatch
|
