FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
msingh_FTNT
Staff
Staff
Article Id 195672

Description


This article describes how to troubleshoot basic IPsec tunnel issues and understand how to collect data required by TAC to investigate the VPN issues.

The process responsible for negotiating phase-1 and phase-2: 'IKE'.

Use the following steps to assist with resolving a VPN tunnel that is not active or passing traffic.

Stephen_G_0-1742309136370.png

 

 

Scope

 

FortiGate.


Solution


Step 1: What type of tunnel have issues:

FortiOS supports:

  • Site-to-Site VPN.
  • Dial-Up VPN.

 

Step 2: Is Phase-2 Status 'UP':

  • No (SA=0) - Continue to Step 3.
  • Yes (SA=1) - If traffic is not passing, - Jump to Step 6.
  • Flapping - SA is flapping between the 'UP' and 'Down' states - Jump to Step 7.

 

How to identify if Phase 2 is 'Up' or 'Down':

Phase-2 status can be found from both GUI and Command-Line.

From GUI:

When Phase2 is Down:

Stephen_G_1-1742309136372.png

 

 
When Phase2 is Up:
 
Stephen_G_2-1742309136376.png

 

 
One more way to check the IPsec monitor status from GUI is by clicking the up or inactive name under status in IPsec tunnel. Image attached below.
 

ipsecfirst.JPG

 

From CLI:

Execute the command 'diagnose vpn tunnel list name <phase1-name>' <- To view the phase1 status for a specific tunnel.     
('diagnose vpn tunnel list', can also be executed to view the phase2 status of all tunnels).
 
When Phase2 is Down:
 
Stephen_G_3-1742309136382.png

 

When Phase2 is Up:
 
Stephen_G_4-1742309136383.png

 

Step 3: Is IKE Phase1 up:
 
  • No (State – 'Connecting') - Continue to Step 4.
  • Yes (State – 'Established') - Jump to Step 5.

 

Execute the command 'diagnose vpn ike gateway list name <phase1-name>' <- To view the phase1 status for a specific tunnel.      

('diagnose vpn tunnel list' can also be executed to view the phase2 status of all tunnels).
 
When Phase1 is Down:
 
Stephen_G_5-1742309136384.png

 

When Phase1 is Up:
 
Stephen_G_6-1742309136384.png

 

 
Step 4: Analyze the IKE phase 1 messages on the responder for a solution. [Phase 1 not up]:

Troubleshooting the IKE Phase 1 problem is best handled by reviewing VPN status messages on the responder firewall.
The responder is the 'receiver' side of the VPN receiving tunnel setup requests.

The initiator is the side of the VPN that sends the initial tunnel setup requests.


Checklist:

 

  1. Check if there is any other device upstream of the firewall.
  2. Check if the VPN Gateway is configured to use the correct outgoing interface.
  3. Check that the remote IP is configured correctly.
  4.  Run packet capture on the outgoing interface and confirm it is possible to see traffic from the remote peer. If not:
  • Make sure that IKE traffic on port 500/4500 is allowed in the network device connected upstream.

Packet capture can be run from the CLI or the GUI:


GUI:

 
Stephen_G_7-1742309136384.png

 

In v7.2 and v7.4, diagnostics are an option instead of packet capture under the network section. Select diagnostics and then select new packet capture.
 

CLI:

diagnose sniffer packet any 'host <remote-peer-ip> and port (500 or 4500)' 6 0 l, control + c to stop

 

  1. If it is possible to see traffic on port 500/4500, follow the steps below to troubleshoot this issue:
  2. Run the below commands(on the receiver) to capture the IKE logs and initiate tunnel/traffic from the remote end.

 

diagnose debug console timestamp enable

diagnose vpn ike log-filter dst-addr4 10.10.100.109 <- 10.10.100.109 is the remote gateway.

diagnose debug application ike -1
diagnose debug application fnbamd -1

diagnose debug enable 
 
To disable debugs:
 
diagnose debug reset
diagnose debug disable
 

Note:

Starting from v7.4.1, the 'diagnose vpn ike log-filter dst-addr4' command has been changed to 'diagnose vpn ike log filter rem-addr4'

 

diagnose vpn ike log filter rem-addr4 10.10.100.109
diagnose debug application ike -1

diagnose debug application fnbamd -1

diagnose debug console timestamp enable

diagnose debug enable

 

To disable debugs:
 
     diagnose debug reset
   diagnose debug disable
 

Note:

Try to run the packet capture and the logs at the same time.  
If VDOMs are enabled, make sure to be in the VDOM context and then execute the above commands.


If there are more active tunnels on VDOM, is possible to reduce a possibly large debug output with the following command (FortiOS 7.4.x) and filter the debugged tunnel by name, for example:

 

diagnose vpn ike log filter name <name of tunnel>


Step 5: Phase1 has been established but Phase2 is down.

Checklist:

 

  1. Confirm if the Encryption and hash algorithms match on both the receiver and initiator.
  2. Check if PFS is enabled, if yes, make sure the configuration is matched on both the units.
  3. Make sure, that the quick mode selectors (interesting traffic) match on both units.
  4. If Phase-2 is still not up, run the packet capture on port 500/4500 and run the below commands.

 

diagnose vpn ike gateway list (Or diagnose vpn ike gateway list name <tunnel-name>)

diagnose vpn ike log-filter dst-addr4 10.10.100.109 <- 10.10.100.109 is the remote gateway.

diagnose debug console timestamp enable  
diagnose debug application ike -1
diagnose debug enable 
 
To disable debugs:
 
diagnose debug reset
diagnose debug disable
 

Note:

If VDOMs are enabled, make sure to be in the VDOM context and then execute the above commands.

Packet capture can be collected as shown below:

 
Stephen_G_8-1742309136385.png

 

 

Step 6: Phase2 is up but traffic is not passing.

Once the tunnel is up, traffic will be encapsulated in ESP (Encapsulating Security Payload) protocol and sent to the remote peer.

Checklist:

 

     1. Make sure the quick mode selector defined in Phase2 is configured properly to allow the traffic flow, which is having the issue.

For example:

Phase 2 define below allows traffic between – 192.168.1.0/24 and 192.168.2.0/24.

 
Stephen_G_9-1742309136388.png

 

 
Let's assume that the IP address of the PC having an issue is 10.10.100.100/24.
If this PC is trying to reach any host in the 192.168.2.0/24 network, FortiGate will drop this traffic because the phase2 quick mode selector does not have this source network included in it.

     2. Check the IPv4 policies and routes are in place to confirm:
  • If there is a policy defined for this traffic flow.
  • If there are any source and destination addresses defined, make sure it is configured to allow this traffic flow.
  • If the routes are added correctly pointing to the tunnel.
  • If the debug flow shows that the traffic is allowed but provides an error stating 'No matching IPsec selector, drop', check if NAT is enabled in the firewall policy for outbound IPsec VPN traffic. If it is, try to disable NAT in this firewall policy and test again. 

 

  1. In cases where ping is used as the diagnostic tool to test connectivity between local and remote sites, it will fail despite having the required firewall policy, phase 2 selectors (if defined), and static routes/blackhole routes configured on the FortiGate. However, when running the debug flow, it will show that the traffic is leaving the correct tunnel interface and gateway.
    Ping to the FortiGate interface and the remote wan interface works. To resolve this, check the Windows firewall settings on the destination devices whether it is enabled. This is a default behavior on the Windows firewall. To allow ICMP requests for testing connectivity, refer to these articles:

     

  2. If the issue persists:

  • Enable packet capture for remote peer’s IP address and set protocol to 50(ESP).
 
Stephen_G_10-1742309136388.png

 

  • Open two SSH sessions and run the below commands:

SSH session 1:

 

diagnose debug console timestamp enable  
diagnose debug flow filter addr <destination-IP>
diagnose debug flow filter proto <1 or 17 or 6> (optional) where 1=ICMP, 6 = TCP, 17 = UDP…
diagnose debug flow show iprope enable
diagnose debug flow trace start 1000

 

To disable debugs:
 
diagnose debug reset
diagnose debug disable

 

Note other protocol numbers can used as well for example OSPF(89).

SSH Session 2:

 

diagnose  vpn tunnel list  (Or diagnose vpn tunnel list name <phase2_tunnel_name> ).

 

SSH Session 3:

To clear the session for the source and destination use the following command:

 

diagnose sys session filter clear

diagnose sys session filter src <source-IP>

diagnose sys session clear

 

diagnose sys session filter clear

diagnose sys session filter dst <destination-IP>

diagnose sys session clear

 

Note:

If VDOMs are enabled, make sure not to be in the VDOM context, and then execute the command above.

Make sure to collect packet capture and the logs mentioned above around the same and attach it to the Fortinet case updates.

Along with this information, attach network topology (if any). 
With this information, TAC will try to decrypt the ESP traffic in Wireshark.
If the remote peer is FortiGate as well, take packet capture on this unit as well which will make sure that this unit received the encrypted traffic or if it was lost in the middle.

Note:
Under Dashboard ->IPsec Monitor, the number of encrypts and decrypts for the affected tunnel can be. In case Encrypts are increasing and decrypts are not, this means that maybe the FortiGate is not receiving the return traffic. In case there are decrypts and no encrypts, this means that the FortiGate is receiving the traffic from the peer end, however, due to some reason the FortiGate is not sending the traffic out to its LAN or the traffic is not received from LAN.
There are chances that both encrypts and decrypts are 0. In this case, first, it needs to be checked which side of the IPsec tunnel is initiating the traffic if the traffic is reaching the FortiGate, and if the firewall policies are in place.

Step 7:  Troubleshoot the IPsec VPN that is flapping.

Checklist:
 
  1. Check whether the issue affects one VPN or all configured VPNs. If all VPN tunnels are affected:
  • Check the Internet connection.
  • Run the following command to find errors/logs associated with the firewall/interface.

 

diagnose debug crashlog read

diagnose sys top 2 50 <- Press Ctrl + C to stop (run for 5 iterations).

get system performance status

diagnose hardware sysinfo conserve

diagnose hardware deviceinfo nic <interface-name>  

execute tac report

 

Note:
If VDOMs are enabled, make sure not to be in the Global context and then execute the commands above.

 

If only one tunnel is flapping, collect the 'VPN Events' log as shown below:

 
Stephen_G_11-1742309136389.png

 

  • Consider whether the VPN was stable for some time and is only now going up and down.
  • If so - Investigate for network or unit changes or if any new network equipment has been added to the environment. If so, confirm changes/additions are correct.
  • If not, collect logs and packet capture as mentioned in Step 4.

 

Make sure to collect packet capture and all the logs mentioned above around the same and attach it to the Fortinet case updates.

Along with this information, attach network topology (if any). With this information, TAC will investigate this issue.

Step 8: Logs to be collected and attached to TAC case.

Checklist:

SSH Session 1:
 
diagnose debug console timestamp enable  
diagnose debug application ike -1
diagnose debug application fnbamd -1
diagnose debug enable
 
SSH Session 2:
 
diagnose debug crashlog read
diagnose sys top 2 50
<- Press Ctrl + C to stop (run for 5 iterations).
get system performance status
diagnose hardware sysinfo conserve
diagnose hardware deviceinfo nic <interface-name>  
diagnose vpn ike gateway list
diagnose vpn tunnel list
execute tac report
 
To disable debugs:
 
     diagnose debug reset
   diagnose debug disable
 
First Packet capture  - IKE Traffic on ports 500/4500.
 
kb_19769_13a.png
  
Second Packet capture - ESP Traffic Protocol 50.
 
kb_19769_14a.png
 

Note: Some ISP blocks traffic on port 500, if no ESP packets are being received force NAT-T under phase1>>network. 

An ISP can block ESP packets even if the tunnel is up as they can filter traffic based on protocol number even after the tunnel is established.

 

Screenshot 2025-03-06 162340.jpg

 

To flush tunnel:

 

diagnose vpn tunnel flush <my-phase1-name>

diagnose vpn ike gateway clear name <my-phase1-name>

 

Related articles:

Technical Tip: How to configure VPN Site to Site between FortiGates (Using VPN Setup Wizard)

Troubleshooting Tip: IPsec VPNs tunnels

Technical Tip: Setting multiple DNS server for IPSec dial-up VPN

Technical Tip: NAT-traversal comparison between site-to-site and dial-up” dynamic” tunnels

Technical Tip: FortiGate Hub with multiple IPSec Dial-up phase1 using IKEv2 and PSK authentication

Technical Tip: How to configure multiple VPN tunnels from the same ISP to the same remote peer ISP.

Technical Tip: IPsec dial-up full tunnel with FortiClient

Technical Tip: Differences between Aggressive and Main mode in IPSec VPN configurations

Technical Tip: Dynamic routing (BGP) over IPsec tunnel

Technical Tip: OSPF with IPSec VPN for network redundancy

Technical Tip: Dynamic dial-up VPN with OSPF

Technical Tip: Fortinet Auto Discovery VPN (ADVPN)

Technical Tip: 'set net-device' new route-based IPsec logic

Technical Tip: Simple OCVPN deployment

Technical Tip: SD-WAN integration with OCVPN

Technical Tip: Configure IPsec VPN with SD-WAN

Technical Tip: SD-WAN with DDNS type IPsec

Technical Tip: SD-WAN primary and backup IPsec tunnel scenario

Troubleshooting Tip: IPsec VPN Phase 1 Process - Aggressive Mode

Technical Tip: Configuring more than one Main-Mode Pre-Shared Key (PSK) *dialup* IPSec phase1 on a F...

Technical Tip: How to configure IPsec VPN Tunnel using IKE v2

Technical Tip: Hard timeout for Dialup IPSEC VPN Tunnel

Technical Tip: IPsec VPN NAT-traversal