Ipsec vpn dial up flapping after 7.2.6

[Fabio]

Hello everybody,

I have a BIG problem and was start when I decide to upgrade my 200F ( Np6 Xlite cpu )at new release 7.2.8.

My scenario is a FGT with multiple a IpSec VPN Dynamic and several SSL VPN Portal .

In the initial version 7.2.3 we had no stability and connection problems. After the update we noticed disconnections after a few minutes and constant instability. We were forced to downgrade at the more stable 7.2.5. Even version 7.2.8 that read in the release notes that the problem had been solved was not a solver of the problem.

We also have several test FWs that have an NP6 process and also on those we have noticed the same problem. ( 300E , 3000D ).
And on these we also put the latest version 7.4.4. with the same result.

Below I show you the typical configuration of a VPN connection:

config vpn ipsec phase1-interface
edit "VPN_XXX"
set type dynamic
set interface "VLAN_7"
set mode aggressive
set peertype one
set net-device enable
set mode-cfg enable
set ipv4-dns-server1 10.12.0.5
set proposal aes128-sha256 aes256-sha256 3des-sha256 aes128-sha1 aes256-sha1 3des-sha1
set comments "Vpn YYYYY XXXX"
set wizard-type dialup-forticlient
set xauthtype auto
set authusrgrp "XXXXX_tec_VPN"
set peerid "XXXXXXXX"
set ipv4-start-ip 192.168.197.1
set ipv4-end-ip 192.168.197.99
set ipv4-netmask 255.255.255.0
set ipv4-split-include "Lan_SPLIT"
set domain "XXX.dom"
set save-password enable
set client-keep-alive enable
set psksecret ENC g78ffaAsPKwd1SnS6MclIaecNAvvtHRX8/
next
end

config vpn ipsec phase2-interface
edit "VPN_XXX"
set phase1name "VPN_XXX"
set proposal aes128-sha1 aes256-sha1 3des-sha1 aes128-sha256 aes256-sha256 3des-sha256
set comments "VPN: XXXXX"
next
end

We have opened a TAC to Fortinet support for a month now but they still haven't figured out the problem. I hope some of you can help me out.

Regards

Fabio

[AEK]

Hello Fabio

Did you try to disable NPU offload?

config vpn ipsec phase1-interface
  edit VPN_XXX
    set npu-offload disable
  end
end

[Fabio]

Hi AEK,

yes already tried in a Lab's FGT 300E.

config vpn ipsec phase1-interface
edit "VPN_LAB"
set type dynamic
set interface "VLAN_7"
set mode aggressive
set peertype one
set net-device enable
set mode-cfg enable
set ipv4-dns-server1 10.12.0.5
set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1
set comments "VPN: VPN_LAB (Created by VPN wizard)"
set npu-offload disable
set wizard-type dialup-forticlient
set xauthtype auto
set authusrgrp "VPN_User"
set peerid "vpnXXXX"
set ipv4-start-ip 192.168.198.5
set ipv4-end-ip 192.168.198.10
set ipv4-netmask 255.255.255.0
set ipv4-split-include "VPN_LAB_split"
set domain "rete.dom"
set save-password enable
set client-keep-alive enable
set psksecret ENC WMlU24fSwys9/
next

config vpn ipsec phase2-interface
edit "VPN_LAB"
set phase1name "VPN_LAB"
set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
set replay disable
set keepalive enable
set route-overlap allow
set comments "VPN: VPN_LAB (Created by VPN wizard)"
next

[amrit]

Hi Fabio
How often does the flap happen, is there any pattern or it's intermittent? 

The tunnel configuration you shared is a remote access FCT tunnel, you can check the following things 

1. Is Forticlient is causing the connection termination ?- Check VPN event logs in the firewall

2. Run continuous IKE logs and try to replicate the issue 

di vpn ike log-filere dst-addr4 <public ip address of the remote PC>

di de app ike -1 

di de console timestamp en

di de en 

to disable logs- di de dis 

3.take pcap on the wan interface for port 500 and 4500(disable npu for this under phase1)

4. Check if there is any packet loss or DPD failure between Fortigate and remote users 

5. Check if rekey in phase 1 or phase2 is causing this issue 

6. Check interface level errors 

fnsysctl ifconfig <vpn phase1 name>

If error are incrementing - try changing the MTU

config system interface

edit <vpn phase1 name>

set mtu-override en

set mtu 1350 

end 

end 

[Fabio]

Hi Amrit,

After the update at 7.2.8 we noticed disconnections after a few minutes, always with Forticlient for Win, Mac but not iPhone embedeed vpn client. I repeat in 7.2.3 and now in 7.2.5 in the same configuration we are NO PROBLEMS.

ALL MY TEST are in 300E that is a FGT in LAB , because my 200F is working and in stable release 7.2.5 .

The disconnection definitely avvien from the client because many times on the FGT the connection stays up and sends IKE msg (R-U-THERE) messages even though the client has already gone down, as you can see from this logs:

2024-07-07 10:18:56.433170 ike 0:VPN_LAB_0:14: sent IKE msg (R-U-THERE-ACK): XXX.YYY.208.56:4500->37.182.91.4:57353, len=108, vrf=0, id=e3f027d315cdc2e9/b1237ee594a12a57:9737b84b
2024-07-07 10:19:45.880034 ike 0:VPN_LAB_0: link is idle 53 XXX.YYY.208.56->37.182.91.4:57353 dpd=2 seqno=1 rr=0
2024-07-07 10:19:45.880056 ike 0:VPN_LAB_0:14: send IKEv1 DPD probe, seqno 1
2024-07-07 10:19:45.880124 ike 0:VPN_LAB_0:14: enc E3F027D315CDC2E9B1237EE594A12A5708100501F227E4E7000000600B000024AE0BB43CFC42C486EF61F00577A2BAED67E4675974FE527BB4B6045A6A8D4E4B000000200000000101108D28E3F027D315CDC2E9B1237EE594A12A5700000001
2024-07-07 10:19:45.880140 ike 0:VPN_LAB_0:14: out E3F027D315CDC2E9B1237EE594A12A5708100501F227E4E70000006C23EECE5926346D97C7795BB386AB72941FAFDC1351E0B53400A85E6F68A3D173DC81DCA3D5098D5D10E773C5C7890D4946FB24660ADA5D909442D85F265C3252A1FC12CE18AC59CBE8414A272DA47C58
2024-07-07 10:19:45.880160 ike 0:VPN_LAB_0:14: sent IKE msg (R-U-THERE): XXX.YYY.208.56:4500->37.182.91.4:57353, len=108, vrf=0, id=e3f027d315cdc2e9/b1237ee594a12a57:f227e4e7
2024-07-07 10:20:05.920030 ike 0:VPN_LAB_0: link is idle 53 XXX.YYY.208.56->37.182.91.4:57353 dpd=2 seqno=1 rr=0
2024-07-07 10:20:05.920051 ike 0:VPN_LAB_0:14: send IKEv1 DPD probe, seqno 1
2024-07-07 10:20:05.920117 ike 0:VPN_LAB_0:14: enc E3F027D315CDC2E9B1237EE594A12A57081005019E1C875C000000600B000024885E04CFACBFF5A116494FE9388240B342204CA429634745223F49FAF2168C2F000000200000000101108D28E3F027D315CDC2E9B1237EE594A12A5700000001
2024-07-07 10:20:05.920133 ike 0:VPN_LAB_0:14: out E3F027D315CDC2E9B1237EE594A12A57081005019E1C875C0000006CB9A7E8E4A259D67BA6763A023EF181EDB478F8CF570190FADA260D676162CDAE0C5ED38675AE81744FEC4A32A219AEFA9D5F1E920216A0DB4CEE5D2DF8C8F849CE4E881BD02F0FF10EADCFBB64D9A8FE
2024-07-07 10:20:05.920158 ike 0:VPN_LAB_0:14: sent IKE msg (R-U-THERE): XXX.YYY.208.56:4500->37.182.91.4:57353, len=108, vrf=0, id=e3f027d315cdc2e9/b1237ee594a12a57:9e1c875c
2024-07-07 10:20:25.960032 ike 0:VPN_LAB_0: link is idle 53 XXX.YYY.208.56->37.182.91.4:57353 dpd=2 seqno=1 rr=0
2024-07-07 10:20:25.960053 ike 0:VPN_LAB_0:14: send IKEv1 DPD probe, seqno 1
2024-07-07 10:20:25.960119 ike 0:VPN_LAB_0:14: enc E3F027D315CDC2E9B1237EE594A12A570810050171CEC089000000600B00002406B3516159CDA3C5CCCADDE4DCE1BA503DFFE7FC77EB0294218419D296C4A6A3000000200000000101108D28E3F027D315CDC2E9B1237EE594A12A5700000001
2024-07-07 10:20:25.960135 ike 0:VPN_LAB_0:14: out E3F027D315CDC2E9B1237EE594A12A570810050171CEC0890000006CB358C345DC46718DD1485BF997E5B05A78081FB802821BF620F98616E2587ABB6C7A47B443949E8DFD94448788AB7FA0035D28ED43F2F764251A03D89063B18034EDB8F49E7B3AF15029D6BF8E7DB358
2024-07-07 10:20:25.960156 ike 0:VPN_LAB_0:14: sent IKE msg (R-U-THERE): XXX.YYY.208.56:4500->37.182.91.4:57353, len=108, vrf=0, id=e3f027d315cdc2e9/b1237ee594a12a57:71cec089
2024-07-07 10:20:46.000040 ike 0:VPN_LAB_0: link fail 53 XXX.YYY.208.56->37.182.91.4:57353 dpd=2
2024-07-07 10:20:46.000062 ike 0:VPN_LAB_0: link down 53 XXX.YYY.208.56->37.182.91.4:57353
2024-07-07 10:20:46.000111 ike 0:VPN_LAB_0: deleting
2024-07-07 10:20:46.000154 ike 0:VPN_LAB_0: sent tunnel-down message to EMS: (fct-uid=FB216EF5E0ED563B9AA73056215ACAE9, intf=VPN_LAB_0, addr=192.168.198.5, vdom=root)
2024-07-07 10:20:46.000161 ike 0:VPN_LAB_0: flushing
2024-07-07 10:20:46.000195 ike 0:VPN_LAB_0: deleting IPsec SA with SPI 0f1d017b
2024-07-07 10:20:46.000208 ike 0:VPN_LAB_0:VPN_LAB: deleted IPsec SA with SPI 0f1d017b, SA count: 0

The fnsysctl ifconfig VPN_LAB command doesn't seem to give me any information, as if it doesn't measure the VPN connection:

FortiGate-300E (root) # fnsysctl ifconfig VPN_LAB
VPN_LAB Link encap:Unknown
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1420 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0 Bytes) TX bytes:0 (0 Bytes)

Anyway, I tried changing the MTU but to no avail.

config system interface
edit "VPN_LAB"
set vdom "root"
set type tunnel
set snmp-index 46
set interface "VLAN_7"
set mtu-override enable
set mtu 1350
next

[Fabio]

I also tried SSL VPN and they don't work either....

Here it is easier...because the web portal is not loaded...and so the connection with the client fails immediately.

I think the 200F and 300E are not devices on which they have tested the new releases properly.. maybe we should buy the new FGTs ?

I would like to know if anyone has tried a release higher than 7.2.5 and with what type of FGT.

Thanks

[amrit]

From the following logs : 

2024-07-07 10:20:25.960156 ike 0:VPN_LAB_0:14: sent IKE msg (R-U-THERE): XXX.YYY.208.56:4500->37.182.91.4:57353, len=108,

vrf=0,

The Fortigate is attempting to check DPD keepalive status , but due to termination of the connection by the forticlient , we are not getting any response. So, I think you can try a different version of the forticlient . If FCT is terminating the connection, I don't think the issue is fortigate
2. For SSLVPN, web portal is not required when connecting from the Forticleint. You can use the full or split tunnel connection 

To check the connection logs use the following commands

di vpn ssl debug-filter src-addr4 <public ip of the FCT PC>

di de app sslvpn -1 

di de en

This issue has nothing to do with the hardware 200F or 300E as both the devices are capable of working with the latest releases

[Fabio · Accepted solution]

Hi guys,

the issue was own problem, but it's strange that happened.

We have three router BGP to access internet one of those discarded or altered some packet in the Ipsec communication ( between them to balance the traffic is enabled protocol internal BGP ). The router are Cisco ASR1001X with release 16.06.03.SPA. When we upgrade at 16.12.08, the issue did not occur again.

For now the whole thing has been verified on the test FGT 300E but in the next few days we will plan to upgrade the FGT 200F to Release 7.2.8 and check for proper operation.

If all goes well after the update of the 200F as well, it will be to understand why these issue were only manifesting with releases 7.2.6 and up and not with lower releases.
I will keep you updated.

bye

Fabio