IPSec SA connect gone crazy

[Andre_Backs]

Hi all,

I have a perfectly normal IPsec tunnel that normaly works fine.

However, once in a while the connection gets lost and the Fortigate goes crazy.

Debug shows thousands of quickmode requests.

Here is a piece of debug after I flushed the tunnel on CLI:

:56 ike 0:p1-000300:55529767: negotiation timeout, deleting :56 ike 0:p1-000300: connection expiring due to phase1 down :56 ike 0:p1-000300: deleting :56 ike 0:p1-000300: flushing :56 ike 0:p1-000300: flushed :56 ike 0:p1-000300: deleted :56 ike 0:p1-000300:p2-000300: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 :56 ike 0:p1-000300:p2-000300: config found :56 ike 0:p1-000300: created connection: 0x3a2e310 6 62.177.226.236->89.146.20.81:500. :56 ike 0:p1-000300: IPsec SA connect 6 62.177.226.236->89.146.20.81:500 negotiating :56 ike 0:p1-000300: no suitable ISAKMP SA, queuing quick-mode request and initiating ISAKMP SA negotiation :56 ike 0:p1-000300:55529796: initiator: main mode is sending 1st message... :56 ike 0:p1-000300:55529796: cookie 2e968ceaf91b81e6/0000000000000000 :56 ike 0:p1-000300:55529796: out 2E968CEAF91B81E600000000000000000110020000000000000000900D00003800000001000000010000002C010100010000002401010000800B0001800C708080010007800E01008003000180020002800400020D000014AFCAD71368A1F1C96B8696FC775701000D0000144048B7D56EBCE88525E7DE7F00D6C2D3000000148299031757A36082C6A621DE00050E18 :56 ike 0:p1-000300:55529796: sent IKE msg (ident_i1send): 62.177.226.236:500->89.146.20.81:500, len=144, id=2e968ceaf91b81e6/0000000000000000 :56 ike 0:p1-000300:p2-000300.2: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 :56 ike 0:p1-000300:p2-000300.2: using existing connection :56 ike 0:p1-000300:p2-000300.2: config found :56 ike 0:p1-000300:p2-000300.2: IPsec SA connect 6 62.177.226.236->89.146.20.81:500 negotiating :56 ike 0:p1-000300:55529796:p2-000300.2:504855592: ISAKMP SA still negotiating, queuing quick-mode request :56 ike 0:p1-000300:p2-000300: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 :56 ike 0:p1-000300:p2-000300: using existing connection :56 ike 0:p1-000300:p2-000300: config found :56 ike 0:p1-000300: request is on the queue :56 ike 0:p1-000300:p2-000300.2: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 :56 ike 0:p1-000300:p2-000300.2: using existing connection :56 ike 0:p1-000300:p2-000300.2: config found :56 ike 0:p1-000300: request is on the queue :56 ike 0:p1-000300:p2-000300: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 :56 ike 0:p1-000300:p2-000300: using existing connection :56 ike 0:p1-000300:p2-000300: config found :56 ike 0:p1-000300: request is on the queue

The last 4 lines repeat over and over as if it were a logical loop.

Obviously this makes debugging this line difficult.

I suspect however that the other side is simply offline or misconfigured.

Any help would be appreciated.

André

[emnoc]

Questions

What's the remote-site type ( CHKP ASA FGT SRX OpenSource or what ? )

Do you have DPD enabled ?

Did you run any diag vpn commands to get phase1  and even phase2 status for this connection?

What fortiOS version ?

Ken

[Andre_Backs]

Hello Ken,

The remote site is a Cisco RV078 or something similar or an AXA5505 (I am not sure since I do not administer this device. But it is a Cisco for sure)

The ForitOS is v5.0,build3608 (GA Patch 7)

I had DPD enabled and also tried it with disabled with the same result.

Below is the output for "diag vpn tunnel list name" :

probib-hfd-fw1a # diag vpn tunnel list name p1-000300 list ipsec tunnel by names in vd 0 ------------------------------------------------------ name=p1-000300 ver=1 serial=13 62.177.226.236:0->89.146.20.81:0 lgwy=static tun=intf mode=auto bound_if=6 proxyid_num=2 child_num=0 refcnt=7 ilast=43183194 olast=43154405 stat: rxp=0 txp=0 rxb=0 txb=0 dpd: mode=off on=0 idle=5000ms retry=3 count=0 seqno=43 natt: mode=none draft=0 interval=0 remote_port=0 proxyid=p2-000300 proto=0 sa=0 ref=2 auto_negotiate=1 serial=3   src: 0:172.20.34.0/255.255.255.0:0   dst: 0:10.94.253.0/255.255.255.0:0 proxyid=p2-000300.2 proto=0 sa=0 ref=2 auto_negotiate=1 serial=5   src: 0:172.16.1.4/255.255.255.255:0   dst: 0:10.94.253.0/255.255.255.0:0

# diag vpn ike log-filter name p1-000300 # diagnose debug application ike -1

gives me the bulk output listed in my previous post

This is the phae1-interface:

config vpn ipsec phase1-interface     edit "p1-000300"         set interface "wan1"         set local-gw 62.177.226.236         set nattraversal disable         set dhgrp 2         set proposal aes256-sha1         set dpd disable         set comments "vpn rtrHeenweg"         set remote-gw 89.146.20.81         set psksecret ENC***

    next end it has 2 phase2 interfaces:

config vpn ipsec phase2-interface     edit "p2-000300"         set auto-negotiate enable         set comments "vpn Heenweg 10.94.253.0"         set dst-addr-type name         set keepalive enable         set pfs disable         set phase1name "p1-000300"         set proposal aes128-md5         set src-addr-type name         set dst-name "net_10.94.253-vpn"         set keylifeseconds 3600         set src-name "net_172.20.34.0_productie"     next end and

config vpn ipsec phase2-interface     edit "p2-000300.2"         set auto-negotiate enable         set comments "vpn Heenweg 10.94.253.0"         set dst-addr-type name         set keepalive enable         set pfs disable         set phase1name "p1-000300"         set proposal aes128-md5         set src-addr-type name         set dst-name "net_10.94.253-vpn"         set keylifeseconds 3600         set src-name "wise_mysql_rep"     next end

so basicaly I don't get a real error message, just: 2016-05-17 14:32:09 ike 0:p1-000300:p2-000300: IPsec SA connect 6 62.177.226.236->89.146.20.81:0 2016-05-17 14:32:09 ike 0:p1-000300:p2-000300: using existing connection 2016-05-17 14:32:09 ike 0:p1-000300:p2-000300: config found 2016-05-17 14:32:09 ike 0:p1-000300: request is on the queue

A packet capture shows only outgoing ISAKMP packets (port 500) to the destination.

André

[emnoc]

The cfg looks good but without phase1 your phase2 is going to be down and in your output your phase2 is down.

[Andre_Backs]

Hi Ken,

i totaly agree with you that phase2 is down, but my concern is that when phase1 goes down (or appears to be up when it is not) the phase2 floods the system with attempts to connect.

And when the p2 request is on the queue: where can I find (the status of) this queue.

The tunnel is currently up so I can't realy test.

Andre