FGSP and two dedicated IPSec tunnels

[lhubschmid]

Hello community,

Following situation:

- provider with 2 sites (let's call them P-A and P-B)

- customer with 2 sites (let's call them C-A and C-B)

- dedicated connection between P-A and C-A (cost 100)

- dedicated connection between P-B and C-B (cost 200)

- one FG 60E (FortiOS 5.4.4) per site to encrypt the dedicated connection (IKEv2/IPSec in interface/route-based mode)

-- wan1 used for dedicated connection

-- internal1 used for LAN

-- internal2 used for HA-sync

- LAN IP redundancy using VRRP (master P-A / C-A)

- Routing with OSPF and BFD

- FGSP to sync sessions (TCP, UDP, IMCP, Expectation) -> IPSec is synced as well and cannot be disabled

Now, after starting the FortiGates, everything works as expected.

Both tunnels (P-A <-> C-A and P-B <-> C-B) are started, routing etc. works.

P-A # diag vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=To-C-A ver=2 serial=1 1.1.1.1:0->1.1.1.2:0
bound_if=3 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
proxyid_num=1 child_num=0 refcnt=20 ilast=2 olast=2 auto-discovery=0
stat: rxp=995 txp=1015 rxb=116060 txb=88102
dpd: mode=off on=0 idle=20000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=To-C-A proto=0 sa=1 ref=3 serial=2 auto-negotiate
  src: 0:0.0.0.0/0.0.0.0:0
  dst: 0:0.0.0.0/0.0.0.0:0
  SA: ref=5 options=2f type=00 soft=0 mtu=1438 expire=70/0B replaywin=1024 seqno=2 esn=0 replaywin_lastseq=00000000
  life: type=01 bytes=0/0 timeout=73/120
  dec: spi=058d3bfa esp=aes key=16 7d1c8ff1d8f2f67ae4177bb342e3c0ca
       ah=sha1 key=20 b24f590bd6cabe082b048ef30f12115f684712ba
  enc: spi=3e35e556 esp=aes key=16 3aa480ced86c3b72fe2b355a2e28e6fa
       ah=sha1 key=20 fa2a6445019beea07f5737391d480be5b518145e
  dec:pkts/bytes=0/0, enc:pkts/bytes=1/152
  npu_flag=01 npu_rgwy=1.1.1.2 npu_lgwy=1.1.1.1 npu_selid=1 dec_npuid=0 enc_npuid=1

P-A # diag vpn ike status
connection: 1/1
IKE SA: created 1/1 established 1/1 times 10/10/10 ms
IPsec SA: created 1/1 established 1/1 times 10/10/10 ms

Issue:

When I shutdown the VPN tunnel (diag vpn tunnel down To-C-A) or after the phase2 keylife (for testing: 120 seconds) has expired, the phase2 does not re-initiate an SA and the VPN tunnel stays down.

Log message (diag debug application ike -1 & diag debug enable) during tunnel is up as well as when tunnel is down (on C-A):

ike 0:To-P-A:To-P-A: IPsec SA connect 3 1.1.1.2->1.1.1.1:0 ike 0:To-P-A: ignoring since HA standalone session syncing is enabled

Log message on SA expiration:

C-A # ike 0:To-P-A: IPsec SA 7d7a6eed/3e35e554 hard expired 3 1.1.1.2->1.1.1.1:0 SA count 0 of 0

P-A # diag vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=To-C-A ver=2 serial=1 1.1.1.1:0->1.1.1.2:0
bound_if=3 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
proxyid_num=1 child_num=0 refcnt=19 ilast=301 olast=1 auto-discovery=0
stat: rxp=1942 txp=1966 rxb=212744 txb=171756
dpd: mode=off on=0 idle=20000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=To-C-A proto=0 sa=0 ref=2 serial=2 auto-negotiate
  src: 0:0.0.0.0/0.0.0.0:0
  dst: 0:0.0.0.0/0.0.0.0:0

P-A # diag vpn ike status
connection: 1/1
IKE SA: created 1/1 established 1/1 times 10/10/10 ms
IPsec SA: created 0/1 established 0/1 times 10/10/10 ms

Only way I found to get the VPN tunnel up again is to restart the IKE process (diag vpn ike restart) or to restart the whole FortiGate.

The IPSec configurations are identical on both peers.

P-A # sh vpn ipsec phase1-interface
config vpn ipsec phase1-interface
    edit "To-C-A"
        set interface "wan1"
        set ike-version 2
        set peertype any
        set dpd on-idle
        set comments "VPN: To-C-A"
        set remote-gw 1.1.1.2
        set psksecret ENC <key>
    next
end

P-A # sh vpn ipsec phase2-interface
config vpn ipsec phase2-interface
    edit "To-C-A"
        set phase1name "To-C-A"
        set auto-negotiate enable
        set comments "VPN: To-C-A"
        set keylifeseconds 120
    next
end

Why does this work on FortiGate startup or IKE restart but not afterwards?

Any help is greatly appreciated!

Kind regards,

Lukas