Technical Tip: FortiGate HA Sessions synchronization

[fgilloteau_FTNT]

Description

This article describes how FortiGate HA session synchronization works

Scope

FortiGate.

Solution

With session pickup enabled under the HA configuration, sessions and their status will be synced to the secondary unit so that sessions do not need to be recreated at the time of a failover. The session creation process will cause the primary unit to instruct a secondary that a new session has been created to keep in its session table (this statement has a few exceptions). 
 
config system ha 
    set session-pickup enable 
end 

The unique conditions, exceptions, and other details are explained in the following article: HA session failover (session pickup) 
When the session protocol state changes (this can be monitored by running the 'diag sys session list' and looking at the proto_state=xx field of the session), the session is synced again to the secondary unit. 
 
The session deletion/expiry process is as follows: 

• Primary unit: 
  1. When a session expires, the primary will send a delete to the secondary unit. 
  2. Following the above step, the primary unit will delete its session. 

• Secondary unit: 
  1. When a session expires, the secondary will delete its own session first, then send a 'query' to the primary unit to check the status of the session. 
  2. If the primary still has that session, it will send 're-create' to the secondary unit. 
  3. The secondary receives the “re-create” message and creates the session.

Each session will have the same timer on both the primary and secondary unit meaning sessions can expire on both the primary and the secondary unit at the same time causing a unique race condition.

This unique race condition happens if the secondary unique queries the primary for the validity of the session status before the primary has deleted the session due to expiry. This will in turn cause the session to be recreated on the secondary unit while it has been deleted on the primary. 

This ends up in the secondary unit having more sessions than the primary unit in edge cases, especially with UDP sessions. This behavior has now been fixed on v7.6.1 and is scheduled for a fix on v7.4.7 as per known issue ID 1064728. 
 

The new behavior will be to add a small delay to the secondary session deletion process to make sure when the query to verify session status is sent to the primary unit the session is already deleted which in turn causes the session to be deleted from the secondary unit. 
 
By default, session sync is done using the heartbeat devices. For better performance, it can be done using a dedicated link when dealing with a massive number of sessions to be synced: HA session-sync-dev configuration - Fortinet Community