Technical Tip: FGSP mismatched session count on peer members

[ssriswadpong]

Description	This article describes steps for troubleshooting a FortiGate Session Persistence (FGSP) cluster member where the session counts on the peer members are not the same.
Scope	FortiGate FGSP.
Solution	Follow the steps below for all members: Check system status and session count. get system status get system performance status Check on the debug application session sync for any indication of errors. diagnose debug reset diagnose debug application sessionsync -1 diagnose debug console timestamp enable diagnose debug enable Find an example of an affected session. For example, select one source IP as a filter, then check the session lists on both devices to see what session state the mismatched sessions are in (focus on proto_state). diagnose sys session filter src <x.x.x.x> diagnose sys session list Check on session sync stats, and session-sync-dev stats if sessions-sync-dev is in use.  diagnose sys session sync diagnose sys ha session-sync-dev A sample of command output that shows the issue: FortiGate1 and FortiGate2 are FGSP members, port5 and port6 are session-sync-dev. FortiGate1 # diagnose sys session sync sync_ctx: sync_started=1, sync_tcp=1, sync_others=1, sync_expectation=1, sync_nat=1, stdalone_sesync=1. sync: create=3180321718:655456, update=464463678, delete=0:0, query=280478267 recv: create=406936:0, update=899932, delete=122:0, query=947755304 ses pkts: send=2598787121, alloc_fail=0, recv=27325296, recv_err=4182392814, sz_err=0, ses_convert_err=0 FortiGate2 # diagnose sys session sync sync_ctx: sync_started=1, sync_tcp=1, sync_others=1, sync_expectation=1, sync_nat=1, stdalone_sesync=1. sync: create=0:0, update=0, delete=0:0, query=604402765 recv: create=2629341889:255085, update=3066393570, delete=782:0, query=534130878 ses pkts: send=29053159, alloc_fail=0, recv=564860091, recv_err=2150194712, sz_err=0, ses_convert_err=0 FortiGate1 # diagnose sys ha session-sync-dev HA sessync ports: 2 port5 probe: HA connected, Standalone probe, peer_mac = 00:00:00:00:00:00 HB pkts: rx=1, tx=4 SES pkts: rx=7876523, tx=5029328044 port6 probe: HA connected, Standalone probe, peer_mac = 00:00:00:00:00:00 HB pkts: rx=47, tx=4 SES pkts: rx=10278244, tx=3406628494 FortiGate2 # diagnose sys ha session-sync-dev HA sessync ports: 2 port5 probe: HA connected, Standalone probe, peer_mac = 00:00:00:00:00:00 HB pkts: rx=2, tx=1 SES pkts: rx=2645962705, tx=9986195 port6 probe: HA connected, Standalone probe, peer_mac = 00:00:00:00:00:00 HB pkts: rx=2, tx=47 SES pkts: rx=3406632869, tx=10278244 The above sample show the difference between update and receive, the error counter (recv_err), and the difference in the TX and RX values on port5 between the two devices. This indicates that there may be packet drops on port5, which could be causing the session sync issue.