Technical Tip: Packet capture and debug behavior in FortiGate HA vCluster Mode

Description

This article describes packet capture, debug flow, and session visibility behavior in FortiGate HA vCluster mode when a VDOM is active on the secondary HA member.

In Active-Standby HA with vCluster enabled, different VDOMs can be active on different cluster members. During troubleshooting, administrators may see sessions in the session table on the primary unit due to HA session synchronization, while packet capture and debug flow show no traffic.

This behavior is expected and occurs because traffic is only processed by the HA member that owns the VDOM.

Scope

FortiGate.

Solution

In the following example, FortiGate HA is configured in Active-Standby mode with vCluster enabled.

• VDOM-1 is active on the primary HA member.

• VDOM-2 is active on the secondary HA member.

During troubleshooting, administrators may observe that sessions related to VDOM-2 are visible on the primary firewall due to HA session synchronization. However, packet capture and debug flow commands executed on the primary unit may not show any traffic because the actual dataplane processing occurs on the secondary HA member that owns the VDOM.

When connected to the primary HA unit (FGT-A):

The following command may show the session:

diagnose sys session list

Explanation:

FortiGate HA synchronizes session tables between cluster members for failover purposes.

Because of this synchronization:

• Sessions can appear on both HA members.

• SYN and established sessions may be visible on the primary unit.

• The primary unit may not actually process the traffic.

Packet capture and debug flow are local dataplane operations and only display traffic processed by the local unit.

If the VDOM is active on the secondary unit:

• Traffic is processed only by the secondary device

• The primary device only contains synchronized session information

• Sniffer and debug flow on the primary unit will not show traffic

• Verify vCluster Ownership

Verify vCluster ownership:

Use the following command to identify the active HA member for each vCluster:

get system ha status

Or:

diagnose sys ha status

Sample Output:

FGT-A (global) # get system ha status

Primary selected using:
  virtual cluster 1:
  virtual cluster 2:
HA Health Status: OK
Model: FortiGate-101E
Mode: HA A-P
Group Name: GRP
Group ID: 0
Debug: 0
Cluster Uptime: 0 days 3:29:35

Configuration Status:
    FG101ETK00000001(updated 4 seconds ago): in-sync
    FG101ETK00000001 chksum dump: c4 30 04 20 30 ab ac b5 d9 e2 81 76 5e ca 7d ed 
    FG101ETK00000002(updated 5 seconds ago): in-sync
    FG101ETK00000002 chksum dump: c4 30 04 20 30 ab ac b5 d9 e2 81 76 5e ca 7d ed 

Primary     : FGT-A        , FG101ETK00000001, HA cluster index = 0
Secondary   : FGT-B        , FG101ETK00000002, HA cluster index = 1

number of vcluster: 2

vcluster 1: work 169.254.0.1
Primary: FG101ETK00000001, HA operating index = 0
Secondary: FG101ETK00000002, HA operating index = 1

vcluster 2: standby 169.254.0.2
Secondary: FG101ETK00000001, HA operating index = 1
Primary: FG101ETK00000002, HA operating index = 0


*******************************************************

FGT-A (global) # diagnose sys ha status
HA information
Statistics
        traffic.local = s:0 p:439870 b:83338712
        traffic.total = s:0 p:439927 b:83375724
        activity.ha_id_changes = 5
        activity.fdb  = c:0 q:0

Model=100, Mode=2 Group=0 Debug=0
nvcluster=2, ses_pickup=1, delay=0

[Debug_Zone HA information]
HA group member information: is_manage_primary=1.
FG101ETK00000001:      Primary, serialno_prio=0, usr_priority=128, hostname=FG101E-4
FG101ETK00000002:    Secondary, serialno_prio=1, usr_priority=100, hostname=FG101E-3

[Kernel HA information]
vcluster 1, state=work, primary_ip=169.254.0.1, primary_id=0
FG101ETK00000001:      Primary, ha_prio/o_ha_prio=0/0
FG101ETK00000002:    Secondary, ha_prio/o_ha_prio=1/1

vcluster 2, state=standby, primary_ip=169.254.0.2, primary_id=1
FG101ETK00000001:    Secondary, ha_prio/o_ha_prio=1/1
FG101ETK00000002:      Primary, ha_prio/o_ha_prio=0/0

Verify vCluster configuration:
To investigate the HA vCluster configuration and identify which VDOM belongs to each vCluster, use the following command:

show system ha

Sample Output:

FGT-A (global) # show system ha 
config system ha
    set group-name "GRP"
    set mode a-p
    set hbdev "ha2" 0 
    set session-pickup enable
    set session-pickup-connectionless enable
    set vcluster-status enable
    config vcluster
        edit 1
            set override enable
            set vdom "root" "VDOM-1"
        next
        edit 2
            set override enable
            set priority 130
            set vdom "VDOM-2"
        next
    end

Recommendation:

Before performing:

• Packet capture.

• Debug flow.

• Routing verification.

• Interface troubleshooting.

Verify which HA member owns the VDOM and perform troubleshooting directly on that node.

Note: In vCluster deployments, session visibility alone does not confirm that the local unit is processing traffic.

Related articles:

Technical Tip: Configure virtual cluster on FortiGate HA cluster

Technical Tip: HA Reserved Management Interface

Technical Tip: How to access the secondary unit from the primary with the 'execute ha manage' command.