Technical Tip: How to confirm that Load Balancing is occurring (HA cluster)

[dbabic]

Description
This article describes how to confirm whether load balancing is occurring in a FortiGate HA (high availability) cluster.
Methods used: packet sniffer and session list.
Solution
Background:

By default, only traffic passing through application proxies is subject to load balancing.
This is true for any UTM inspection in which a proxy-based (instead of flow-based) profile is used.  All other traffic will be handled only by the Master unit.

This behavior can be modified by using the following command:

#config system ha
set load-balance-all enable  #<--  now, all TCP traffic is load-balanced.

Sample Configuration:
For the example in this article, two units are configured for an HA A-A cluster in NAT/router mode:

#FGT60D4613****** # show system ha
config system ha
    set group-id 222
    set group-name "HA-act-act"
    set mode a-a
    set password ENC ZVLZUjueG5Rb6kWYzsnMFElBWK7QIGwTNsPPxruyal3TJTm/4madjZRQTkEANVltgDMlYXy81yMcKBQiKqzG0wWk+dtXzEe7QWgrnVtBNnJLIYpm5XM+x1YtwEF64swA1QMVZN4msYFravJ4YtfClOt+M0q8PtvlCPprsTcs8JgOKCNTk/qL9WahaWFfyANY/9HbFQ==
    set hbdev "dmz" 50 "wan2" 0
    set override disable
    set monitor "wan1"
    set load-balance-all enable
end

FGT60D4613****** # get system ha status
Model: FortiGate-60D
Mode: a-a
Group: 222
Debug: 0
ses_pickup: disable
load_balance: enable
load_balance_udp: disable
schedule: Round robin.
upgrade_mode: unset
master:128 FGT60D4613****** FGT60D4613****** 0
Slave :128 FGT60D4613****** FGT60D4613****** 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
master:0 FGT60D4613******
Slave :1 FGT60D4613******

How to Confirm Load-Balancing:

Specifically, if any traffic is being forwarded by one of the slave units, then load-balancing is occurring.

Method 1: Packet Capture

Open an SSH session to the subordinate (aka slave) unit.  And then start a packet capture.

FGT60D4613****** # execute ha manage 1
FGT60D4613****** $ diag sniffer packet any "port 80" 4

interfaces=[any]
filters=[port 80]
44.973557 internal in 192.168.1.101.49672 -> 65.52.62.25.80: syn 3070985030
45.040951 internal in 192.168.1.101.49674 -> 199.27.79.175.80: syn 625644188
45.139591 wan1 out 172.17.97.66.49672 -> 65.52.62.25.80: syn 3070985030
45.139955 wan1 out 172.17.97.66.49674 -> 199.27.79.175.80: syn 625644188
45.140247 wan1 in 65.52.62.25.80 -> 172.17.97.66.49672: syn 2597438395 ack 3070985031
45.140471 wan1 in 199.27.79.175.80 -> 172.17.97.66.49674: syn 1315870843 ack 625644189

Conclusion: Packets are arriving (internal in) and leaving (wan1 out).  The slave unit is therefore forwarding traffic -- i.e., load-balancing is occurring.

Method 2: Session List

Check the session list to determine whether sessions are being offloaded to the slave unit (ha_id=1):

FGT60D4613****** # diag sys session list | grep ha_id=1
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/
ha_id=1 policy_dir=0 tunnel=/

Conclusion: Sessions with ha_id=1 (slave unit's HA ID) indicate session is being handled by the slave.

Note: For clusters with more than 1 slave unit, ha_id=X where X=1 or higher (for the slave unit) will be shown.
To measure how equally traffic is being load-balanced, click on "View HA Statistics" in HA monitor of the FortiGate GUI.  It is then possible to compare network utilization for each cluster member.

Related Articles

IPS Load Balancing with HA

Technical Tip: How to enable TCP load balance in HA with active-active mode