Technical Tip: FortiAnalyzer HA configuration and troubleshooting

Description

This article describes how each FortiAnalyzer node is assigned its own unique IP address (Primary and Secondary), while both share a common Virtual IP address. This shared VIP ensures seamless failover and continuous network availability through VRRP.

Scope

FortiAnalyzer.

Solution

• Source address: The primary IP address of the interface the packet is being sent.

• Destination IP address: 224.0.0.18 (Multicast IP address).

This is a link-local scope multicast address. Routers should not forward a datagram with this destination address, regardless of its TTL:

• TTL: 255.

• Protocol: The IP protocol number assigned by the IANA for VRRP is 112 (decimal).

• MAC Address in the following format: 00-00-5E-00-01-{VRID}.

Cluster settings:

• Peer IP: 10.123.5.198 (Secondary FortiAnalyzer IP address).

• Peer SN: FAZVMXXXXXX (Secondary FortiAnalyzer Serial Number).

 
Interface settings: 

• port2 and port1 must be set with the interface IPs, as the cluster settings do not automatically set them. 

• Primary - port2 = 10.123.5.189.

• Primary - port1 = 10.109.21.189.

• Secondary - port2 = 10.123.5.198.

• Secondary - port1 = 10.109.21.198.

• Group name: HA (any name can be given, but it should be the same on the primary and secondary units).

• Group ID: 9 (any number (1-255) can be given, but it should be the same on the primary and secondary units).

• Password: Fortinet@123 (any password can be provided, but it should be the same on the primary and secondary units).

• Heartbeat Interval: 4.

• Priority: 120 (Higher Value states as Primary device).

• Log Data Sync: Enable the toggle option.

On FortiAnalyzer-Secondary device: under System Settings -> HA -> Operation mode, select 'High Availability'.

Configure the following details:

Preferred Role: Secondary.

Cluster Virtual IP:

• Interface: port1.

• IP Address: 10.109.21.123 (make sure this IP is not used in the network).

Cluster Settings:

• Peer IP: 10.123.5.189 (Primary FortiAnalyzer IP address).

• Peer SN: FAZVMXXXXXX (Primary FortiAnalyzer Serial Number).

• Group Name: HA (Any name can be given, but it should be the same on the primary and secondary units).

• Group ID: 9 (Any number (1-255) can be given, should be the same on the primary and secondary units).

• Password: Fortinet@123 (any password, but it should be the same on the primary and secondary units).

• Heartbeat Interval: 4.

• Priority: 100 (lower value states as Secondary device).

• Log Data Sync: Enable the toggle option.

After configuring the primary and secondary units, this output on the GUI can be seen:

Important note:

When performing the initial synchronization of the HA cluster, only the Secondary node (the member with preferred-role = secondary or lower priority) must use the 'Initial-sync' setting (enabled). This is also confirmed by the Configuring HA options section of the admin guide.
If both nodes initiate the sync simultaneously, each node may attempt to push its own database to the peer, causing the HA pairing to fail or remain in 'Secondary-Sync' state.

Once the HA cluster has been formed, it can be accessed via the GUI using the cluster IP address. Here is an example of how it may look once logged in: it will show the hostname in the top-left corner of the page and the HA status in the top-right corner:

Troubleshooting commands:

On Primary-FortiAnalyzer:

diagnose ha status 

get system ha

 On Secondary-FortiAnalyzer:

diagnose ha status

get system ha

With these 2 commands, troubleshoot the status and the configured values for FortiAnalyzer-HA.

Restart the Cluster Process and collect its debug output:

After checking the configuration settings, restart the clustering process on both the primary and secondary devices.

diagnose test application clusterd 1
diagnose test application clusterd 99 

If the Active-Active HA (GEO redundant HA) is required, the following command must be used to enable the unicast HA:

config sys ha
    set unicast enable
end

If the initial logs sync has completed but the configuration has not synced, run the following command to sync the config:

diagnose ha force-cfg-resync

In later versions, the command may be:

diagnose ha force-resync

 
Collect the cluster debug output on both the primary and secondary devices.

diagnose debug application clusterd 255
diagnose debug enable 

Also, the following command will show the HA live connection status, like keep-alive messages:

diagnose ha dump-datalog 

Troubleshooting commands:

Run the top command before forming the HA to see all errors within the process, and note that 'execute shell' has been disabled in FortiAnalyzer v7.2.10/v7.4.5/v7.6.2.

diagnose debug app ha 255
diagnose debug enable

diagnose ha status
diagnose ha stats
diagnose test app cluster 1
diagnose test app cluster 3 conn
diagnose test app cluster 3 log
diagnose test app cluster 97 status

execute shell 

cat /etc/keepalived/keepalived.cfg
cat /drive0/private/clusterd/faz-ha.log
cat /drive0/private/clusterd/keepalived.log

diagnose sniff packet portX "vrrp" 3

Both nodes attempted initial sync: 

If initial sync is enabled on both nodes concurrently, the cluster may never form. The fix is to stop sync on one node (or reboot), then only enable initial sync on the secondary node.

Related documents: 

• Setting up a FortiAnalyzer HA cluster

• Troubleshooting Tip: FortiAnalyzer HA could not be performed with specific error message 'Firmware version mismatch'

• Technical Tip: How to configure FortiAnalyzer HA instance in Google Cloud Platform (GCP)

• Technical Tip: File system check on FortiAnalyzer in a High Availability (HA) setup

• Troubleshooting Tip: FortiAnalyzer HA configurations that will not synchronize

• Technical Tip: FortiManager HA cluster setup and troubleshooting