Description
This article describes methods to troubleshoot intermittent HA (High Availability) synchronization issues in FortiGate devices using FortiLightHouse.
Scope
FortiGate.
Solution
FortiLightHouse CLI (FlhCli) is a monitoring utility designed to collect critical outputs from FortiGate firewalls. It gathers the necessary debug information to assist Fortinet TAC and Engineering teams in performing in-depth analysis.
Download the FlhCli tool on the system from the link below:
Once the binary file has been downloaded, open the Command Prompt by typing the commands in the search bar. Navigate to the download directory from the Command Prompt & execute flhcli.exe to start the application.
Run the command flhcli.exe setup to create the required XML configuration file.
A monitoring script is required to run against the predefined device.
A sample HA script. It can be adjusted as per the requirement.
<flhscript version="1">
<parameters name="default">
<list name="status">
<query type="pids">
<option name="process">cpu</option>
</query>
</list>
</parameters>
<settings name="default">
<ics>100ms</ics>
<cycleDuration>60s</cycleDuration>
</settings>
<cycle name="performance" parameters="default" settings="default">
<command context="global">get system ha status</command>
<command context="global">print global</command>
<command context="global">diagnose sys ha checksum cluster</command>
<command context="global">diagnose sys ha checksum cached global</command>
<command context="global">diag sys ha checksum cached root</command>
<command context="global">diagnose sys ha hasync-stats all</command>
<command context="global">diagnose sys ha sync-stats</command>
<command context="global">diagnose sys ha hasync-stats most-recent</command>
<command context="global">diagnose sys ha hasync-stats most-time</command>
<command context="global">diagnose sys ha hasync-stats most-time</command>
<command context="global">diagnose debug console timestamp enable</command>
<command context="global">diagnose debug application hasync -1</command>
<command context="global">diagnose debug application hatalk -1</command>
<command context="global">diagnose debug enable</command>
</cycle>
</flhscript>
Ensure that the monitoring script and the flhcli XML files are stored in the same directory.
The script output is saved in a file named HUBFGT_monitor.txt. The script is configured with a 30-second cycle interval, which can be adjusted as required.
Execute the command below to run the script.
flhcli -force-pty -redirect <<<device_name>>>_monitor --rotate-age 4h --name <<<device_name>>> cmd xml --xml <<script name>> --cycle performance
For example:
flhcli -force-pty -redirect HUBFGT_monitor --rotate-age 4h --name HUBFGT cmd xml --xml IPSEC_VPN.xml --cycle performance
Output can be viewed from the file HUBFGT_monitor:
