Troubleshooting Tip: Restore Cluster HA when there are many differences in tables

[pabarro]

Description	This article describes HA Cluster is out of sync because there are too many checksum differences between different tables or simply because a FortiGate in the cluster is behaving abnormally and is not synchronized in any way. This procedure shown is to recover a FortiGate that belongs to the HA Cluster. The procedure is based on when the override is disabled.
Scope	Any FortiGate up to 3K from v6.0 up to v7.6.
Solution	For test purposes, FortiGate-500E v7.0.15, build0566, 231024 is used. Confirm if override is disabled by checking the sample below (the command used is 'get sys ha status'): Hostname: Fortigate-A FG-SERIALXXX_A Primary Priority 200 FortiGate-500E v7.0.15,build0566,231024 (GA.M) Override disable Mode: Active - Pasive -------------------------- Hostaname: Fortigate-B FG-SERIALXXX_B Secondary Priority 100 FortiGate-500E v7.0.15,build0566,231024 (GA.M) Override disable Mode: Active - Pasive Preparation: A person must be on-site to be able to connect to the devices. Make a full backup of the Primary (Fortigate-A). Download the FortiOS FortiGate-500E v7.0.15,build0566,231024 (GA.M) image (for the FortiGate-500E hardware) to do a clean install of the Secondary (Fortigate-B). Activity Summary: Currently, Fortigate-B has many problems and let's perform the procedure on this device. Perform a Flash Format on Fortigate-B. Load the required image FortiGate-500E v7.0.15,build0566,231024 (GA.M), load a backup, edit it, add it to the cluster, and resynchronize the HA. Technical Tip: Formatting and loading FortiGate firmware image using TFTP Procedure: Make a full backup of Fortigate-A (Active FortiGate with serial terminal FG-SERIALXXX_A). Proceed to download from the support website (https://support.fortinet.com) the FortiOS required for the hardware. In this case, FortiGate-500E v7.0.15, build0566,231024 (GA.M) Make sure it is for the hardware (FortiGate-500E). Save it in a folder to avoid confusion with any other version. Disconnect (all cables) from the HA cluster the Fortigate-B. (Secondary Fortigate with serial terminal FG-SERIALXXX_B). Leave only the Fortigate-A connected (Active FortiGate with serial terminal FG-SERIALXXX_A). This equipment will remain operational to avoid service affectation. See Figure 1:                                   Note: If the cables are not labeled, proceed to identify them to avoid confusion when reconnecting them later. Work with the FortiGate that is disconnected from the HA, that is, Fortigate-B (Hardware with serial number FG-SERIALXXX_B). Proceed to perform a flash format and load the FortiGate-500E v7.0.15,build0566,231024 (GA.M) image as indicated in the link shared above. Once step 6 is completed and with the unit up, log in to the FortiGate via GUI. Load the Backup made in Step 1 to this FortiGate. Once the backup is loaded, the Fortigate will reboot. Log back into the Fortigate and Edit the Hostname and basic HA configuration via cli as follows: config system global     set hostname Fortigate-B end config system ha     set priority 100 --> Let's set the value to 100, originally 200. end                      --> Save the changes. Check from CLI that the changes have been taken. show full system global | grep hostname  --> Fortigate-B should be the hostname show full system ha | grep priority      --> 100 should be the Priority Once these configuration settings have been made and verification has been performed, physically connect the secondary device (Fortigate-B) to the HA cluster and wait for it to sync. See Figure 2: If the units DO NOT sync, open a ticket with support and call support for immediate assistance.