Troubleshooting Tip: HA devices out of sync after a firmware upgrade

While upgrading HA using Uninterrupted upgrade, both devices should upgrade simultaneously:

Uninterrupted upgrade

After the upgrade device may be out of sync when the following occurs:

1. Only one of the devices is upgraded to the next firmware version.

For example, if upgrading HA devices from v7.0.1 to v7.0.3, the primary gets upgraded to v7.0.3, and the secondary stays in 7.0.1, then:

1. Boot the primary device to the previous version by selecting the alternate firmware version to boot.
   Technical Tip: Selecting an alternate firmware for the next reboot
   
   Or:
   
   
2. Remove the secondary from the cluster (Technical Tip: Disconnect a member from a cluster (remove a device from an HA cluster)), upgrade the secondary device, and join it back to the cluster.

2. Due to configuration differences, recalculate HA checksum on both devices primary and secondary using: diagnose sys ha checksum recalculate.

    

Technical Tip: Troubleshooting a checksum mismatch in a FortiGate HA cluster

Run the following commands:

execute ha synchronize stop
diagnose debug reset
diagnose debug enable
diagnose debug console timestamp enable
diagnose debug application hasync -1
diagnose debug application hatalk -1
execute ha synchronize start

diagnose sys ha checksum recalculate

diagnose debug disable   --> To stop the debugs.

Allow a couple of minutes to verify the differences in the cluster.

3. Compare the configuration between the two FortiGates in the HA cluster and update the configuration.

Troubleshooting Tip: Allocate config disparity for HA out-of-sync

Procedure for HA manual synchronization - Fortinet Community

Note:
To confirm uninterruptible-upgrade is enabled, use the following command:

show full system ha | grep uninterruptible-upgrade

From FortiOS v7.4.1 and later, the option uninterruptible-upgrade has been replaced with upgrade-mode.

config system ha

    set upgrade-mode {simultaneous | uninterruptible | local-only | secondary-only}

end

The default setting for upgrade-mode is uninterruptible, which follows the same behavior as the previous set uninterruptible-upgrade enable. Similarly, the behavior of set uninterruptible-upgrade disable is now mapped to set upgrade-mode simultaneous.

Note:

HA is out of sync after upgrade to FortiGate VM OS version 7.2.12. (FortiGate VM Platform):

On the FortiGate VM HA cluster, the cluster becomes out of sync after upgrading to Forti VM OS version 7.2.12. This occurs due to configuration loss related to the heartbeat interface on the secondary (slave) unit.

The issue can be observed by running the following command:

get system ha status

Primary selected using:

HA Health Status: OK

Model: FortiGate-300E

Mode: HA A-P

Group Name: Cluster

Group ID: 47

Debug: 0

Cluster Uptime: 0 days 21:43:4

Cluster state change time: 2026-02-12 01:54:34

    <2026/02/12 01:54:34> vcluster-1: FGVMXXXXXXXXXX55 is selected as the primary because its override priority is larger than peer member FGVMXXXXXXXXXX56.

ses_pickup: disable

override: enable

Configuration Status:

   FGVMXXXXXXXXXX55(updated 2 seconds ago): in-sync

   FGVMXXXXXXXXXX55 chksum dump: 17 f2 5c 92 26 ad 53 96 17 5f 0c c9 67 b0 4a 31

   FGVMXXXXXXXXXX56(updated 0 seconds ago): out-of-sync

   FGVMXXXXXXXXXX56 chksum dump: 00 00 00 00 00 00 00 96 00 00 00 00 00 00 00 00

System Usage stats:

    FGVMXXXXXXXXXX55(updated 2 seconds ago):

        sessions=25, average-cpu-user/nice/system/idle=0%/0%/1%/98%, memory=30%

    FGVMXXXXXXXXXX56(updated 0 seconds ago):

        sessions=0, average-cpu-user/nice/system/idle=0%/0%/1%/98%, memory=29%

HBDEV stats:

    FGVMXXXXXXXXXX55(updated 2 seconds ago):

        ha: physical/1000auto, up, rx-bytes/packets/dropped/errors=215962142/485447/0/0, tx=444026837/614176/0/0

    FGVMXXXXXXXXXX56(updated 0 seconds ago):  <----- Config lost for the heartbeat interface for slave unit.

Solution:

Restart or kill the hasync and hatalk processes by executing the commands below:

fnsysctl killall hasync

fnsysctl killall hatalk

After restarting the relevant processes, the lost configuration on the secondary (slave) unit will be restored, and HA synchronization will resume.

Note that executing the above commands will trigger an HA failover to the secondary unit and may result in session drops. Therefore, it is strongly recommended to perform this procedure during a scheduled maintenance window.

Related article:

Troubleshooting Tip: How to troubleshoot HA synchronization issue using GUI and CLI on FortiGate/FortiProxy