Technical Tip: Understanding the HA uptime for Chassis based device.

[adecottignies_FTNT]

Description	This article describes the HA uptime in a chassis-based device, as well as the difference between device uptime and cluster uptime.
Scope	FortiGate 6000 and 7000.
Solution	For a chassis-based device (FortiGate 6000 and FortiGate 7000), the primary unit selection follows the following sequences: FortiGate 6000: Override Disabled: Any Active FPCs -> Connected Monitored Ports -> Number of Active FPCs -> HA uptime -> Priority -> Serial Number. To force a failover when override is disabled, reset the uptime from the current Master Unit with the following command: diag sys ha reset-uptime Override Enabled:  Any Active FPCs -> Connected Monitored Ports -> Number of Active FPCs -> Priority -> HA uptime -> Serial Number. To force a failover when override is enabled, change the priority to the current primary unit. From the CLI, on the current primary device: config global config system ha set priority <priority number> The unit with the highest priority will become the primary. FortiGate 7000: Override Disabled:  Any Active FPMs -> Number of active FIM -> Connected Monitors ports -> Number of active FPMs -> HA uptime -> Priority -> Serial Number. To force a failover when an override is disabled, reset the uptime from the current Master Unit with the following command: config global diag sys ha reset-uptime Override Enabled: Any Active FPMs -> Number of active FIM -> Connected Monitors ports -> Number of active FPMs -> Priority -> HA uptime -> Serial Number. To force a failover when override is enabled, change the priority to the current primary unit. From the CLI, on the current primary device: config global config system ha set priority <priority number> The unit with the highest priority will become the primary unit. As detailed above, the HA uptime is a tiebreaker when override is disabled, or if the priority is the same on both Chassis of the cluster. Device uptime, Cluster uptime and HA uptime are three different elements. The device uptime: It provides information on how long the member is up. This information can be displayed with the following command: get sys per status | grep Uptime As this command is broadcasted to all the slots of the Chassis, the outputs will be available for all slots. Example with 6500F: F6K (global) # get sys perf status | grep "Uptime\|SN" Slot: 1  Module SN: FPC6KFT111111111 Uptime: 1 days,  5 hours,  19 minutes Slot: 2  Module SN: FPC6KFT222222222 Uptime: 1 days,  5 hours,  19 minutes Slot: 3  Module SN: FPC6KFT333333333 Uptime: 1 days,  5 hours,  19 minutes Slot: 4  Module SN: FPC6KFT444444444 Uptime: 1 days,  5 hours,  19 minutes Slot: 5  Module SN: FPC6KFT555555555 Uptime: 1 days,  5 hours,  18 minutes Slot: 6  Module SN: FPC6KFT666666666 Uptime: 1 days,  5 hours,  18 minutes Slot: 7  Module SN: FPC6KFT777777777 Uptime: 1 days,  5 hours,  18 minutes Slot: 8  Module SN: FPC6KFT888888888 Uptime: 1 days,  5 hours,  18 minutes Slot: 9  Module SN: FPC6KFT999999999 Uptime: 1 days,  5 hours,  18 minutes Slot: 10  Module SN: FPC6KFT101010101 Uptime: 1 days,  5 hours,  18 minutes MBD SN: F6KF51T00000000 Uptime: 1 days,  5 hours,  19 minutes As for every broadcasted command, the last one concerns the MBD. The chassis is considered as working when at least one FPC is ready to handle the traffic. Consequently, with this output, this device is up for 1day, 8hours, 19 minutes. The Cluster uptime: It detailed how long at least one member of the cluster can handle the traffic. Even if a failover occurs, this time will not be reset. This information is available with the command: F6K (global) # get sys status | grep uptime Cluster uptime: 1 days, 3 hours, 16 minutes, 33 seconds In this example, the cluster was able to handle the traffic for 1 day, 3 hours, 16 minutes, 33 seconds. Ha uptime: This is a timer used for the election process of the primary device in a cluster A-P. This value detailed the time that a device has been primary without an event that would trigger a new election process. This information is available from the following command: F6K (global) # diagnose sys ha dump-by group …         pingsvr_flip_timeout/expire=3600s/0s         'F6KF51T000000001': ha_prio/o=0/0, link_failure=0, pingsvr_failure=0, flag=0x00000001, mem_failover=0, uptime/reset_cnt=98686/0, active_worker=10/10, chassis_id=1, set_as_primary=0         ' F6KF51T000000002': ha_prio/o=1/1, link_failure=0, pingsvr_failure=0, flag=0x00000000, mem_failover=0, uptime/reset_cnt=0/0, active_worker=10/10, chassis_id=2, set_as_primary=0 This output details that the device with SN F6KF51T000000001 is the primary unit and is able to handle the traffic for 98686 seconds. In other words, 1 day, 3 hours, 24 minutes and 46 seconds, without any event that could trigger an HA election. Note that by default, if there is less than 300 seconds of differences of HA uptime between the devices in an HA, this criteria will not be considered during the HA primary election. This value can be configured in the ha setting. F6K (global) # config system ha F6K (ha) # set ha-uptime-diff-margin <value> F6K (ha) # end Lab example 1: Override disable. F6K1 has been restarted recently. show system  ha | grep priority     set priority 200 show system  ha | grep priority     set priority 120 diagnose sys  ha dump-by group …         'F6KF51T000000001': ha_prio/o=1/1, link_failure=0, pingsvr_failure=0, flag=0x00000000, mem_failover=0, uptime/reset_cnt=0/0, active_worker=10/10, chassis_id=1, set_as_primary=0         'F6KF51T000000002': ha_prio/o=0/0, link_failure=0, pingsvr_failure=0, flag=0x00000001, mem_failover=0, uptime/reset_cnt=90035/0, active_worker=10/10, chassis_id=2, set_as_primary diagnose sys ha status … F6KF51T000000002:      Primary, ha_prio/o_ha_prio=0/0 F6KF51T000000001:    Secondary, ha_prio/o_ha_prio=1/1 This example detailed that, when the override is disabled, and as the difference of HA uptime is higher than 300 (ha-uptime-diff-margin), it is the HA uptime that is considered to choose the primary. Lab example 2 (same HA configuration): Override disable. F6K1 and F6K2 has been restarted recently. show system  ha | grep priority     set priority 200 show system  ha | grep priority     set priority 120 diagnose sys  ha dump-by group         'F6KF51T000000001': ha_prio/o=0/0, link_failure=0, pingsvr_failure=0, flag=0x00000001, mem_failover=0, uptime/reset_cnt=0/0, active_worker=10/10, chassis_id=1, set_as_primary=0         ' F6KF51T000000002': ha_prio/o=1/1, link_failure=0, pingsvr_failure=0, flag=0x00000000, mem_failover=0, uptime/reset_cnt=30/0, active_worker=10/10, chassis_id=2, set_as_primary= The uptime difference is less than 300 (ha-uptime-diff-margin), so this criteria is ignored for the HA primary election. Consequently, the priority is now considered. F6K1 (global) # diagnose sys  ha status … F6KF51T000000001:      Primary, ha_prio/o_ha_prio=0/0 F6KF51T000000002:    Secondary, ha_prio/o_ha_prio=1/1 F6K1 is therefore the primary unit. Related documents: Technical Tip: HA Master Election Process on FortiGate-6000/7000 Series HA uptime and system uptime different things? - Fortinet Support forums