Skip to main content
gmanea
Staff
gmanea
Staff
January 24, 2020

Technical Tip: How to revert HA cluster unit to the previous firmware image

  • January 24, 2020
  • 0 replies
  • 27220 views

Description

 

This article describes how to revert FortiGate to the previous firmware image when using an HA cluster, including steps specific to HA.

 

Scope

 

HA with FortiGate physical appliances. FortiGate Virtual Machines do not have the dual boot option.


Solution

 
In some cases, firmware upgrades may cause unexpected issues. In this case reverting to the previous image is often a fast fix worth considering and is strongly recommended instead of downgrading the firewall.
 

With the first upgrade, a physical FortiGate device creates a second boot partition. When FortiGate firmware is upgraded, the new firmware image is stored on the new partition, while the previous firmware image remains stored on the existing partition as a backup image. FortiGate virtual machines do not maintain separate boot partitions - an alternative for VMs is to create snapshots before the upgrade.

 

Reverting a FortiOS HA cluster to previous firmware and configuration:

 

  1. Determine if reverting the cluster to previous firmware and configuration is a reasonable step to take by considering the following conditions:
    • Whether all HA cluster members have the same firmware version currently.
    • Whether all HA cluster members had the same firmware version before the upgrade.
    • Whether all devices in the same HA cluster were in the previous firmware version/configuration?
    • Whether there are no significant configuration changes made on the current firmware version that could cause a network outage or lost management access if they were reverted. Examples include aggregate interface configuration, routing, IPsec tunnel changes, IP address changes, and HA password or group-id.
    If all of the above conditions are met, the cluster may be reverted with the following steps. Immediately after a successful firmware upgrade of an HA cluster, all of these conditions will be true.

  2. Login to primary FortiGate management GUI with an administrator with the super_admin profile and take a global configuration backup as shown in Configuration backups and reset.
  3. Login to the secondary cluster member using serial console, HA reserved management interface, or 'execute ha manage' command from the primary member, see the article Technical Tip: Managing individual cluster units with the CLI command 'execute ha manage'. Serial console or HA reserved management interface access are preferred.
  4. On the secondary member, find the current active partition using the 'diagnose sys flash list' command.

    diagnose sys flash list
    Partition      Image                       TotalSize(KB)         Used(KB)          Use%      Active
    1   FGT61E-7.02-FW-build1517-230606            253920             102716            40%          Yes
    2   FGT61E-7.02-FW-build1262-221109            253920              98304            39%          No
    3   ETDB-90.06786                             3021708             232936             8%          No
    Image build at Jun 6 2023 16:47:58 for b1517

    In the output above, partition 1 is the active partition and holds the current firmware image FortiOS v7.2.5, while partition 2 is on v7.2.3 and holds the previous firmware. Partition 3 never contains firmware and can be ignored.
  5. On the secondary HA member, set the desired active partition using the 'execute set-next-reboot' command, see the article Technical Tip: Selecting an alternate firmware for the next reboot.

     

execute set-next-reboot {primary | secondary}

 

Example output:

 

execute set-next-reboot
primary partition
secondary partition

 

Note: In the context of the 'execute set-next-reboot' command, 'primary' and 'secondary' refer to partition number 1 or partition number 2 respectively and have no relation to HA 'primary' or 'secondary' members.

  1. Repeat steps 4 and 5 on the primary HA member. Note that depending on upgrade history, HA primary and secondary members may have a different partition flagged as active. This is expected.
  2. On the secondary HA member, reboot the device using the 'execute reboot' command.
  3. (Optional) Wait for the secondary HA member to rejoin the cluster. Note that a cluster containing HA members with different firmware versions will always show as out-of-sync, and this is expected. Current cluster members can be viewed in the GUI (Go to System -> HA), or in the CLI (Using the command 'get system ha status').
  4. Reboot the primary HA member using the 'execute reboot' command. Note that if both cluster members are rebooted at the same time, there will be a network outage until at least one cluster member boots successfully.
  5. The units will boot with the newly selected firmware image, and the Active unit will be selected according to the FortiOS HA election process. See the article Technical Tip: FortiGate HA Primary unit selection process when override is disabled vs enabled.
  6. Once both units have the intended firmware version, verify network functionality. Note that it can take up a few minutes for HA configuration to sync immediately after boot, even if configuration was synced before.
 
Notes:
  • The commands above are not synchronized and must be used on every FortiGate unit in the cluster.
  • It is strongly recommended to keep the serial console available and have physical access to the device during the firmware reversion process.
  • If serial console access is not available, HA reserved management interface can be used if it was configured on the previous firmware version. If neither serial console access nor reserved management interface are available, 'execute ha manage' can be used. In either case, direct physical access remains strongly recommended.
  • Booting to backup partition reverts all configuration changes applied since the last firmware upgrade.
 
Related articles:
Terms & ConditionsAccessibility statement