Description
This article describes a process for disconnecting a FortiGate unit from an existing High-Availability (HA) cluster using the 'Remove device from HA cluster' feature (or execute ha disconnect via the CLI). This article will include steps for completing this process using the GUI for v5.2/5.4 as well as FortiOS 5.6 and beyond (6.x, 7.x, etc.) as well as via the CLI.
This process can generally be completed without disrupting the rest of the network, as it will make the following changes on the removed FortiGate during the removal process:
- Remove/clean all interface IP address configurations. Existing interfaces will be retained, but all IP addresses and administrative access options assigned to the interface will be removed to prevent conflicts with the remaining HA cluster.
- Configure a new IP address on a specified port. This is done so that the removed unit is accessible over the network for further management/reconfiguration.
- Change the removed FortiGate's HA mode to standalone.
The removed FortiGate will retain the vast majority of its current configuration when removed from the cluster, including but not limited to:
- Firewall Policies.
- Interfaces/names (but as noted above, not IP address assignments).
- Admin users/passwords (including REST API admins/API keys, SSO Admins, etc.).
- Security Inspection profiles.
- HA configurations (e.g. group names, passwords, etc., but notably not HA management interfaces or the mode).
This means that it is relatively straightforward to put a removed FortiGate back into the HA cluster (though see the notes further below for additional considerations).
Note:
It is still recommended to plan the removal process around a scheduled maintenance window, and it is also not recommended to use this option in cloud environments, as devices often have different IP addresses assigned to each node.
Scope
FortiGate High Availability, v5.2/5.4, v5.6 through v6.x/v7.x and later.
Solution
Disconnecting a FortiGate from the HA cluster - GUI Method (v5.6 through v6.x/v7.x and later):
1. Log in to the Primary FortiGate via the Web GUI.
2. Navigate to System -> HA (in the Global VDOM, if VDOMs are enabled). A list of FortiGate cluster members will be present.
3. Select the unit to disconnect, then select the Remove device from HA cluster button:
Select an Interface from the drop-down list after the pop-up window appears. This interface will be used as a management interface for the removed FortiGate:
Configure an IP/Netmask for the specified interface. This IP address should be reachable over the network so that the removed FortiGate can be managed, as all other interfaces on the removed FortiGate will have the IP addressing removed/cleared to avoid conflicts with the existing HA cluster.
Select OK to commit the change. Once the change is committed, the cluster will no longer show the unit as being connected to the cluster, and the disconnected unit's HA mode will be changed to standalone. If the current FortiGate (where the GUI session is logged in) is selected to be disconnected, the change will be executed locally to remove all IP addresses and configure the management interface. If the other unit in the HA is to be disconnected, the change to remove all IP addresses will be pushed to the other unit.
Note:
If the disconnected FortiGate is going to be reconnected to the cluster, consider the following:
- Reboot the disconnected unit before proceeding with cluster re-establishment, or run diagnose sys ha reset-uptime. This will ensure that the cluster uptime value will be lower than the current HA Primary FortiGate, which helps to ensure that the disconnected unit does not attempt to assume the Primary HA role.
- Assign a Device Priority that is lower than the existing Primary FortiGate's priority for the same reason mentioned above (the highest priority can lead to a unit assuming the HA Primary role, default value is 128).
- For example, If the active Primary unit has a priority of 100, then it is recommended to set the disconnected FortiGate priority to a value lower than that.
- Re-configure any other custom HA configuration that was present before the disconnect, such as dedicated HA management interfaces.
- Once the cluster is reconnected, it will take several minutes to fully synchronize the configuration and bring all interface configurations with it.
Disconnecting a FortiGate from the HA cluster - CLI Method (all FortiOS versions):
To disconnect a FortiGate the the HA cluster via the CLI, use the following command:
execute ha disconnect <FortiGate Serial Number> <Interface> <IP Address> <Netmask>
Consider the following example of the above command:
execute ha disconnect FGT800Dxxxxxxxx internal 192.168.1.2 255.255.255.0
The above command will disconnect the FortiGate with the serial number FGT800Dxxxxxxxx, and it will also set up the internal interface with an IP address of 192.168.1.2 and a network/subnet mask of 255.255.255.0. Same as GUI, if the FGT800Dxxxxxxxx is the FortiGate where the current CLI session is connected, the change will be executed locally to remove all IP addresses and configure the management interface. If it is the other unit in the HA, the change to remove all IP addresses will be pushed to the other unit.
As mentioned above, the existing HA cluster members will continue to provide service to the network without disruption, and the disconnected FortiGate will be accessible via the above interface and IP address (assuming routes exist as well to reach remote destinations).
Disconnecting a FortiGate from the HA cluster - GUI Method (Legacy v5.2/v5.4):
1. Log in the to Web GUI of the Primary FortiGate.
2. Navigate to System -> Config -> HA. A list of FortiGate cluster members will be present.
3. Select the unit to disconnect, then select the disconnect button/icon:
In the Disconnect Cluster Member window, specify the Interface to be used for management access to the disconnected unit.
Configure an IP/Netmask for the specified interface. This IP address should be reachable over the network so that the removed FortiGate can be managed, as all other interfaces on the removed FortiGate will have the IP addressing removed/cleared to avoid conflicts with the existing HA cluster.
Select OK to commit the change. Once the change is committed, the cluster will no longer show the unit as being connected to the cluster, and the disconnected unit's HA mode will be changed to standalone.
Important Note:
When a FortiGate disconnects itself from an HA cluster, it removes interface IP addresses using the CLI command unset ip. However, IP removal may fail if an interface has configuration dependencies. In such cases, the interface IP will remain configured after the unit becomes standalone. For example below, if an interface IP is referenced by other features (e.g., BFD or OSPF), the removal will fail:
config system interface
edit "port1"
set vdom "root"
set ip 10.56.242.195 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
set snmp-index 1
next
end
config router bfd
config neighbor
edit 10.56.242.129
set interface "port1"
next
end
end
The interface IP address of port1 will not be removed. Attempting to unset the IP manually shows the dependency error:
yoda-kvm06 (port1) # unset ip
This interface is used in router.bfd.neighbor.
Please delete the bfd neighbor there first.
Command fail. Return code -23
This interface is used in router.bfd.neighbor.
Please delete the bfd neighbor there first.
Command fail. Return code -23
In this scenario, the IP address on port1 remains after HA disconnection. If network cables remain connected, this may result in IP conflict or a split-brain condition.
Thus, after disconnecting a unit from the HA cluster, it is suggested to:
-
Log in to the device via new management IP or console.
-
Verify that all interface IP addresses have been successfully removed.
-
For any IP address that remains:
-
Remove all related configuration dependencies and unset the IP, or
-
Perform a factory reset on the unit.
-
From v7.6.1, a new feature has been added in the CLI console to display which interfaces failed to get IP address removed when executing the CLI command on the FortiGate console to get itself disconnected from the HA cluster.
For example, below command is executed on the FortiGate to disconnect itself from HA. This warning clearly indicates that port1 still has an active IP address and requires manual cleanup.
201G-B-4627 # execute ha disconnect FGVM02TM2001363x mgmt 172.18.14.99 255.255.255.0
Starting disconnect self from HA cluster.
[__cli_action_hidden_entry:112] action error with hidden entry -23
failed to change interface port1 ip(-23)
WARNING: To avoid IP conflict, you need to manually change/unset the above interfaces' IP on the disconnected FGT.
But there are certain limitations for this feature in v7.6 branch that:
-
The warning message is only shown on the unit disconnecting itself.
-
If a unit disconnects another device from the cluster, the warning will not appear.
-
This visibility issue will be addressed in FortiOS 8.0.0.
Additionally, when the disconnect command to disconnect itself is executed through an IP-based CLI session (SSH or GUI CLI):
-
The warning does not appear
-
Because the FortiGate removes all IP addresses during the process, which causes the session drops before the warning can be displayed.
Related articles:
Troubleshooting Tip: FortiGate Cluster upgrade gets stuck when not all members have upgraded.
Technical Tip: How to confirm that Load Balancing is occurring (HA cluster).
Technical Tip: How to view the routing table on Slave/Secondary/Subordinate units in HA cluster.
Technical Tip: HA Reserved Management Interface's hidden VDOM (vsys_hamgmt VDOM).
Technical Tip: How to break a HA cluster and use one of the members as standalone
Troubleshooting Tip: Backup & Restore Cluster HA when there are many differences in tables
