| Description |
This article describes how to configure and troubleshoot issues with zero-touch provisioning of a HA FortiGate cluster. |
| Scope | FortiManager. |
| Solution |
Configure the following settings:
Note: If the HB interface is not configured, an error during the ZTP process (pushing of the configuration) will appear (illustrated in the Troubleshoot section):
Below is an example HA device model in the device manager panel:
The device is then added after all of the steps are validated:
Once validated, the HA model device must appear under the managed devices with a grey icon:
Ensure the FortiGate admin password is updated under the device database (the admin is required to change the password after the FortiGate resets). This new password must be manually added to FortiManager.
'Right-click' on the FortiGate model recently added, select Edit, enter the admin password, and select OK.
Next, select the FortiGate model and select Administrator -> Admin, select local password, enter the admin password (twice), and select OK.
Each of these methods has its own merits that will not be fully explored in this article, but the result is the same: a FortiManager IP/FQDN and Serial Number are added to the FortiGate configuration.
Note: The FortiManager SN must be configured in the FortiGate to authenticate the incoming FGFM management request in all of the ZTP scenarios.
In the example below, a batch mode script is used on FortiGate to configure the central management settings and the serial number, because the serial number cannot be set in normal CLI mode.
exe batch start config system central-management set type fortimanager set fmg 10.x.x.x <--- IP address or FQDN of the FortiManager. set serial "FMG-Serial-Number" <--- If the SN is not set, the FortiGate will be unable to authenticate the FortiManager request. end
exe batch end
Note: The batch script must be run on both FortiGates as the ZTP task would otherwise not be triggered on FortiManager. Once the script has been run on the first FortiGate, the FortiManager will wait for all secondary nodes to start the auto-link task (see the screenshot below).
exe ping <FortiManager Management ip>
exe telnet <FortiManager Management ip> 541
Before running the Batch script or using another deployment method, run the CLI commands below. These commands will help monitor the ZTP process and collect debug output to help troubleshoot issues during the provisioning process.
The operation is similar to HA model device operation and standalone FortiGate.
On FortiManager:
diag debug service sys 255 diag debug application depmanager 255 diag debug enable diag fwmanager fwm-log diag debug application fgfmsd 255 <fortigate_device_name>
On FortiGate:
diagnose debug cli 8 diag debug application fgfmd 255 diag debug enable
Below is an example of debug traces when an HA setup task fails. If this, it means the Heartbeat interfaces have not been set into the HA model device, which leads to a HA setup failed task:
Example of debug traces when auto-link and push config to device operation fail:
checkin_sched: check in configuration __checkin_handler: new rev=1 ==INST CONF==>install done state, finish! ==INST CONF==>install and save finished status=FAILED Release token oid=160, pid=867.0x26d81b6c. Response: { "id": 1, "result": [{ "status": { "code": 0, "message": "OK"}, "url": "acquire\/dvm\/token"}]} __sched_inst_save_queue,565: cur_inst=1, max_inst=480. destroy_service.59:mark sconn 0x268bb3c0 done. destroy_service.59:mark sconn 0x268bb3c0 done. Destroy sconn 0x268bb3c0, connSize=0.
Under System Settings -> Task Monitor, the two operations have a 'failed' status: it is possible to get more details on the error(s) by also selecting the 'Push config to device' task.
Under Device Manager, the configuration is in conflict status:
The details of errors when the provisioning task is completed will provide enough information to find the problem:
Start installing FortiGate-VM64-KVM $ config system interface FortiGate-VM64-KVM (interface) $ delete "port10" Physical interfaces cannot be deleted. command_cli_delete:6711 delete table entry port10 unset oper error ret=-3 Command fail. Return code -3 FortiGate-VM64-KVM (interface) $ delete "port9" Physical interfaces cannot be deleted. command_cli_delete:6711 delete table entry port9 unset oper error ret=-3 Command fail. Return code -3 …………. Port 8, 7……………………….. FortiGate-VM64-KVM (interface) $ end FortiGate-VM64-KVM $ config system global FortiGate-VM64-KVM (global) $ set hostname "FGVM01TMxxxxxxx" FortiGate-VM64-KVM (global) $ set alias "FortiGate-VM64" FortiGate-VM64-KVM (global) $ end FGVM01TM22-----9$ config system admin FGVM01TM22-----9(admin) $ edit "admin" FGVM01TM22-----9 (admin) $ unset password incomplete command in the end Command fail. Return code -160
Issues may appear during the deployment of the FortiGate configuration.
Most failures occur because either the password is not the same on the device model and/or because the device model does not contain the same number of ports as the real FortiGate.
By default, FortiManager tries to flush the delta configuration and warns it is not possible to install this configuration on the FortiGate. After this occurs, the configuration status of the newly added FortiGate becomes a 'conflict'. (Capture shown above.) The above example illustrates issues with the admin password and the number of interfaces.
To fix the interface issue (as manually creating an interface is not possible through the device database GUI), it is necessary to create a script applied to the device database.
Below is a script example that may be used if this issue appears:
In the HA Zero Touch Provisioning section, push the script on all nodes of the HA cluster:
Below, all of the missing interfaces have been correctly created by the script. All nodes of the HA model device:
The batch script can then be run again on the FortiGate side to trigger the auto-link and 'push configuration to device' tasks.
During the ZTP HA operation, three tasks are created instead of two (compared to ZTP for a standalone FortiGate model).
The first step of that Zero Touch Provisioning process is the configuration of the HA cluster. FortiManager will use the settings entered during the HA model device step to set up and build the FortiGate HA cluster. Using the troubleshooting cli commands, it is possible to observe that step in detail:
HA cluster configuration pushed to the real devices shown in CLI (troubleshooting commands are used on FortiManager to see all the HA configuration):
During the HA setup process, the secondary node of the HA cluster will synchronize the configuration from the primary node. In that way, the secondary node FortiGate will receive the HA configuration to build the HA cluster. The HA setup task must appear as successful under System Settings -> Task Monitor:
Using the troubleshooting command (diag debug application depmanager 255), it is possible to see in the CLI that the HA cluster is correctly built:
The auto-link and push configuration to device tasks will be triggered and the ZTP operation should be successful.
Go to Device Manager and check the configuration status of the HA FortiGate cluster. This one must be synced.
Finally, go to Device Manager and confirm if the configuration status is 'synchronized'. Notice the IP address and all information have been automatically set to build the HA cluster.
If a deployment failure occurs, send the install log to the TAC team:
Select View Install Log and Install Log.
Related document: Technical Tip: ZTP basic configuration and troubleshooting for a standalone FortiGate Adding a model FortiGate HA cluster DOCS: Auto-link setting is exposed to control configuration installation during ZTP 7.4.1 DOCS: Zero-touch and low-touch provisioning DOCS: Perform installation to apply Jinja template configurations to branches |
