Description
This article describes how to upgrade an ADOM on FortiManager and how to perform basic troubleshooting in case of an ADOM upgrade failure.
Scope
FortiManager versions between v5.4.x andv 7.6.x.
Solution
Prerequisites and Notes:
ADOM upgrade requires system-level administrator permissions and access to the respective ADOM/s (eg., Super_User admin profile).
In the firmware versions within the scope of this article (v5.4.x to v7.6.x), an ADOM can only be upgraded after all the devices within this ADOM have been upgraded.
Starting in v7.0.1, the ADOM version can be upgraded without first updating all devices and the ADOM can manage N+1 devices during migration. This is something called mixed mode or migration mode.
Starting in v7.4.2, it is possible to have one ADOM in 7.4 that can manage FortiOS in v7.0, v7.2, and v7.4.
Starting in v7.4.6, managing any FortiGate version in any ADOM is now possible.
When a FortiManager unit is upgraded, ADOMs are not upgraded automatically. The ADOM upgrade operations have to be done separately after the FortiManager upgrade.
Although it is possible to manage FortiGates with different versions within the same ADOM, there are a few limitations:
- 'Import Policy' could be limited if the FortiGate version is different than the ADOM version:
- Configuration features implemented in the newer FortiGate version may not be available in the older ADOM version.
- There might be a mismatch in the CLI syntax of some ADOM objects, causing installation or verification errors (eg., new syntax implemented in FortiOS which is not available in the database of older ADOM versions).
- Go to System Settings -> All ADOMs.
The same view in CLI:
-
Select an ADOM and select 'Upgrade', or select an ADOM, select 'More', and then 'Upgrade' from the toolbar.
Select 'OK' in the confirmation dialog box to upgrade the device.
In the above/below picture the ADOM has been successfully upgraded. The new ADOM version is then displayed in the 'Firmware Version' column.
Under v6.4 and above select the ADOM that will be upgraded and go to More -> Upgrade
If all units within the ADOM are not already upgraded, the upgrade will be stopped and an error message will be shown.
The example below illustrates the failed ADOM upgrade: 'Please upgrade all devices to 5.6 before upgrading the ADOM'.
In some cases, (if all the FortiGate into the ADOM are upgraded), the ADOM procedure may fail for many reasons. Find below the different reasons preventing a customer from upgrading an ADOM (Non exhaustive list):
- Name conflicts in wildcard FQDN address on SSL/ssh profile (for ADOM 5.6, 6.0, and 6.2).
- Different CLI syntax on objects/profiles.
- Miscellaneous inconsistencies on firewall objects.
FortiGate object table size limitation changes.
All of these errors mentioned above are known fixed bugs and still exist in the customer environment because the ADOM has never been upgraded in the past even though the FortiManager is using the latest firmware version. Different firmware versions have different features, and therefore different CLI syntax. That is the reason why the ADOM upgrade might not be successful.
FortiManager Debug commands.
In such a case, Fortinet recommends using the below CLI troubleshooting commands for FortiManager versions before v7.4.x:
diagnose debug service cdb 255
These CLI commands will help to localize and identify the root cause of the problem that prevents to upgrade of the ADOM. In most cases, removing the concerned object/profile/interface allows to fix the issue and successfully upgrade the ADOM. Another scenario can happen: many errors are preventing to upgrade of the ADOM.
Find the first error, then fix it and try to upgrade the ADOM: without success. In such a case, use the same method and CLI commands to identify the object/profile/interface causing the problem.
If the concerned object is used and/or important in the configuration (cannot be modified), contact Fortinet support for further assistance. The ADOM upgrade debugging will always stop on the concerned error.
Below are some examples of FortiManager debugging after a failed ADOM upgrade:
Example 1:
--> commit copy firewall address.autoupdate.opera.com(soid=149) to dparent=1227, fail: err=-2, Name conflicts with an entry in wildcard FQDN address
name: autoupdate.opera.com ---> autoupdate.opera.com
subnet: 0.0.0.0 0.0.0.0 ---> 0.0.0.0 0.0.0.0
type: fqdn ---> fqdn
start-ip: 0.0.0.0 ---> 0.0.0.0
end-ip: 0.0.0.0 ---> 0.0.0.0
fqdn: autoupdate.opera.com ---> autoupdate.opera.com
associated-interface: any ---> any
wildcard: 0.0.0.0 0.0.0.0 ---> 0.0.0.0 0.0.0.0
cache-ttl: 0 ---> 0
color: 0 ---> 0
visibility: enable ---> enable
uuid: 2fe03af0-43b8-51ea-1233-d6844b291acd ---> 2fe03af0-43b8-51ea-1233-d6844b291acd
allow-routing: disable ---> disable
obj-id: 0 --->
Explanations of the previous error: By default, in 6.0 ADOM some firewall addresses have the same name as wildcard FQDN i.e.: 'autoupdate.opera.com', 'google-play', etc.
When upgrading to v6.2, it will hit the newly added check of not allowingthe firewall address to have the same name as a wildcard FQDN.
copy firewall address.autoupdate.opera.com(soid=149) to dparent=1227,: fail.
-> commit copy ssh.(soid=12315) to dparent=13106, fail: err=-2,Must set at least one port or enable ssh inspect-all.
======= Dump sentry and dentry======
12315 ---> 13113
status: deep-inspection ---> deep-inspection
inspect-all: disable ---> disable
unsupported-version: bypass ---> bypass
ssh-policy-check: disable --->
ssh-tun-policy-check: disable ---> disable
ssh-algorithm: compatible ---> compatible
===================================
copy ssh.(soid=12315) to dparent=13106, :fail.
copy firewall ssl-ssh-profile.custom-deep-inspection(soid=11436) to dparent=12315, :fail.
Global Database Specifics:
Only the 'Upgrade' option should be used for upgrading the Global Database to a higher version.
If the Global Database version is edited (modified) by selecting 'Edit' and then selecting either a higher or lower version, all data is deleted.- Go to System Settings -> All ADOMs.
- Select Global Database -> 'More' from the top menu bar -> Upgrade.
* If the ADOM has already been upgraded to the latest version, this option will not be available. - Select 'OK' in the Upgrade ADOM dialog box.
- After the upgrade finishes, select 'Close' to close the dialog box.
Related documents:
