Troubleshooting Tip: Partial Configuration Loss after Upgrade from FortiOS v7.4.11 and FortiOS v7.6.6

Description

This article describes an issue where system global settings, including hostname, time zone, admin-SCP, VDOM mode, and 'switch-controller' settings, may be lost after upgrading from ForitOS v7.4.11 to v7.4.12, or from FortiOS v7.6.6 to v7.6.7. The loss of these settings can cause the system to revert to default or unintended configurations, potentially affecting device identification, time synchronization and logging, administrative access through SCP, VDOM operation and management, and 'switch-controller' functionality.

Scope

FortiOS v7.4.11, v7.4.12, v7.6.6, v7.6.7.

Solution

The issue has been identified as caused by the configuration 'web-svc-auto-restart' set to enabled under system global settings.
This configuration was introduced as a workaround to the high memory issues caused by Node.Js, as discussed in this article. Troubleshooting Tip: High memory usage of node process.
The workaround discussed in this article does not impact the workaround for the Node.Js issue as the setting is recommended to be disabled temporarily before upgrade and to be re-enabled post upgrade.

config system global
      set web-svc-auto-restart enable
  end

Affected versions:

• FortiOS 7.6.6 -> 7.6.7 GA.

• FortiOS 7.4.11 -> 7.4.12.

The following configuration file snippet shows the difference between before upgrade and after upgrade system global settings.

Before upgrade:

#config-version=FG200F-7.6.6-FW-build3652-
#conf_file_ver=6199549070009958
#buildno=3652
#global_vdom=1
config system global
    set admin-forticloud-sso-login enable
    set admin-scp enable
    set admin-server-cert "Fortinet_Factory"
    set admin-sport 4433
    set admintimeout 15
    set alias "FG200Fxxxxxxxxxx"
    set application-bandwidth-tracking enable
    set autorun-log-fsck enable
    set gui-auto-upgrade-setup-warning disable
    set gui-device-latitude "xxx"
    set gui-device-longitude "xxx"
    set hostname "FW01"
    set lldp-transmission enable
    set log-uuid-address enable
    set miglogd-children 15
    set proxy-auth-timeout 5
    set switch-controller enable
    set timezone "US/Eastern"
    set web-svc-auto-restart enable
end

After upgrade:

#config-version=FG200F-7.6.7-FW-build3704-
#conf_file_ver=826270103563501
#buildno=3704
#global_vdom=1
config system global
    set alias "FortiGate-200F"
    set hostname "FortiGate-200F"
    set timezone "US/Pacific"
end

Issues observed post upgrade (but not limited to):

• When FortiGate is managing FortiSwitches, post upgrade, all FortiSwitches could appear offline due to the 'switch-controller' settings changed to 'disabled'.

• Administrative access is also affected due to the management port being reset.

Sample config error output:

FortiGate-200E (global) (Interim)# diagnose debug config-error-log read

>>> "next" @ global.system.interface.npu0_vlink0:failed command (error -56)
>>> "next" @ global.system.interface.npu0_vlink1:failed command (error -56)
>>> "next" @ global.system.interface.npu1_vlink0:failed command (error -56)
>>> "next" @ global.system.interface.npu1_vlink1:failed command (error -56)
>>> "set" "image-base64" "" @ global.system.replacemsg-image.logo_fw_auth:value parse error (error -56)
>>> "set" "image-base64" "" @ global.system.replacemsg-image.logo_v2_fnet:value parse error (error -56)
>>> "set" "image-base64" "" @ global.system.replacemsg-image.logo_v2_fguard_wf:value parse error (error -56)
>>> "set" "image-base64" "" @ global.system.replacemsg-image.logo_v2_fguard_app:value parse error (error -56)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABC9C ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_RSA2048:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABDzg ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_DSA1024:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABD5a ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_ECDSA256:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABAYn ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_ECDSA384:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABAMY ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_ECDSA521:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABCmJ ... @ global.firewall.ssh.local-key.g-Fortinet_SSH_ED25519:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABC3P ... @ global.firewall.ssh.local-ca.g-Fortinet_SSH_CA:failed command (error -26)
>>> "set" "private-key" "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAACmFlczI1Ni1jdHIAAAAGYmNyeXB0AAAAGAAAABDd2 ... @ global.firewall.ssh.local-ca.g-Fortinet_SSH_CA_Untrusted:failed command (error -26)

Workaround:

1. Temporarily disable 'web-svc-auto-restart' before upgrading:
   
   

  config system global
      set web-svc-auto-restart disable
  end

2. After the upgrade is complete, re-enable it again.
   
   

  config system global
      set web-svc-auto-restart enable
  end

3. Run the command below to further verify configuration integrity, post upgrade:
   
   

show system global
diagnose debug config-error-log read

This is documented under Release Notes - Known Issues with ID 1277454:
Release Notes - Known Issues 7.6.7

Resolution:

The fix for this is included in FortiOS v7.4.13, v7.6.8, and v8.0.1.