Description
This article explains how to troubleshoot an update failure on a FortiGate that occurs with a 'Server certificate failed verification' warning to check if a failed certificate is responsible.
Scope
FortiGate.
Solution
FortiGate may fail to fetch an update from FortiGuard for multiple reasons. This article will focus on certificate issues.
Follow this step-by-step process to isolate the problem.
- Check internet connection and DNS resolution:
execute ping guard.fortinet.net
execute ping service.fortiguard.net
execute ping update.fortiguard.net
If they are working as expected, a response will be received. If so, move on to step 2.
- Perform a debug-enabled update to check which process fails during the update.
diag debug reset
diag debug application update 1
diag debug enable
exec update-now<-----Triggers the update process, obtaining the output of the debug action. Let it run for around 1 minute.)
To disable and deactivate the debug process:
diag debug disable
diag debug application update 0
diag debug reset
If FortiGate is in multi-VDOM mode, the update must be run in global VDOM.
Example output:
__upd_peer_vfy[331]-Server certificate failed verification. Error: 19 (self signed certificate in certificate chain), depth: 1, subject: /C=US/ST=California/L=Sunnyvale/O=Fortinet/OU=Certificate Authority/CN=support/emailAddress=support@fortinet.com.
ssl_connect_fds[392]-Failed SSL connecting (5,0,Success)
upd_comm_connect_fds[476]-Failed SSL connect
upd_act_HA_contract_info[779]-Error updating FSCI -1
If all the certificate pass, move to the next step
diag hardware certificate
Checking Fortinet_CA.cer integrality ........Passed
Checking Fortinet_Factory.cer integrality ........Passed
Checking Fortinet_Factory.cer key-pair integrality ........Passed
Checking Fortinet_Factory.cer Serial-No. ........Passed
Checking Fortinet_Factory.cer timeliness ........Passed
Checking Fortinet_Factory.key integrality ........Passed
Ensure that all the certificate are up-to-date.
In this case, the Fortinet SSL certificates are expired. VPN certificates can be renewed with this command:
execute vpn certificate local generate default-ssl-key-certs
For additional information on how to renew expired certificates, refer to the following article:
Renew Certificate Expired on FortiGate - Fortinet Community
Re-enable the debug and trigger the udpate
diag debug reset
diag debug application update 1
diag debug enable
exec update-now
If the certificate error is still present, enable FortiGuard Anycast mode
It is also possible that FortiGate is using 'Fortiguard-anycast disabled' and this is the reason for the error:
[212] ssl_add_ftgd_hostname_check: Add hostname checking 'usupdate.fortiguard.net'...
[922] ssl_set_hostname: Set hostname 'fortinet-ca2.fortinet.com'
[720] __ssl_info_callback: before SSL initialization
[720] __ssl_info_callback: SSLv3/TLS write client hello
[720] __ssl_info_callback: SSLv3/TLS write client hello
[720] __ssl_info_callback: SSLv3/TLS read server hello
[720] __ssl_info_callback: TLSv1.3 read encrypted extensions
[720] __ssl_info_callback: SSLv3/TLS read server certificate request
[362] __ssl_crl_verify_cb: Cert error 19, self-signed certificate in certificate chain. Depth 2
__upd_peer_vfy[329]-Server certificate failed verification. Error: 19 (self-signed certificate in certificate chain), depth: 2, subject: /C=US/ST=California/L=Sunnyval
e/O=Fortinet/OU=Certificate Authority/CN=fortinet-ca2/emailAddress=support@fortinet.com.
[1063] ssl_connect: SSL_connect failes: error:0A000086:SSL routines::certificate verify failed
ssl_connect_fds[393]-Failed SSL connecting (5,0,Success)
[207] __ssl_data_ctx_free: Done
[1108] ssl_free: Done
[199] __ssl_cert_ctx_free: Done
[1118] ssl_ctx_free: Done
upd_comm_connect_fds[478]-Failed SSL connect
do_update[690]-UPDATE failed
Verify whether the FortiGate location is US. Try the change for 'Fortiguard-anycast enable', whereupon FortiGate will start to use the default protocol 'https' with port '443'.
config system fortiguard
set fortiguard-anycast enable
unset port
unset sdns-server-ip
unset protocol
end
Attempt to run the update:
diagnose debug application update -1
diagnose debug enable
execute update-now
If the command 'diag hardware certificate' displays the error 'Not Matched', a failed certificate verification is the cause of the problem. Run a diagnosis on hardware certificates:
diag hardware certificate
Checking Fortinet_CA.cer integrality ........Passed
Checking Fortinet_Factory.cer integrality ........Passed
Checking Fortinet_Factory.cer key-pair integrality ........Passed
Checking Fortinet_Factory.cer Serial-No. ........[Not Matched]
Checking Fortinet_Factory.cer timeliness ........Passed
Checking Fortinet_Factory.key integrality ........Passed
The [Not Matched] certificate is the root cause.
To fix this, download the highlighted certificate and open it:
The certificate 'issued to' information will most likely be missing. This information contains the FortiGate serial number from the device.
To solve this issue, perform a fresh install of the OS:
Technical Tip: Installing firmware from system reboot
Always verify the downloaded OS with checksum validation to confirm the security and integrity of the installer:
Technical Tip: How to verify downloaded firmware checksum.
For a High Availability firewall, the step-by-step process to solve the issue will be similar:
- Perform 'diag hardware certificate' on both firewalls.
- It may be necessary to failover to another device due to the fact that only the master unit will perform the update. In this case, isolate which device has the certificate issue. Refer to the above process.
- As in the solution above, perform a fresh install of the OS of the affected device. See the attached .pdf file for step-by-step breaking and rebuilding instructions for an HA cluster.
- Verify the certificate bundle version is up-to-date on both units using the below command:
diagnose autoupdate versions | grep -A7 'Certificate Bundle'
There could be several reasons, as shown in the following command output:
Certificate Bundle
---------
Version: 1.00052
Contract Expiry Date: n/a
Last Updated using manual update on Thu Oct 10 17:45:00 2024
Last Update Attempt: Tue Oct 22 06:48:32 2024
Result: Connectivity failure <- Make sure this is updated.
Or:
Certificate Bundle
---------
Version: 1.00052
Contract Expiry Date: n/a
Last Updated using manual update on Thu Oct 17 14:54:47 2024
Last Update Attempt: Tue Oct 22 10:17:34 2024
Result: No Updates <- Make sure this is updated.
If the certificate above mentioned is there in both the units in the cluster, and if the certificate status shows 'Pass' with the above commands, it is recommended to reboot the primary and secondary devices. Wait for a while and the issue will be fixed.
Related articles:
Technical Tip: Failed to contact FortiGuard servers due to unknown CA
Technical Tip: How to check most updated security version database on FortiGate
