No fortiguard, forticloud, forticare on branch FGT

Hi,

I have problem on the branch fortigate it route all traffic to hq fortigate, and I cannot run fortiguard (Unable to connect to FortiGuard servers.) Forticloud - failed to load data etc.

In the branch console I can ping all fortiguard servers, I can telnet to 514 ports like:

execute telnet 173.243.132.171 514
Trying 173.243.132.171...
Connected to 173.243.132.171.

on hq fortigate I see in logs that traffic from branch destined to fortiguard like:

13.248.131.62 173.243.132.27

are allowed and go out to the internet but these service are not working, in log I have something like:

[3038] fds_download_image_list: TRACE [41] fds_queue_task: req-1 is added to fds-update [579] fds_https_start_server: server: 13.248.131.62:443 [580] fds_https_start_server: source-ip: 0.0.0.0:0 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [82] https_create: proxy server 0.0.0.0 port:0 [185] forticldd_add_hostname_check: Add hostname checking 'globalupdate2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 [1894] SSL_dump_handshake_err: Certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalupdate2.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed [477] fds_https_connect: https_connect(13.248.131.62:443) failed: ssl_connect() failed: 0 (error:00000000:lib(0):func(0):reason(0)). [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done [1037] ssl_disconnect: Shutdown [238] fds_svr_default_on_error: fds-update: ip=13.248.131.62:443, reason=4 [255] fds_svr_default_on_error: fds-update: Conn failes 1/1 [275] fds_svr_default_on_error: fds-update: req-id=1, num_try=1, read=0, reason=4 [2993] tsk_send_image_list: num=-1 [465] fds_send_reply: Sending 0 bytes data. [421] fds_free_tsk: cmd=1; req.noreply=1 [421] fds_free_tsk: cmd=1; req.noreply=0 [186] fds_svr_default_task_xmit: try to get IPs for fds-update [254] fds_resolv_addr: resolve 'globalupdate2.fortinet.net' [186] fds_get_addr: name=globalupdate2.fortinet.net, id=14041, cb=0x9588e8 [102] dns_parse_resp: DNS globalupdate2.fortinet.net -> 13.248.131.62 [102] dns_parse_resp: DNS globalupdate2.fortinet.net -> 76.223.2.16 [137] fds_svr_default_pickup_server: fds-update: 13.248.131.62:443 [3274] fds_handle_request: Received cmd 116 from pid-1298, len 0 [465] fds_send_reply: Sending 8 bytes data. [3274] fds_handle_request: Received cmd 116 from pid-1298, len 0 [465] fds_send_reply: Sending 8 bytes data. [3274] fds_handle_request: Received cmd 101 from pid-1298, len 0 [41] fds_queue_task: req-101 is added to message-controller [579] fds_https_start_server: server: 173.243.132.27:443 [580] fds_https_start_server: source-ip: 0.0.0.0:0 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [82] https_create: proxy server 0.0.0.0 port:0 [185] forticldd_add_hostname_check: Add hostname checking 'globalmsgctrl2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 [1894] SSL_dump_handshake_err: Certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalctrl.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed [477] fds_https_connect: https_connect(173.243.132.27:443) failed: ssl_connect() failed: 0 (error:00000000:lib(0):func(0):reason(0)). [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done [1037] ssl_disconnect: Shutdown

upd_daemon[1782]-Received update now request upd_daemon[1508]-Found cached action=00000002 do_update[492]-Starting now UPDATE (final try) upd_fds_load_default_server6[1105]-Resolve fds ipv6 address failed. upd_comm_connect_fds[458]-Trying FDS 76.223.2.16:443 tcp_connect_fds[234]-Binding to interface 122 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [166] ssl_add_ftgd_hostname_check: Add hostname checking 'globalupdate2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 __upd_peer_vfy[331]-Server certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalupdate2.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed ssl_connect_fds[392]-Failed SSL connecting (5,0,Success) [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done upd_comm_connect_fds[476]-Failed SSL connect upd_comm_connect_fds[458]-Trying FDS 13.248.131.62:443 tcp_connect_fds[234]-Binding to interface 122 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [166] ssl_add_ftgd_hostname_check: Add hostname checking 'globalupdate2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 __upd_peer_vfy[331]-Server certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalupdate2.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed ssl_connect_fds[392]-Failed SSL connecting (5,0,Success) [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done upd_comm_connect_fds[476]-Failed SSL connect do_update[504]-UPDATE failed do_check_wanip[655]-Starting getting wan ip upd_fds_load_default_server6[1105]-Resolve fds ipv6 address failed. upd_comm_connect_fds[458]-Trying FDS 76.223.2.16:443 tcp_connect_fds[234]-Binding to interface 122 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [166] ssl_add_ftgd_hostname_check: Add hostname checking 'globalupdate2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 __upd_peer_vfy[331]-Server certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalupdate2.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed ssl_connect_fds[392]-Failed SSL connecting (5,0,Success) [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done upd_comm_connect_fds[476]-Failed SSL connect upd_comm_connect_fds[458]-Trying FDS 13.248.131.62:443 tcp_connect_fds[234]-Binding to interface 122 [113] __ssl_cert_ctx_load: Added cert /etc/cert/factory/root_Fortinet_Factory.cer, root ca Fortinet_CA, idx 0 (default) [480] ssl_ctx_use_builtin_store: Loaded Fortinet Trusted Certs [486] ssl_ctx_use_builtin_store: Enable CRL checking. [493] ssl_ctx_use_builtin_store: Enable OCSP Stapling. [755] ssl_ctx_create_new_ex: SSL CTX is created [782] ssl_new: SSL object is created [166] ssl_add_ftgd_hostname_check: Add hostname checking 'globalupdate2.fortinet.net' [359] __ssl_crl_verify_cb: Cert error 20, unable to get local issuer certificate. Depth 0 __upd_peer_vfy[331]-Server certificate failed verification. Error: 20 (unable to get local issuer certificate), depth: 0, subject: /jurisdictionC=US/jurisdictionST=Delaware/businessCategory=Private Organization/serialNumber=3321792/C=US/ST=California/L=Sunnyvale/O=Fortinet, Inc./CN=globalupdate2.fortinet.net. [1001] ssl_connect: SSL_connect failes: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed ssl_connect_fds[392]-Failed SSL connecting (5,0,Success) [203] __ssl_data_ctx_free: Done [1046] ssl_free: Done [195] __ssl_cert_ctx_free: Done [1056] ssl_ctx_free: Done upd_comm_connect_fds[476]-Failed SSL connect do_check_wanip[659]-Failed getting wan ip

something with certificates I see but how to fix this? On branch I don't use deep ssl scan, only at hq when going to internet.