Troubleshooting Tip: Unable to connect to FortiGuard servers

[patelr]

Description	This article explains the possible reasons why FortiGate is unable to connect FortiGuard servers and offers steps to troubleshoot the problem. The 'Unable to connect to FortiGuard servers' error message can be seen in two places: Dashboard -> Status -> Licenses. System -> FortiGuard -> FortiGuard Updates. FortiOS v7.6.x: Dashboard -> Status -> Licenses. System -> FortiGuard -> Subscriptions -> Licensed.
Scope	FortiGate, FortiWeb.
Solution	The communication between FortiGate and FortiGuard for web filtering and antispam is different from the communication for antivirus and IPS. Check connectivity to FortiGuard servers by checking to ensure FortiGate correctly resolves DNS with the following hostnames: execute ping service.fortiguard.net execute ping update.fortiguard.net execute ping guard.fortinet.net If the DNS resolves, move to Step 2: If it is not resolving, check the DNS configuration. One of the most possible causes is when the 'Use FortiGuard Servers' option is changed to 'Specify' for use with an internal DNS server, without switching the DNS protocols or validating if the new DNS Server supports DoT (default setting of FortiGuard servers) which uses TCP 853 or DoH that uses TCP 443. If that happens the DNS server will be unreachable and DNS resolution will not work as seen in the next picture.  After changing to UDP 53 the DNS will be reachable, and resolution should start working. Following are the commands to change the protocol via CLI: config system dns     set protocol cleartext <- DoT is for using TCP/853, and cleartext is used for UDP/53. end Run 'diagnose debug rating' in the CLI: diagnose debug rating The output of the command 'diagnose debug rating' displays flags next to servers: I: The server initially connected to validate the license and fetch the server list. Typically, only one server will have this flag. The IP address FortiGate received when resolving the name service.fortiguard.net. If the administrator has not overwritten the FortiGuard FQDN or IP address in the FortiGuard configuration, there are usually two or three servers with this flag. S: The IP address FortiGate received from FortiManager. T: The server is not replying to FortiGate queries. F: The server is down. If all servers in the list have F(ailed), this may mean either all FortiGuard servers on the Fortinet side are down (unlikely), or that this FortiGate has a problem reaching them at the network level. If the PPPoE interface is being used as a WAN interface, review the MTU value and configure TCP-MSS on the PPPoE interface. fnsysctl ifconfig -a The PPPoE interface name will be displayed as 'ppp(x)'. ppp1 Link encap:Point-Point Protocol inet addr:x.x.x.x Mask:255.255.255.255 UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1 The MSS value should be = MTU - 40. Cannot view some web sites when using PPPoE. For example, if the MTU of the PPPoE interface is 1492: config system interface     edit <interface name>          set tcp-mss 1452  end Under global settings make sure that set cloud-communication is set to enable. FGT # config system global FGT (global) # set cloud-communication ? enable     Allow cloud communication. disable    Disable all cloud-related settings. FGT (global) #set cloud-communication enable FGT (global) #end If there are VDOMs enabled, all communication to the FortiGuard network is initiated from the management/root VDOM only. config sys global     set management-vdom "root" end However, from firmware v7.2 onwards in multi-VDOM mode, users can choose from which VDOM FortiGuard services and updates are initiated, instead of being locked to the management VDOM. config global     config system fortiguard         set vdom "root"     end end The VDOM specified should be able to reach the internet and should be able to resolve DNS queries. To set up FortiGuard services on a non-management VDOM: Specify the VDOM to be used under 'config system fortiguard'. Ensure that the specific VDOM has connectivity to the internet. FortiGate should be able to resolve the DNS from within the VDOM, so that the FortiGuard services may resolve the server name using the specific VDOM. Verify Antivirus & IPS Definition and Filtering Services: Check Antivirus & IPS Definition under System -> FortiGuard -> AntiVirus & IPS Updates -> Update AV and IPS Definitions. FortiOS v7.2.x: The active subscriptions under System -> FortiGuard are being updated with the 'FortiGuard Updates', the options for 'AntiVirus & IPS Updates' have been removed. FortiOS v7.6.x: The active subscriptions under System -> FortiGuard -> Subscriptions -> Licensed are being updated with the 'FortiGuard Updates' System -> FortiGuard -> FortiGuard settings. Check Filtering Services under System -> FortiGuard -> Filtering. FortiOS v7.6.x: Check Filtering Services under System -> FortiGuard -> FortiGuard settings and then expand the Filtering option. Anycast servers: It is recommended to disable anycast and switch back to unicast servers (Anycast communication) config system fortiguard     set fortiguard-anycast disable     set protocol udp     set port 8888     set sdns-server-ip 208.91.112.220 173.243.140.53 210.7.96.53    end For FortiGuard's SDNS rating service, there are two modes in FortiOS: Mode 1: Use of unicast network, which uses DNS over UDP protocol for FortiGate and FortiGuard transactions. Mode 2: Use of anycast network (default), which uses DNS over TLS protocol for FortiGate and FortiGuard transactions. To use UDP/53:  config system fortiguard     set fortiguard-anycast disable     set sdns-server-ip 208.91.112.220 173.243.140.53 210.7.96.53 end To use DoT (TCP/853). config system fortiguard     set fortiguard-anycast enable end With Anycast, FortiGate is only aware of one single server IP. This is a floating IP address that will connect to the closest server geographically, and if this server is down, it will point to another server instead. With Unicast, the FortiGate must maintain a list of servers that it tries and if one stops working it then switches over to another. In many cases, problems related to FortiGuard are caused by ISPs. Some ISPs block traffic on port 53 that is not DNS or that contains large packets. In those cases, the solution is to use port 8888. Other ISPs block traffic to HTTPS port 8888. In those cases, the solution is to use UDP port 53.  Related port information: Encrypted Virus Samples auto-submitted to FortiGuard – 25. DNS lookups – 53 UDP. FortiGuard Server List requests to FortiGuard – 53 UDP. AntiSpam or Web Filtering rating lookup queries to FortiGuard – 53 UDP or 8888 UDP. URL/AS rating lookup queries to FortiGuard – 53 UDP. Real-time Black List (RBL) lookup requests to RBL services – 53 UDP. Fortinet Device Registration to FortiGuard – 80 HTTP. Firmware and Signature Downloads from FortiGuard – 443 HTTPS. FortiGuard Server List requests to FortiGuard – 1027 UDP / 1031 UDP. AntiSpam and Web Filtering rating lookup requests – 1027 UDP / 1031 UDP. AV/IPS Push / FortiGuard to FortiGate – 9443 UDP. To configure FortiGate to use worldwide servers or only servers located in the USA, run the following command in the CLI: config sys fortiguard     set update-server-location [usa/any] end This can also be done under System -> FortiGuard -> FortiGuard Update in the GUI. FortiOS v7.6.x: This can also be done under System -> FortiGuard -> FortiGuard settings. Some debug commands for FortiGuard: diagnose debug reset diagnose debug application update -1 diagnose debug enable Force FortiGuard update after running debug application update -1: execute update-now The following command can also fix various issues with FortiGuard servers. In this example, 212.48.23.12 is used as the IP for the interface to the ISP router. config system fortiguard     set fortiguard-anycast disable     set protocol udp     set port 8888     set sdns-server-ip 208.91.112.220 173.243.140.53 210.7.96.53     set ddns-server-ip 173.243.138.225     set source-ip 212.48.23.12 end config system dns     set primary 8.8.8.8     set source-ip 212.48.23.12 end Source IP should match the IP address on the Wan port of FortiGate (ISP port), if Source IP is changed, DNS and FortiGuard settings should be changed as well. Note: Despite changing the FortiGuard settings to anycast disable and configuring UDP protocol to reach the FortiGuard servers, the connection still fails because the DNS servers will still use the 'DNS over TLS' settings which is a default setting, in this case, public IPs will be reachable, but DNS resolution fails. Changing the DNS settings to use 'DNS over UDP' will help in resolving the domain names properly. Note: Suppose connectivity to FortiGuard is required through one of the outgoing interfaces. In that case, it is possible to use 'interface-select-method' with the 'specify' action and then select the outgoing interface under the FortiGuard configuration. See Technical Tip: Functionality of 'set interface-select-method' for local-traffic with SD-WAN for more information. config system fortiguard     set interface-select-method {auto|sdwan|specify}     set interface WAN Interface Try to take the following debugs : diagnose debug reset diagnose debug application update -1 diagnose debug enable execute update-now In this case, the output of the debugs looks like: upd_act_setup_with_action[190]-Trying FDS 173.243.138.68:443 tcp_connect_fds[231]-select() timed out upd_comm_connect_fds[423]-Failed TCP connect Try to use 'interface-select-method' with the 'specify' action and then select the outgoing interface under the FortiGuard configuration as mentioned above. Starting from v7.2, initiating FortiGuard services and updates is not locked to the management VDOM. See Allow FortiGuard services and updates to initiate from a traffic VDOM for more information. If the issue persists, contact Fortinet support. License errors may be found in two places as shown below: Dashboard -> Status -> Licenses. System -> Config -> FortiGuard. If DNS issues occur, the following output will be seen in the CLI: Verify DNS settings in FortiWeb under Network -> DNS. Verify the reachability of the DNS IP. If the DNS is private, change to a public DNS and verify the connectivity. If the DNS resolves, check reachability. If there is any firewall, allow the ICMP for FortiWeb. If there is any firewall in place, allow traffic to all destinations for testing purposes. If this fixes the license issue, it indicates the issue is on the firewall policy level. If all checks have been performed successfully, collect the following logs and share them in a ticket created with support: diagnose debug application fds 7 diagnose debug application update 7 diagnose debug enable execute update-now To disable the debug: diagnose debug disable diagnose debug reset FortiGuard licenses and the communication to FortiGuard are viewable via GUI through System -> FortiGuard as well as through the CLI using the following commands: diagnose autoupdate versions The output is quite long, but it is possible to pipe the output to grep: diagnose autoupdate versions | grep -A6 "IPS Attack Engine" If the communication to FortiGuard is OK, but still the Web Filter is shown as 'disabled' in the 'diag debug rating' output: diagnose debug rating Locale : english Service : Web-filter Status : Disable  <-- Service : Antispam Status : Disable Service : Virus Outbreak Prevention Status : Disable The Web Filter should be licensed: Check the firewall policy. If there is no Web Filter Profile configured in the firewall policy, the web filter will be shown as disabled in the 'diag debug rating' output.  Note: If the error persists after applying changes, try accessing the same page in another browser or a private window, as the browser might be displaying a cached version of the page. Check if there are any upstream devices towards the internet and make sure to allow UDP port 53 or port 8888 services in the upstream devices and not block these services for the FortiGuard Connectivity. Related articles:  Technical Tip: FortiGuard Flags and Meaning Technical Tip: FortiGuard is not reachable via Anycast default method Technical Tip: FortiGuard Overview and Troubleshooting