FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Sachin_Alex_Cherian_
Article Id 301596
Description This article describes every aspect of FortiGuard-related communications initiated from a device such as FortiGate.
Scope

FortiGuard - Introduction:

  • FortiGuard Subscriptions/Services.

FortiGuard Methods - Live Querying vs Databases:

  • Live Querying.
  • Live Querying - Rating Errors.
  • Live Querying - Rating Server availability.
  • Database/Package updates.
  • Database/Package updates - Package verification.
  • Database/Package updates - Manual update/downgrade of a package.
  • Scheduling Package Updates.
  • Live Querying and Database/Package Updates Comparison.

FortiGuard Connectivity:

  • FortiGuard Connectivity - Anycast.
  • FortiGuard Connectivity - Multi-VDOM environment.
  • FortiGuard Connectivity - SD-WAN environment.
  • FortiGuard Connectivity - Specifying Source IP.

FortiGuard Licensing in High Availability (HA).

FortiGuard Licensing through FortiManager.

FortiGuard Licensing through Proxy Server.

FortiGuard Troubleshooting.

Solution

FortiGuard - Introduction.

 

FortiGate receives the most recent threat intelligence from FortiGuard. FortiGuard Distribution Network (FDN) provides it through data centers located in North America, Europe, and Asia. FortiGate can connect based on server load or choose to connect to the closest location. Through the FDN, FortiGuard offers many services, all of which need a license.

 

FortiGuard Subscriptions/Services:

 

The details of FortiGuard services and subscriptions are available at FortiGuard Service Offerings.

 

Services.JPG

 

FortiGuard Methods - Live Querying vs Databases.

FortiGate uses two fundamental techniques to stay up-to-date with the FDN.

  1. Live Querying.
  2. Database/Package updates.

Live Querying:

Live querying is carried out by features that use category-based filtering, such as the Web Filter and DNS Filter. Each request must be checked against FDN to determine the appropriate category of the domain being requested. To boost efficiency, the URL/domain classification is cached.

 

The category dictates whether the connection is refused or authorized, depending on the action defined for the category in the relevant Security Profile. If anycast is enabled, it connects to globalguardservice.fortinet.net, otherwise to service.fortiguard.net. It communicates via TCP port 443 with anycast enabled.

 

However, if anycast is disabled, this can be changed to utilize the UDP port 53/8888. If the location is specifically configured to connect to the United States, for example, the connection is forwarded to usguardservice.fortinet.net. Live querying necessitates continuous connectivity to FDN and a valid service license.

If the license expires, there is a two-day grace period. After that, initiated requests will be blocked unless the functionality is turned off.

 

LiveQuerying-web.JPG

 

LiveQuerying-DNS.JPG

 

Below is a sample debug of the flow-based web filter categorization request and the FortiGuard response. The debug commands used are below:

 

diagnose debug application urlfilter -1

diagnose debug enable

 

urlfilter.JPG

 

If the traffic volume is too much, the debug output can be restricted by filtering based on the source IP of the client machine.

 

diagnose debug urlfilter src-addr <IP address>

 

Live Querying - Rating Errors:

If live queries to categorize a URL fail, site access is blocked by default. This could be due to connectivity issues with FDN/FortiGuard or an expired web filter service license. End users may encounter an error in the browser as shown below.

 

webPageBlocked.JPG

 

In these situations, enable the option ‘Allow websites when a rating error occurs’ as shown below under the security profile so that the websites are not blocked. The same option is available for DNS filtering security profiles as well.

 

allow rating error.JPG

 

If the rating error occurs due to connectivity issues with FortiGuard, checking the following points would be helpful:

  • With Anycast enabled, FortiGate uses a TCP (SSL port 443) connection. Any issues with connection such as an upstream firewall intercepting SSL could cause issues with connectivity. In such cases, anycast can be disabled and the user can switch to using UDP.

config system fortiguard

    set fortiguard-anycast disable

    set protocol udp

    set port 53

end

  • In some networks where DNS compliance checks are enforced, traffic on UDP port 53 that is not related to DNS is dropped. In such scenarios, it is possible to switch to using port 8888.
  • Some scenarios result in connections being rejected because the source port used falls close to the range of well-known ports, such as 1024. In such cases, the source port can be manually changed to use a higher port range.

config sys global

    set ip-src-port-range <>

end

 

Live Querying - Rating Server availability:

The FortiGuard server which the FortiGate is connecting to can be checked by issuing the command:

 

diagnose debug rating

 

debug-rating.JPG

 

The above implies that the web filter license is under contract and enabled. With anycast enabled, communication goes through port 443. If there is a connectivity issue, the 'Curr Lost' figure will continue to increase. The 'RTT' indicates the round trip time.


Accordingly, a weight is allocated. A server is picked based on its weight and RTT values.
For a thorough understanding of how a certain server is selected from the list, see Troubleshooting Tip: Resolving FDS Communication Issues (FortiGuard Distribution Servers).

 

Database/Package updates:

Packages are kept locally on FortiGate. The changes in data are not particularly frequent. As such, the frequency of updates happening is low. Package/Database updates can be scheduled.

When anycast is enabled, the connection to globalupdate.fortinet.net is made via TCP port 443. If anycast is disabled, it connects to update.fortiguard.net also via TCP port 443. There are several package objects available such as IPS, AV, Internet Service, Device and OS Identification, and IP Geography databases.

The following commands can be used to view various databases/packages available:

 

diag autoupdate versions

get sys fortiguard-service status

 

The various packages and services which gets verified when running execute update-now can be reviewed in the article Technical Tip: Deciphering FortiGuard database abbreviations and subscriptions/services.

 

Database/Package updates - Package verification:

AV and IPS packages are signed by the Fortinet CA to ensure the authenticity of the packages before using the packages. During automatic updates, only signed and validated packages are accepted. During manual package updates, the following applies:

  1. Level-0: accept the new package even if it is unsigned.
  2. Level-1: display a warning and request a user confirmation to accept.
  3. Level-2: display an error and reject the image.

Security levels 0-1 are pre-configured on the BIOS.

Verify whether the packages being used are signed by Fortinet by using the following commands:

 

diagnose autoupdate signature check-all

diagnose autoupdate versions

 

Database/Package updates - Manual update/downgrade of a package:

Packages can be upgraded directly from the GUI. 

 

updates-GUI.JPG

 

The example shown above demonstrates various options to update packages. The IPS engine and IPS Database are updated using the same 'Action' tab against IPS. This is the same case for AV engines and AV databases as well. 

The same can be achieved through the CLI using the following command:

 

exec restore ips/av/other-objects tftp <filename> <serverIP>

 

For example, to update the Industrial OT database, use the command:

 

exec restore other-objects tftp <filename> <serverIP>

 

Note:

Not all package updates can be done from GUI the Industrial DB is such an example. Older firmware might show the feasibility of IPS and AV alone to be updated from GUI. Other package updates need to be done from the CLI.

 

A package update does not cause the whole device to reboot. However, the corresponding process restarts. For example, when updating the IPS engine or database all ipsengine processes restart.

 

In some scenarios, the package needs to be downgraded. Downgrading is possible only after enabling the below command over CLI:

 

diag autoupdate downgrade enable

 

Scheduling Package Updates:

Updating definitions can cause a momentary increase in CPU. Updates can be scheduled during off-peak hours when network usage is at a minimum to ensure that network activity will not be affected.

 

schedule-GUI.JPG

 

In the CLI, this can be configured under:

 

config system autoupdate schedule

 

The default setting is automatic and keeps checking every few hours.

A schedule of once a week means any urgent updates will not be pushed until the scheduled time. If an urgent update is required, select the Update Licenses & Definitions Now button to manually update the definitions.

 

Live Querying and Database/Package Updates Comparison.

 

Live Queries:

  1. service.fortiguard.net (without anycast) or globalguardservice.fortinet.net (with anycast)
  2. UDP (53 or 8888)/TCP (443).
  3. Requires constant connectivity to FDN.
  4. Web Filter, DNS Filter, Anti Spam etc.
  5. If connectivity lost with FDN, traffic is blocked.
  6. The frequency of change is high.

Database/Package Updates:

  1. update.fortiguard.net (without anycast) or globalupdate.fortinet.net (anycast)
  2. TCP port 443 (SSL)
  3. Maintains local database on Fortigate, needs connectivity to FDN only to update the database.
  4. AV Database, IPS Database, Application Control Database etc.
  5. If connectivity is lost with FDN - Traffic still works with the existing database but new signature updates are missed.
  6. The frequency of changes is low.

 

FortiGuard Connectivity:

This section describes deployment strategies that can be used in a variety of scenarios along with the Anycast-based FortiGuard connectivity technology.

 

FortiGuard Connectivity - Anycast (see Anycast settings):

The default FortiGuard access mode is Anycast. It enforces SSL connections on port 443 and validates them using the OCSP (Online Certificate Status Protocol) stapling check. FortiGate gets a single IP for the domain name of each FortiGuard service. It improves routing efficiency by connecting to the nearest server. With Anycast enabled, FortiGate terminates a connection with FortiGuard if any of the following conditions apply:

  1. The CN in the server certificate does not match the domain.
  2. The OCSP's status is not good.
  3. The issuer's CA is revoked.

Below is an example Wireshark capture that shows the OCSP check:

 

Anycast-wireshark.JPG

 

Below are a few connection URLs that FortiGate uses when Anycast is enabled:

 

Anycast-URL.JPG

 

FortiGuard Connectivity - Multi-VDOM environment:

In multi-VDOM mode, users can choose from which VDOM FortiGuard services and updates are initiated, instead of being locked to the management VDOM. However, this is possible only from firmware version 7.2 or above.

 

multi-vdom.JPG

 

config global

    config system fortiguard

        set vdom "root"

    end

end

 

The VDOM specified should be able to reach the internet and should be able to resolve DNS queries.

To set up FortiGuard services on a non-management VDOM:

  1. Specify the VDOM to be used under 'config system fortiguard'.
  2. Ensure that the specific VDOM has connectivity to the internet.
  3. FortiGate should be able to resolve the DNS from within the vdom, so that the FortiGuard services may resolve the server name using the specific VDOM.

FortiGuard Connectivity - SD-WAN environment:

The interface selection can be chosen based on individual requirements.

 

The interface-select-method command makes it possible to choose from the following options:

  1. Auto: This is the default setting; uses the interface based on reachability as seen in the routing table lookup.
  2. SD-WAN: Uses route lookup for selecting the interface which is part of SD-WAN. SD-WAN rule can be further used to select a specific interface from SD-WAN.
  3. Specify: Manually specifies a specific interface.

The SD-WAN rule will only take effect once the interface-select-method is set to SD-WAN. The same can be done from Local-Out-Routing: go to Network -> Local Out Routing to configure the available types of local out traffic.

 

By default, Local Out Routing is not visible in the GUI. Go to System -> Feature Visibility to enable it. From the CLI, the interface-select-method can be chosen as SD-WAN.

 

sdwan-rule.JPG

 

FortiGuard Connectivity - Specifying Source IP:

In specific scenarios, it is required to initiate the connections from a specific IP as this IP could be whitelisted in the network. This can be achieved by configuring the following:

 

config system fortiguard

set source-ip x.x.x.x

end

 

Make sure that the IP being used is routable/reachable within the network. A random IP cannot be configured or used as a source IP for FortiGuard connectivity, and the IP being used as the source should belong to an interface on FortiGate.

Note: Use the command ‘get system source-ip status’ to retrieve which config parts are using specific source-ip.

 

FortiGuard Licensing in High Availability (HA):

  • In HA, the master device is responsible for updating and maintaining the packages and initiating live querying. 
  • All devices that are part of HA should have a valid contract. If one device does not have a valid license, then HA will show that the cluster does not have a valid license. If the license is different, the cluster reflects the lowest-level license.

  • In HA, if a dedicated management interface is enabled under ‘config system ha’ for a specific interface, that interface cannot be used for FortiGuard communication. This is because the interface once reserved for HA management becomes part of vsys_hamgmt which is a hidden VDOM.

  • The dedicated mgmt interface’s IP cannot be used as source IP as well.

  • Regardless of the mode ie Active-Passive or Active-Active the updates are done by the primary device.

  • Manual upgradation of the package on the primary also pushes the same to the other devices in the cluster.

FortiGuard Licensing through FortiManager:

FortiManager can act as a local FortiGuard server. The FortiGate connects to FortiManager instead of directly connecting to FDN over the Internet.

 

fmg-GUI.JPG

 

config system central-management

set type fortimanager

set fmg "192.168.62.100"

config server-list

edit 1

set server-type update rating

set server-address 192.168.62.100

next

end

end

 

The server-type helps decide whether FortiManager handles live querying, package updates or both. 

When fmg-update-port is set to 443, the update process will use port 443 to connect to the override update server, which is the local FortiGuard server in the FortiManager. If this is not set, the update process will use port 8890.

 

FortiGuard Licensing through Proxy Server:

A FortiGate unit can use a proxy server to connect to the FortiGuard Distribution Network (FDN). The configuration for this can only be set up through the CLI:

 

config system autoupdate tunneling

set status enable

set address "192.168.62.254"

set port 8080

end

 

FortiGate connects to the proxy server using the HTTP CONNECT method on the configured port number.

The proxy server must not inspect the HTTPS traffic used for FortiGate communication.

FortiGate sends to the proxy server an HTTP CONNECT request that specifies the IP address and port required for the FDN connection. The proxy server establishes the connection to FDN and passes information between FortiGate and FDN.

Debugging the update daemon would show the below logs for connectivity with the proxy server:

 

proxy-deb.JPG

 

The connect request here is for an IP and not a URL because the URL is resolved by the FortiGate system DNS.

If the system DNS fails to resolve the URL, the connect request would be for the URL directly.

 

FortiGuard Troubleshooting:

Below is a flowchart intended to demonstrate how to troubleshoot FortiGuard issues. Click on the image to enlarge it.

 

Flow-Chart.jpg

 

On FortiGate, the 'update' daemon is responsible for validating the license and keeping the packages up-to-date.

When issues occur, check the daemon logs to see what exactly is going wrong:

 

diagnose debug reset

diagnose debug application update -1

diagnose debug console timestamp enable

diagnose debug en

 

Once the above debug commands have been enabled, initiate 'execute update-now' from the CLI or select 'Update Licenses & Definitions Now' in the GUI under System -> FortiGuard.

The update process will now be shown in the debugs.

Packet capture can also be initiated to validate the connection if a TCP or SSL connection error is seen in the debug trace. The capture should be filtered using the IP seen in the update daemon debug trace.

A few common errors observed are explained below:

 

upd_fds_load_default_server[939]-Resolve and add fds globalupdate.fortinet.net ip address failed.

 

This highlights a DNS issue. FortiGate is not able to resolve globalupdate.fortinet.net.

 

tcp_connect_fds[242]-Failed to connect (Network is unreachable).

 

This highlights a routing issue. Check the routing on FortiGate and verify the interface selection methods.

 

upd_comm_connect_fds[474]-Failed TCP connect.

 

This highlights a TCP connection issue. Perform a FortiGate telnet to the FDS IP on port 443 to re-validate. Alternatively, switch to using UDP by disabling anycast.

 

upd_comm_connect_fds[447]-Failed SSL connect.

 

This highlights an issue related to SSL connection. Check if any SSL inspection is done upstream. Packet capture also helps to verify if the SSL handshake is being completed successfully. Alternatively, disable anycast and use UDP port 53/8888. 

 

The following are the key points to consider while troubleshooting FortiGuard-related issues:

  1. Validate the device license using the 'diagnose auto update versions' command.
  2. Validate that every device in the cluster has a valid contract.
  3. Verify services are enabled. For example, AV and IPS updates are triggered when they are applied at policy level. Make sure force-off settings are not enabled under 'config system fortiguard'.
  4. Validate that the FortiGate can resolve URLs using its system DNS.
  5. Check reachability to the FDN by performing a simple ping.
  6. If a TCP or SSL connection is not getting established, validate upstream whether any devices are doing inspection or filtering connections.
  7. Troubleshoot using the update daemon debug logs and by performing a packet capture if required.