Troubleshooting Tip: FortiGate occasionally applies 'inaccurate' Web Filter category when comparing to Web Filter Lookup

[GusZ]

Description	This article describes a FortiGate issue involving inexact Web Filter categorization and how to prevent and/or solve it.
Scope	FortiGate units and Web Filter profile usage.
Solution	In a given period of time, when a host created multiple connections to a particular URL, an upstream FortiGate unit with enabled Web Filter profiles might, at infrequent or irregular intervals, apply a 'wrong' Web Filter category to the associate FQDN (despite a correct Web Filter Lookup). Due to that, the connection might be subjected to a different behavior (e.g. SSL inspection and final action), causing application outages. For example: consider the FQDN 'citrix.cloud.com', which is categorized as 'Information Technology' (in Web Filter Database version 234.09913). When a given host establishes a connection to https://citrix.cloud.com, it is expected that the session will be categorized as 'Information Technology'. However, the same host might make a second connection to https://citrix.cloud.com a few minutes later, and this new session might handled by the FortiGate as a different Web Filter category. In this case, depending on the FortiGate configuration, the second connection could be dropped (instead of accepted) or subjected to SSL inspection (instead of exempted). In the example above, the FortiGate unit is operating nominally. In short, the issue is due to the DNS records associated with the FQDN 'citrix.cloud.com'. Using the command-line tool dig to query 'citrix.cloud.com', the reply shown below includes multiple CNAME records and one A record: $ dig citrix.cloud.com ... ;; ANSWER SECTION: citrix.cloud.com.   291   IN   CNAME   ctxwsp-citrix-fdproxy-global.trafficmanager.net. ctxwsp-citrix-fdproxy-global.trafficmanager.net.  51  IN  CNAME  ctxwsp-cc-frontdoor.azurefd.net. ctxwsp-cc-frontdoor.azurefd.net.  2631  IN  CNAME  azurefd-t-prod.trafficmanager.net. azurefd-t-prod.trafficmanager.net.  51  IN  CNAME  shed.dual-low.s-part-0014.t-0009.t-msedge.net. shed.dual-low.s-part-0014.t-0009.t-msedge.net.  29  IN  CNAME  s-part-0014.t-0009.t-msedge.net. s-part-0014.t-0009.t-msedge.net.  29  IN  A  13.107.246.42 ... Now, when performing a Web Filter Lookup for the FQDNs pointed in each CNAME record, the following results are obtained (in Web Filter Database version 234.09913): citrix.cloud.com Category: Information Technology ctxwsp-citrix-fdproxy-global.trafficmanager.net Category: Information Technology ctxwsp-cc-frontdoor.azurefd.net Category: Information Technology azurefd-t-prod.trafficmanager.net Category: Information Technology shed.dual-low.s-part-0014.t-0009.t-msedge.net Category: Business s-part-0014.t-0009.t-msedge.net Category: Business Thus, depending on the DNS lookup result and how the host presents the connection request, different sessions might be treated differently by the FortiGate due to distinct Web Filter categories, which explains the issue experienced. The solution is straightforward: make sure that all DNS records with the CNAME or ALIAS type associated with a given FQDN have the same category. In the example above, the FQDNs 'shed.dual-low.s-part-0014.t-0009.t-msedge.net' and 's-part-0014.t-0009.t-msedge.net' have to be re-categorized as from 'Business' to 'Information Technology'. To accomplish this, see the related documents below. Related documents: Technical Tip: How to request a category change for a website or domain incorrectly listed under the... Troubleshooting Tip: FortiGuard Web Filtering problems