FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Juancacst
Staff
Staff
Article Id 382794
Description This article explains the behavior of Web Filtering on FortiGate, where connections to a website may be intermittently blocked or allowed when multiple websites share the same IP address. 
Scope All currently supported FortiOS versions.
Solution

Every time a new site is accessed, the IP address for this site is added to the web filtering cache along with its category if it does not already exist in the cache. WAD and IPS engines will use the category linked to the IP from the web filtering cache to determine whether to allow or block access to the site. 
 
See this article: Troubleshooting Tip: Verify the webfilter cache content

 

A new rating request will be sent to FortiGuard only if there is no cached record in the Webfilter cache for the Website IP address in question. 

 

Here are several configuration scenarios and the expected web filtering behavior on FortiGate. 

 

Flow-Based Policy + Certificate Inspection.

For a new rating request from a user, the IPS engine will use the website's IP address to look up the web filtering cache and determine whether the traffic should be allowed or blocked based on its previous rating. 

 

Flow-Based Policy + Deep Inspection.

IPS engine uses the SNI information in the client Hello to determine if the traffic should be allowed or blocked regardless of the web filtering cache. If SNI is not available, the Common Name of the server certificate will be used. 

 

Proxy-Based Policy + Certificate Inspection.

For a new rating request from a user, the WAD daemon on FortiGate will first check the IP address of the website in web filtering cache to determine whether the traffic should be allowed or blocked based on its previous rating. 

 

Proxy-Based Policy + Deep Inspection.

WAD daemon on FortiGate will first check the IP address of the site in the web filtering cache to determine whether the traffic should be allowed or blocked based on its previous rating. 

 

For example:
Internet Services such as google.com often use the same IP for multiple sites like chat.google.com, calendar.google.com, etc. This can cause the IP of a site, which is configured to be blocked, to be added to the web filtering cache with a category that is allowed. 

 

Also, navigating to yahoo.com loads several URLs at once adding multiple IP addresses with the category of yahoo.com (Search Engines and Portals). When a user then tries to access mail.yahoo.com, if this site resolves to an IP that is already in the web filtering cache, it will be allowed even if a static URL is configured to block mail.yahoo.com. 

 

Related document:

Virtual IPs with port forwarding