Technical TIp: Allow referred URLs for websites

[abalachandran]

Description	This article describes how the 'referrer-host' function on the webfilter URL filter can be used to allow referred URLs from a website whenever webfilters are used.
Scope	FortiProxy, FortiGate.
Solution	The observations below may be observed in scenarios where either of the following are true: A website has been set under the 'Allow' or 'Warning' category, but the referred URLs from the website belong in a 'Warning' or 'Blocked' category. The website URL has been configured with a static URL filter and set to 'Allow', but the referred URLs from the website belong in a 'Warning' or "Blocked" category. In this scenario, the user has configured a webfilter profile with 'FortiGuard Category Based Filter' turned on and most of the categories have 'Action' set to 'Warning' or 'Blocked'. However, when the user chooses to proceed from the FortiGuard warning page, the page fails to load or images and content seem to be missing from it. Example of a page failing to load: FortiGuard warning page: The page does not load after selecting Proceed: Example of images/content missing on the website: FortiGuard warning page: Images/content missing on the page after selecting Proceed: FPX Webfilter Logs: Note that there are many referred URLs from the original site (blog.naver.com & www.blogger.com), which the user tried to access and was blocked from. To resolve the issue: On the webfilter profile, go to Static URL Filter -> URL Filter -> Create New. Configure a wildcard URL '*' with the action set to 'exempt' and configure the referrer-host. Example configuration for the site blog.naver.com: FortiProxy GUI: CLI: Example configuration for the site www.blogger.com: FortiProxy GUI: CLI: Note: The option to add a referrer in the GUI is only available for FortiProxy. For FortiGate, the referrer-host needs to be added via the CLI on the webfilter urlfilter entry.  After adding the config above: Website Pages Load Successfully: blog.naver.com: www.blogger.com:  Note: Details of the referrer urls can be obtained from the webfilter logs by referring to Technical Tip: How to get referrer URI in web filter logs, based on which the referrer-host can be specified in the URL filter accordingly.