Technical Tip: How to allow or exempt a URL in a blocked category

Description

This article describes an example of how to create a URL filter rule to 'Allow' or 'Exempt' a particular URL from a 'Blocked' Category in FortiGuard Web Filtering by using regex (Regular Expression).

Scope

FortiGate.

Solution

In the Web-based Manager (GUI):

• Go to Security Profiles -> Web Filter -> Static URL Filter and enable URL Filter.

• Select 'Create New' or select an already available list.

• Select 'Create New' to create an entry for each of the following exempt rules.

URL = .*\.example\.com.* 
Type = regex 
Action = allow

URL= .*\.fortinet\.com.* 
Type= regex 
Action =exempt

• Select OK to save the new URL Filter entry and select OK again to save changes to the Web Filter profile. 

Starting from FortiOS v7.6.3, it is possible to select which feature is exempted from the URL filter.

Command Line Interface (CLI).

config webfilter profile
    edit "URL_Filter"
        config web
            set urlfilter-table 1  
end  

Here, 'urlfilter-table 1' is the table ID that will be edited in the following configuration.

Make sure to double escape special characters with backslashes ('\'). One of them is added automatically by FortiGate, for the same reason.

 

config webfilter urlfilter
    edit 1
        set name "Static URL Filter - URL_Filter Web Filter Profile"
            config entries
                edit 1   
                    set url  ".*\\.example\\.com.*".                                                                                                                                                      
                    set type regex
                    set action allow
                next
                edit 2
                    set url ".*\\.fortinet\\.com.*"
                    set type regex
                next
            end
    next
end

Explanation regarding the use of regex special characters:

To match a special character such as '.' or ‘*’, use the escape character '\'.  For example, to match fortinet.com, the regular expression should be fortinet\.com.

In Perl regular expressions, '*' means match 0 or more times of the character before it, not 0 or more times of any character. For example, forti*.com matches fortiiii.com but does not match fortinet.com.

To match any character 0 or more times, use ‘.*’ where '.' means any character, and the '*' means 0 or more times. For example, the wildcard match pattern forti*.com should therefore be forti.*\.com.

Note: The admin could report that the exempt action is not working. The configuration is done through FortiManager.

FortiManager offers the option 'referrer-host', but that configuration is unnecessary when trying to exempt a URL because it will cause the condition to not match.

The CLI configuration looks like this when referrer-host is configured. Here, referrer-host should not be configured.

config webfilter urlfilter
    edit x 
        set url "deepl.com"
        set type regex
        set action exempt   
        set referrer-host "deepl.com"  
    end
end

To disable the referrer-host by CLI, it is possible to configure the following:

config webfilter urlfilter
    edit x
        unset referrer-host
    end
end

Related articles: 

• Technical Tip: Explanation of the Allow, Block, Exempt, and Monitor static URL filter actions

• Technical Tip: Using a static URL filter feature to allow/block web sites