Technical Tip: Using a static URL filter feature to allow/block web sites

[mzainuddinahm]

Description

This article explains how to exempt or block access to a website using the URL filter feature.

Scope

FortiGate.

Solution

There are three types of URLs that can be defined.

1) Simple: A simple URL filter entry could be a regular URL.

For example: www.fortinet.com
- URL: fortinet.com
- URL: fortinet.com/support

2) Wildcard: A wildcard can be used to include one or more URLs to a simple URL
For example:
- URL: *.fortinet.com (everything before ".fortinet.com" will match this rule, like support.fortinet.com)
- URL: www.fortinet.com/* (everything after "www.fortinet.com/" will match this rule, like www.fortinet.com/contact)

3) Regular Expressions (regex): Regex is used to include one or more URLs related -or not related- to a pattern using some Perl syntax
For example:- "*" symbol means: match 0 or more times of the character before the symbol, but no match with any character.

For example:"fortinet*.com" will match "fortinetttttttt.com" but not "fortinetsupport.com"
"/i" symbols means: makes the pattern case sensitive.

For example:"/FORTINET/i" will not match with "fortinet"
"^" symbols means: at the beginning of the string.

For example:"^fo" will match 'fortinet.com'
'.' symbol means: match the same or different character than the one before the symbol, but is followed by the rest of the sentence.

For example:'fortinet.com' will match 'fortinetacom', 'fortinetbcom', 'fortinetzcom'

Configuring a URL filter:

GUI:

1) Go to Security Profiles -> Web Filter.
2) Select a web filter to edit.
3) Under Static URL Filter, enable URL Filter, and select Create New.
4) Enter the URL, without the “http”, for example: www.example*.com
5) Select a Type: Simple , Regular Expression, or Wildcard. In this example, select Wildcard
6) Select the Action to take against matching URLs: Exempt, Block, Allow, or Monitor.
7) Select 'Enable'.
8) Select 'OK'.

CLI:

The syntax in the CLI for configuring an entry is:

#config webfilter urlfilter
edit <ID>
config entries
edit 1
set url <url>
set referrer-host <url>
set type {simple | regex | wildcard}
set action {block | allow | monitor | exempt}
set status {enable | disable}
end
end

end

To attach the URL filter table to an existing or a new webfilter profile:

# config webfilter profile
    edit "webfilter"               <----- Name of the web filter profile.
        # config web
            set urlfilter-table 1  <----- Where x is the URL filter table ID, this number can be found in '3 config webfilter urlfilter' the URL filter created with ID number.
        end
        # config ftgd-wf
            unset options
        end
    next
end

If exempt is only needed from Fortiguard filtering then 'set exempt fortiguard' can be used, instead of all

For all exempt actions: ? is used to show all the available options:
# set exempt
av --> Antivirus filtering.
web-content --> Web filter content matching.
activex-java-cookie --> ActiveX, Java, and cookie filtering.
dlp --> DLP scanning.
fortiguard --> FortiGuard web filtering.
range-block --> Exempt range block feature.
pass --> Pass single connection from all.
all --> Exempt from all.

Note :
Some sites will be using multiple sub-domains which fall under different FortiGuard category so it will be required to exempt all sub-domains as well in order to access the site.

To check the sub-domains used by a particular site, check browser developer tools.
In Chrome -> Ctrl+Shift+I -> Sources : Here it is possible to check all the sub-domain details.
SSL/SSH deep/full inspection is mandatory for static URL filter working.