Description
This article describes that for the Static URL Filter to work properly when it is activated on the Web Filter profile, it must be defined with the correct type of entry. This article describes the three options that can be chosen, how they operate, and examples of their usage.
Scope
FortiGate (relevant as of FortiOS 7.2).
Solution
Simple: A simple URL Filter entry must be in the format of a standard URL, and they can include sub-domains and paths.
- Examples include: 'fortinet.com', 'fortinet.com/support', 'support.fortinet.com', 'net.com', etc.
- Note that to check/match a URL's path (e.g. 'www.fortinet.com/products/next-generation-firewall'), SSL Deep Inspection must be configured within the Firewall Policy (assuming the traffic is encrypted HTTPS).
- The HTTP Path is in the body of the HTTP request, and HTTPS encrypts that information away from the FortiGate's view (unless Deep Inspection is utilized).
- The FortiGate can still filter based upon the Domain name without needing SSL Deep Inspection, as this name is present in the TLS Certificate used by the HTTPS web server. Unencrypted HTTP traffic also does not require SSL Deep Inspection on the FortiGate.
- It is important to mention that the behavior of the Simple entry will be different depending on if the FortiGate is set for flow-based inspection or proxy-based inspection (either system-wide in FortiOS 6.0 and earlier, or on a per-policy basis in FortiOS 6.2 and later.
- In flow-based inspection, a Simple entry can match (and subsequently block) a given URL if it is a sub-string within that URL:
- For example, 'net.com' in a Simple entry will match both 'net.com' and 'fortinet.com'.
- This also applies to matching within URL paths as well (e.g. 'net.com/prod' will match 'www.fortinet.com/products/next-generation-firewall').
- In proxy-based inspection however, Simple entries are only matched to a given URL if they are a literal/exact match.
- e.g. 'net.com' in a Simple entry will match 'net.com' but not 'fortinet.com'.
Wildcard: A Wildcard entry can be used to flexibly match multiple URLs based on the specified keyword/string. Notably, Wildcard entries do not need to follow a specific structure/format.
- A few notes regarding Wildcard behavior:
- Wildcard entries assess a given URL from left to right, and they will be considered matched as long as there is at least one match found.
- For example, the entry 'for*guard' will match URLs like 'fortiguard.com'.
- A Wildcard star character (*) is needed at the start of the entry if the string is expected to be in the middle of the URL. For example:
- The entry '*or*guard' will match 'fortiguard.com', but 'or*guard' (no leading wildcard star) will not (since fortiguard.com starts with the letter 'f').
- Since the string matching is performed from left to right, a Wildcard star does not appear to be necessary at the end of the entry (e.g. '*fortinet' should work the same as '*fortinet*'.
- Putting 'space' after '*' on an expression (example: * yahoomail.com <----- There is a space between * and yahoomail) will match all URLs.
Note.
For entries formatted as '*.<domain>.<tld>' (e.g. '*.fortinet.com), FortiOS will match to the sub-domains of the URL (e.g. 'support.fortinet.com', '
www.fortinet.com', etc.), and it will
also match to the main domain of that URL (e.g. 'fortinet.com').
- This is true as of FortiOS 6.0.5 and beyond.
Regular Expressions (regex): Regex is used to include one or more URLs related -or not related- to a pattern using some Perl syntax, for example:
- '*' symbol means: match 0 or more times of the character before the symbol, but no match with any character.
- For example: 'fortinet*.com' will match 'fortinetttttttt.com' but not 'fortinetsupport.com'.
- '/i' symbols means: makes the pattern case sensitive.
- For example: '/FORTINET/i' will not match with 'fortinet'.^' symbols means: at the beginning of the string.
- For example: '^fo' will match 'fortinet.com' but not 'support.fortinet.com' or 'notfortinet.com'.
- '.' symbol means: match any single character (can be combined with '*' to match any number of any characters).
- For example: 'fortinet.com' will match 'fortinetacom', 'fortinetbcom', 'fortinetzcom'.
- '\' symbol means: escape a special character (like '.' or '*') so that it acts as a literal character instead.
- For example: 'fortinet\.com' will now match 'fortinet.com' but not 'fortinetacom', or 'fortinetbcom'.
- Combined with the above options: '^forti.*\.com' can match 'fortiguard.com' and 'fortinet.com', but not 'support.fortinet.com' (not starting with 'forti'), 'fortinet.net' (wrong top-level-domain, or 'fortizcom' ('.' is checked as a literal character and does not match 'z').
- For further reference, check out the following third-party documentation regarding Perl Regular Expressions (regex): https://perldoc.perl.org/perlre
- The following is a third-party site useful for testing regular expressions: https://regex101.com/
