Skip to main content
rmetzger
Staff
rmetzger
Staff
May 14, 2009

Troubleshooting Tip: Web filtering rating problems when using FortiGuard rating by IP and URL

  • May 14, 2009
  • 0 replies
  • 35447 views

Description

 

This article describes the scenario where the FortiGuard Web Filtering option "Rate URLs by domain and IP address" is enabled.

In this situation, the rating response from a FortiGuard Distribution Server (FDS) for a particular URL might differ from its IP address. This is very common in scenarios of Virtual Hosting, where one IP address of one physical server will host multiple services and URLs.

Therefore, if the IP address rating belongs to a blocked category, access to the URL will be blocked regardless of the rating of the URL.

Summary

  • How to check if a URL gets two different ratings, one for the IP address and one for the URL.
  • How to look for category numbers.
  • How to make a live verification of rating response:

 

How to check if a URL gets two different ratings, one for the IP address and one for the URL.


Use the following command to dump the FortiGuard WEB cache:
 
diagnose test application urlfilter 3    <-- Only works if WEB filtering cache is enabled.

Example of output:
 

diagnose test application urlfilter 3

Saving to file [/tmp/urcCache.txt]

Cache Contents:

-=-=-=-=-=-=-=-

Cache Mode:   TTL

Cache DB Ver: 93.4437

Domain |IP       DB Ver T URL

29000000|34000000 93.4437 E http://www.mytestrating.fr/

34000000|34000000 13.28635 E http://www.fortinet.com/


Notes:
  • In the above example, the domain www.mytestrating.fr is in category (Hex) 29, while the IP address is in category (Hex) 34.
  • The numbers above are given in hexadecimal, while in the FortiGate CLI configuration, the numbers are displayed in decimal. The www.fortinet.com domain will therefore be in category Hex34 = Dec52 (see example of category below, 52 = Information Technology).
 
How to look for category numbers:
The numbers displayed in the columns IP and domains are the rating returned by the FortiGuard servers. The corresponding categories can be retrieved as explained in the related article: FortiGuard Web Filtering Category and Classification numbers / FortiGate configuration and troubleshooting.

Example:
 
config firewall profile
    edit TEST
(TEST) get
 
ftgd-wf-enable      :
  g01 Potentially Liable:
      1 Drug Abuse
      2 Occult
      3 Hacking
      4 Illegal or Unethical
      5 Racism and Hate
      6 Violence
     57 Marijuana
     58 Folklore
[...]
  g07 Business Oriented:
     49 Business
     50 Information and Computer Security
     51 Government and Legal Organizations
     52 Information Technology
     53 Armed Forces
[...]
 

How to make a live verification of rating response:

Use the following CLI commands to check the rating requests and responses:
 
diagnose debug enable
diagnose debug application urlfilter -1        <-- To stop it: diagnose debug application urlfilter 0.
## id=93000 msg="pid=57 urlfilter_main-723 in main.c received pkt:count=91, a=/tmp/.thttp.socket/21" id=22009 msg="received a request /tmp/.thttp.socket, addr_len=21: d= ="www.goodorg.org:80, id=12853, vfid=0, type=0, client=192.168.3.90, url=/" id=99501 user="N/A" src=192.168.3.90 sport=1321 dst=<dest_ip> dport=80 service="http" cat=43 cat_desc=“Organisation" hostname="www.goodorg.org" url="/" status=blocked msg="URL belongs to a denied category in policy"
 
Tested on v7.0 and later:
 Web Filter Configuration: The Job Search category is set to block for demonstration.
 
2024-09-20 18 52 56.png

 

Upon accessing https://careers.floridadental.org/, the page is blocked. The category is Job Search.
 
blockpage.png

 

Perform an NSlookup:
 

>nslookup careers.floridadental.org 8.8.8.8
Server: dns.google
Address: 8.8.8.8

Non-authoritative answer:
Name: cname.boxwoodtech.com
Address: 144.202.255.70
Aliases: careers.floridadental.org

 

Check the category on the FortiGate:

 

Security Profiles -> Web Rating Overrides -> Create New -> Enter URL -> Lookup rating.

 

2024-09-20 18 56 02.png

 

2024-09-20 18 56 23.png
 

diagnose webfilter fortiguard cache dump
Caution: This command is for diagnostic purposes ONLY. The bigger the cache size is set, the more impact on performance the command has.
Do you want to continue? (y/n)y


Saving to file [/tmp/urcCache.txt]

Cache Contents:
-=-=-=-=-=-=-=-
Cache Mode: TTL
Cache DB Ver: 233.50234

Rating DB Ver DOT SLASH ORIG_FLAG T URL

22000000|22000000 233.50234 0 0 00000001 P Ahttp://144.202.255.70/
21000000|21000000 233.50234 1 0 00000001 P Ahttp://careers.floridadental.org/

........

 

22 Hex is 34 in Decimal.

21 Hex is 33 in Decimal.

 

get webfilter categories | grep 34
34 Job Search

get webfilter categories | grep 33
33 Health and Wellness

 

Solution:

If the rating for an IP address blocks access to a site, the solution is to disable 'Rate URLs by domain and IP address'. Alternatively, the IP address can be overridden to a different category that is allowed.

Related documents:

Rate site by URL and IP address

Verify the webfilter cache content 

Troubleshooting FortiGuard

FortiGuard Web Filtering Category and Classification numbers / FortiGate configuration and troubleshooting

FortiGuard Web Filtering Override Guide ; configuration examples

Terms & ConditionsAccessibility statement