URL still blocked after allowing in web filter & DNS filter

[simonl]

Hi all,

A weird one. I've added a URL in the web filter and DNS filter and set it to allow. However the URL still gets blocked. I've also added a web rating override to make it unrated and still no luck. I'm not sure if this is a bug (couldn't find evidence of one), or if I'm missing something.

Web Page Blocked!

You have tried to access a web page which is in violation of your internet usage policy.

URL: ***** Category: Spam URLs User name: Group name:

Any help is appreciated.

[hubertzw]

Can you show us logs from Web and DNS? There should be information about policy ID, security profile name etc.

[simonl]

Ah, the web filter logs show that the request passes through

Profile Name: default

Request Type: direct

Direction: outgoing

URL Filter Index: 1

URL Filter List: default

Message: URL was allowed because it is in the URL filter list

Then the next entry says it's been blocked

Profile Name: default

Request Type: direct

Direction: outgoing

Method: domain

Category: 86

Category Description: Spam URLs

Message: URL belongs to a denied category in policy

So that makes even less sense. The web filtering policy is quoted in both the pass through and blocked log entries.

[hubertzw]

What is your software version?

Inspection order 1) static URL filter 2) FortiGuard category filter 3) advanced filters

so the check should stop on the 1st entry - static URL filter and permit the traffic.

Just to be sure: the logs came from the same policy ID and from the same profile, right? Web or dns?

[simonl]

Software version is 6.0.2 build0163. Fortigate 100E

Correct, the entries are created from the same policy ID and profile. These are taken from the web filter log.

The DNS query logs don't show anything interesting. AAAA and A query types. A couple of entries say the "domain was allowed because it is in the domain-filter list." So that doesn't look like the issue.

[hubertzw]

I found two bugs: 486171 and 490377 here:

https://fortinetweb.s3.amazonaws.com/docs.fortinet.com/v2/attachments/d3b43c6c-1a20-11e9-9685-f8bc12...

486171 - The "Web Rating Overrides" doesn't work with flow-mode. in 6.0.2 both should be resolved but in 6.0.4 I see 486171  again as resolved I know the bugs are not exactly what we see but I would try to upgrade to the 6.0.5 I don't see any known issues for web filtering: https://docs.fortinet.com/document/fortigate/6.0.5/fortios-release-notes/933609/known-issues

[emnoc]

And is your URL subscription up to date ? I would verify web filtering is "green" and no "?" on the dashboard. If the web-filtering is not active where items like this can happen.

Ken Felix

[simonl]

Yes, the web filtering license is up to date.

I'll give the firmware update a go and see if that resolves it.

[Dave_Hall · Accepted solution]

Assuming URL filtering hasn't changed that much since the 4.3 days, setting the URL to allow will still subject the URL to other UTM rules, you may want to Exempt the URL assuming it is a trusted site.   reclassifying a URL to unrated has it's own problems, depending how the fgt handles those type of sites (either blocks or allows) by default.  You may have better luck assigning a local rating or reclassifying the url as a known category (such as a government site).