Strange Web Filter Issue Related To Google Searches/Prepending

[BWiebe]

So - scenario.

We have a client running 5.2.1 and using Fortiguard web filtering, in flow mode, to block a number of categories outright and they've whitelisted a number of sites that fall onto a number of those categories.  This generally works well and they have no issues getting into Youtube, when they've blocked the streaming media category (youtube is whitelisted).

The whitelisted sites are done using regexp, like .*youtube\.com for example to capture youtube.com entries, and the various other permutations of youtube that exist.

What they've recently noticed is, that if they happen to do a google search for a video, or something related to one of the other whitelisted sites and click on the link, the link itself comes back as blocked.  I've tested and confirmed the same behavior.  The URL gets a bunch of google detail prepended to it (like the following) https://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&cad=rja&uact=8&sqi=2&ved=0CCkQjBAwAg&... and this seems to confuse the lookup against the whitelisted entry.

Has anyone seen this sort of behaviour, and does anyone have a way to correct this?

I've also opened a case with TAC about this.

Thanks!

[Adrian_Buckley_FTNT]

The link is actually a redirect to Youtube,  so there is no confusion.

Web Filtering operates on the information in the HTTP Get or the SNI (if you aren't deep inspecting)

This looks perfectly normal.

[BWiebe]

I agree - it's a redirect to Youtube.

We allow youtube, via whitelist and regex entry.

The redirect is being blocked and I want to find a way to prevent that from happening.

[Adrian_Buckley_FTNT]

What do the logs show it being blocked for, or as?

[BWiebe]

The logs show that google.ca was blocked because it is part of Streaming Media.

Same goes for whichever other sites are being blocked - it points it as google and whichever blocked category.

Had a call with TAC and we have a temp fix of exempting *google*, which is strange, but they believe I've uncovered a bug in the way the traffic is being seen.

[Adrian_Buckley_FTNT]

google.ca or *.google.ca?

*.google.ca is the CA that shows up on all of googles certificates.  If there's no SNI, it's what would be expected to show up in the logs and be acted on IF you were not doing SSL Deep Inspection.

I don't think you mention if you were doing SSL Inspection or CA inspection,  or am i just missing that?

[BWiebe]

Adrian Buckley_FTNT wrote:

google.ca or *.google.ca?

*.google.ca is the CA that shows up on all of googles certificates.  If there's no SNI, it's what would be expected to show up in the logs and be acted on IF you were not doing SSL Deep Inspection.

I don't think you mention if you were doing SSL Inspection or CA inspection,  or am i just missing that?

It showed up in the logs as www.google.ca and was blocked because of the blocked category so I believe it was incorrectly blocking it.

We aren't doing SSL Inspection or CA inspection and this was happening with https and http links until we added the *google* wildcard exemption (which seems like a strange fix to me when www.google.ca is not blocked).

Thanks

[Adrian_Buckley_FTNT]

In the Web Profile there is the option to "rate the URL by Domain and IP address".  Do you have that enabled?

If the category of the IP does not match the category of the URL, then the ForitiGate uses an internal weighting system to decide which category to use for rating.

Some categories are considering "worse" then others so the category with the highest weight is checked against the Profile actions.

[BWiebe]

Adrian Buckley_FTNT wrote:

In the Web Profile there is the option to "rate the URL by Domain and IP address".  Do you have that enabled?

If the category of the IP does not match the category of the URL, then the ForitiGate uses an internal weighting system to decide which category to use for rating.

Some categories are considering "worse" then others so the category with the highest weight is checked against the Profile actions.

No - that setting is not enabled for the policy.

[mramon79]

Hi, There is a known bug in 5.2.1 related to url filter exeptions.Its going to be solved in 5.2.2 but only with reg.expressions.Universal wildcards like *domain.com. are not going to be accepted. regards