Terrible perfomance with Web filtering

[KingBarries]

Hi - I would like to get some input from the community... We are using fortigate 60C firewall with AD integration. We activated the UTM Web filter for a certain group. However the performance is terrible. Sites time out and sometimes cant even be reached. CPU usage at 20% and Memory at 55% about when tested. It seems like timeouts - although I cannot establish what causes this. The profiles that are not web filtered or do not use UTM runs 100%. Any advice?

[Carl_Wallmark]

Hi, In this case I would check communication with FortiGuard, search for it on kb.fortinet.com

[SAG_perimeter]

The Fortigate' s ability to use WCF is dependent upon the systems ability to query a database of URLs. Each time it sees a web-access attempt it searches its cache to see if it already knows the category for that url. If it does not have the url in the cache it generates a UDP query to that database and asks for the current info about the url. If the Fortigate cannot reach that database it can, essentially, hold the traffic until it times-out. It sounds like this is what you' re seeing. By default the Fortigate tries to go to, I believe, " service.fortiguard.net" to find that database. If it cannot resolve that name properly it will not be able to complete the url queries and you' ll see the type of behaviour you' re describing. Check your Fortigate' s DNS settings and make sure you' ve got addresses for functioning public DNS servers listed. Also, from the CLI, can you ping (by name) " service.fortiguard.net" ? If not, once again, review your DNS settings on the Fortigate. Of course I' m assuming you' ve got a valid and active WCF license on the Fortigate...

[KingBarries]

Hi That is the strange thing - I have checked with ISP and there are no filtering on the UDP ports at all. It is on a best effort basis however. If I do what that article says I also get 20+ IP address of where the databases could sit. Yes all licenses are in place. Any other ideas? Resources seem to be managable... Thanks

[Dave_Hall]

Just off the top of my head... If the Fortigate is doing NAT make sure it is enabled on the firewall policy. (Assuming the Fortigate is on MR3 firmware...) For the UTM Web filter Profile, is it set to Proxy or Flow-based? What options are checked under " Advanced Filter" ? (You may need to check " Allow Websites When a Rating Error Occurs" .) Check the Protocol Options that the firewall policy uses to ensure HTTP/HTTPS are using the proper ports. You may need to check " Comfort Clients" .

[ede_pfau]

Good point! The default setting is port 0 for HTTP, HTTPS meaning these protocols will be detected on any port. This might cause a higher CPU load than necessary. BTW, WF is always ' flow mode' AFAIK.

[abc987]

@KingBarries Have you solved this issue? If so, what was the problem?

[FortiRack_Eric]

WcF is per default proxy mode and can be set to flow mode. Beware that flow mode has limitations (especially in 4.x) as you can present the user a replacement message they will receive a HTTP xxx error

[KingBarries]

Hi I think it is somewhat better - seems when we block social for example then facebook will be blocked. Not sure if the result is cached but the next requests are instant. We haven' t pin pointed anything. We are now trying to add a second fortigate to do only the web filtering on our WAN. The routing, IPS etc will be done on the main fortigate. @Dave Hall - I have tried to check the Allow websites when rating error occurs. I think this might solve the timeout issue.. Will report. I have a valid license but so far we sent logs to fortinet support but withou much luck. Thanks for all the suggestions! The 60C unit should cater for 100 odd users right? Thanks

[KingBarries]

@ede_pfau We have entered the correct ports and checked comfort clients.. What does the comfort clients actually do?