Fortigate process " wad" consuming 62% of memory.

[plindgren]

i get the " CFG_CMDBAPI_ERR" when i try to make changes on my fortigate. which is other than that operational. I tired the command " diag test application ipsmonitor 99" but it did not work. So i used the command " diag sys top 1" to see what was hogging all that memory. And i found a process named " wad" that uses 62% of the memory. However this machine is in production and i dont know what the process does and i cant seem to find it anywhere. Can i kill it? What does it do? Is there a process reference for fortios out there somewhere?

[pcraponi]

wad daemon is used for cache, explict-proxy and wan optimization. You use some of these resources? You are on 4.0 MR3 firmware? The next patch (patch15) will improve some wad resources... No ETA yet for release date. Regards, Paulo Raponi

[plindgren]

We do use it, however it wont really matter if that function " goes down" since we have a good internetline which we never really had any problems with bandwidth(even prior to using wan opt). My question simply is, what will happen when i kill the process? Can it affect the whole system or is it safe to just kill it? Also how did you find out that wad was that? i cant seem to find any documentation of what all these processes do. Would be nice for future troubleshooting. edit: im on FortiOS 5.0.2 Cant even turn off web cache and wan opt on my policy objects >.<

[plindgren]

I talked to Fortinet webchat and he told me it would be a problem to kill it i ran diag sys kill 11 <pid> where pid was process id. And it worked fine my fortigate is functioning normally

[pcraponi]

Be carefully. When will kill the wad process, all users lost their HTTP/HTTPS access / downloads, etc... All sessions need be restarted. Regards, Paulo Raponi

[IyyappanD]

@Paulo: Is there another than killing it ? Production was impacted already and Internet doesn't work. So seems better to having it killed and this seems to be a common issue till 5.4.10 which we are using. Not sure if the same is resolved in 5.6.X. Can someone comment on this

[sashag]

Same with 5.6.6 With upgrade from 5.6.3 and flow inspection mode to 5.6.6 and proxy mode, "wad" process ate 40% of memory in less than 10 hours. After reaching 90% of memory consumption fortigate entered "conserve mode" which killed all internet connections in office. Had to kill process and return to flow mode for further investigation.

[IyyappanD]

We had an issue on 5.4.4 and were asked to upgrade 5.4.10 and did not solve the issue. Right now, we are doing failover and rebooting the affected box. This is an ongoing issue and i thought issue is resolved at 5.6.6.

So is there a Solution to this now ?

[james_heyworth]

Hi all

We are on 5.6.6 1500D's and we experienced the same issue on both of our Firewalls (1500D's) and the WAD process was the culprit. I submitted a ticket to Fortinet to investigate, and provide us with a version that this issue is remediated in. I will keep the forum updated if possible.

[AtiT]

We have the same problem on our 1500Ds FGT with 5.6.6 version of OS. It is configmed as a bug:

Bug is fixed in FortiOS 6.0.3 GA and interim build 5.6.7

We have to restart every 3-5 days the WAD process, sometimes the worker sometimes other process.

When the 5.6.7 will be out the support could not tell as they did not have this info when I asked.

We have a customer with 1000Ds in cluster and had the same issue on 5.6.4 and we updated it to 5.6.5 almost two months ago. They have did not experienced any issue so far. That is the reason we are not recommended to upgrade to 5.6.6 as it is confirmed having a bug.