Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
Jeroen
Contributor

Crashing of Application WAD

 

Hello to you all,

 

I have a problem with crashing application within the Fortigate 1500D. So far I can see it is the WAD application (Cache & Wan optimization deamon). The strange thing that some traffic seems to have problems with it crashing. The next strange thing is that the feature “WAN Link Load Balancing” isn’t activated. What can be the cause of this application to be crashing all the time.

 

Firmware: V5.2.1

Type Fortigate: 1500D

 

Has somebody have a idea?

 

date=2014-12-31 time=13:24:02 logid=0100032546 type=event subtype=system level=warning vd="root" logdesc="Application crash" action=crash msg="Pid: 28447, application: wad, Firmware: FortiGate-1500D v5.2.1,build0618b618,140915 (GA) (Release), Signal 11 received, Backtrace: [0x011ad238] [0x0120cab8] [0x0120cfc0] [0x012136fb] [0x0126ee7a] [0x0126f0a6] [0x0126e5ed] [0x012a1a78] [0x0043d35c] [0x0043a3e3] [0x2a95c40475] [0x0043a889]" date=2014-12-31 time=13:20:50 logid=0100032546 type=event subtype=system level=warning vd="root" logdesc="Application crash" action=crash msg="Pid: 28442, application: wad, Firmware: FortiGate-1500D v5.2.1,build0618b618,140915 (GA) (Release), Signal 11 received, Backtrace: [0x011ad238] [0x0120cab8] [0x0120cfc0] [0x012136fb] [0x0126ee7a] [0x0126f0a6] [0x0126e5ed] [0x012a1a78] [0x0043d35c] [0x0043a3e3] [0x2a95c40475] [0x0043a889]" date=2014-12-31 time=13:19:52 logid=0100032546 type=event subtype=system level=warning vd="root" logdesc="Application crash" action=crash msg="Pid: 28441, application: wad, Firmware: FortiGate-1500D v5.2.1,build0618b618,140915 (GA) (Release), Signal 11 received, Backtrace: [0x011ad238] [0x0120cab8] [0x0120cfc0] [0x012136fb] [0x0126ee7a] [0x0126f0a6] [0x0126e5ed] [0x012a1a78] [0x0043d35c] [0x0043a3e3] [0x2a95c40475] [0x0043a889]" date=2014-12-31 time=13:19:02 logid=0100032546 type=event subtype=system level=warning vd="root" logdesc="Application crash" action=crash msg="Pid: 28435, application: wad, Firmware: FortiGate-1500D v5.2.1,build0618b618,140915 (GA) (Release), Signal 11 received, Backtrace: [0x011ad238] [0x0120cab8] [0x0120cfc0] [0x012136fb] [0x0126ee7a] [0x0126f0a6] [0x0126e5ed] [0x012a1a78] [0x0043d35c] [0x0043a3e3] [0x2a95c40475] [0x0043a889]" date=2014-12-31 time=13:18:08 logid=0100032546 type=event subtype=system level=warning vd="root" logdesc="Application crash" action=crash msg="Pid: 28431, application: wad, Firmware: FortiGate-1500D v5.2.1,build0618b618,140915 (GA) (Release), Signal 11 received, Backtrace: [0x011ad238] [0x0120cab8] [0x0120cfc0] [0x01216efa] [0x012252d4] [0x0121378c] [0x0126ee7a] [0x0126f0a6] [0x0126e5ed] [0x012a1a78] [0x0043d35c] [0x0043a3e3] [0x2a95c40475] [0x0043a889]"

 

 

 

1 Solution
Jeroen

We have updated the 1500D unit to version 5.2.2 and this solved the issue at our side. We have planned a update to version 5.2.3 as I see your message this is maybe not a good idea as the problems seems to return at version 5.2.3. I am curious at what Fortinet will return in the ticket.

 

ISOffice wrote:

Hi all,

 

We have had an on-going issue with web caching with our 100D Cluster (v5.2.3, build 670) and I'm starting to see the same error messages as Jeroen. Due to caching not working on v5.2.2, we upgraded to v5.2.3. I have a call logged with Fortinet in relation to this issue and will post any further developments.

 

John P

View solution in original post

15 REPLIES 15
dieter
New Contributor

In our case the crashes seem to be related to specific ssl decrypted traffic.

TA suggested upgrading to 5.6.3, but without much explanation (bug report or other related issue)...

rickyrickuk

dieter wrote:

In our case the crashes seem to be related to specific ssl decrypted traffic.

TA suggested upgrading to 5.6.3, but without much explanation (bug report or other related issue)...

We are getting these multiple times a minute (FG500E) during peak times and every couple of minutes during off peak, we are on 5.6.3 already so not sure that will fix.

zack

Anyone find a solution to this? We have a Fortigate 200D cluster that is doing the same thing. The WAD daemon keeps crashing when we have UTM policies in place. It was happening on 5.6.8 and support recommended we upgrade. We're on 5.6.12 now and it's still happening. So far support is stumped.... If anyone came across a solution, sharing it here would be much appreciated. 

 

We're not using any caching..

(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
kphed
New Contributor III

zack wrote:
Anyone find a solution to this? We have a Fortigate 200D cluster that is doing the same thing. The WAD daemon keeps crashing when we have UTM policies in place. It was happening on 5.6.8 and support recommended we upgrade. We're on 5.6.12 now and it's still happening. So far support is stumped.... If anyone came across a solution, sharing it here would be much appreciated.    We're not using any caching..
Same issue on 5.6.11!! Any help?!
zack
New Contributor

I opened a ticket with Fortinet and they acknowledged it was a bug. Furthermore, since our 200Ds were end of life and not receiving firmware updates anymore - there would be no "official" fix. Since I was under a support agreement and I escalated the issue (since they could offer no solution), Fortinet ended up creating a custom firmware image off of the 6.0.9 branch for us which contained the fix. 

 

The ticket number was:

3909060

 

We had to run for an extended time with our proxy based inspection down which meant no UTM policies. I was NOT happy about that. 

 

But there's good news. Fortinet sold us some new firewalls at a discount because of all the issues. We got 3 100Fs. Bad news - the WAD service still crashes on them too running 6.2.4. I have a ticket in on that one too. Ticket # 4088435. Still no fix!

 

My next firewall will NOT be a Fortinet. I buy these firewalls to run UTM policies. If I can't run them because it causes the firewall to crash, then WTF am I paying for? I can download a basic port filtering firewall/router for free! (Untangle of PFSense).

 

I know it sucks to hear, but my only recommendation can be to buy another brand at this point. 

(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
zack
New Contributor

You have to disable proxy based inspection (SSL inspection). Set it to certificate only as opposed to full SSL inspection. By the way - this will disable the effectiveness of your UTM policies since the firewall can't look inside SSL sessions to inspect the traffic. But at least they'll stop crashing. 

 

You need to engage Fortinet support and ask for a custom firmware image for your firewalls that contain the fix. 

(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
(2) FortiGate 300A (clustered) 4.2.9 (1) Fortigate 310B 4.2.9 (1) Fortianalyzer 100C 4.2.4
Labels
Top Kudoed Authors