Skip to main content
sviusa
New Member
June 3, 2018
Solved

[IPS] Crashing all the time on FortiOS 6

  • June 3, 2018
  • 6 replies
  • 27035 views

Hello,

 

I recently tried the FortiOS 6.0 and I have to rollback to 5.6.4.

After upgrade, I notice high CPU usage, IDP is constantly crashing and made every policy using AV + WebFilter + APP unusable.

I have not the same behavior on 5.6 version currently running.

 

I have tried to create another APP/Webfilter, removing AV the policy, enable it on IPv4 policy only but the pb was the same.

 

Does anyone face this kind of pb ?

 

Here an extract of my system log.

 

date=2018-06-02 time=13:30:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527939029 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99" date=2018-06-02 time=13:30:29 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527939028 logdesc="Application crashed" action="crash" msg="Pid: 00259, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32122730] [0x32121d4c] [0x32121fb8] [0x321200a8] [0x321276fc] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:30:18 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527939018 logdesc="Application crashed" action="crash" msg="Pid: 00257, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:30:07 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527939007 logdesc="Application crashed" action="crash" msg="Pid: 00256, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:56 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938995 logdesc="Application crashed" action="crash" msg="Pid: 00250, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:45 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938985 logdesc="Application crashed" action="crash" msg="Pid: 00248, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:30 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938970 logdesc="Application crashed" action="crash" msg="Pid: 00239, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938969 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99" date=2018-06-02 time=13:29:20 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938959 logdesc="Application crashed" action="crash" msg="Pid: 00238, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:10 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938950 logdesc="Application crashed" action="crash" msg="Pid: 00237, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:29:00 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938940 logdesc="Application crashed" action="crash" msg="Pid: 00233, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:28:51 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938930 logdesc="Application crashed" action="crash" msg="Pid: 00226, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:28:41 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938920 logdesc="Application crashed" action="crash" msg="Pid: 00225, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864ca0] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:28:31 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938910 logdesc="Application crashed" action="crash" msg="Pid: 00220, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32122730] [0x32121d4c] [0x32121fb8] [0x321200a8] [0x321276fc] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:28:14 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938894 logdesc="Application crashed" action="crash" msg="Pid: 00212, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:28:03 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938883 logdesc="Application crashed" action="crash" msg="Pid: 00210, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:27:53 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938873 logdesc="Application crashed" action="crash" msg="Pid: 00201, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:27:41 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938861 logdesc="Application crashed" action="crash" msg="Pid: 00197, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:27:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938849 logdesc="CPU usage statistics" action="cpu-usage" cpu=98 msg="CPU usage reaches: 98" date=2018-06-02 time=13:27:25 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938845 logdesc="Application crashed" action="crash" msg="Pid: 00195, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:27:16 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938836 logdesc="Application crashed" action="crash" msg="Pid: 00190, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:27:06 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938826 logdesc="Application crashed" action="crash" msg="Pid: 00185, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:26:53 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938813 logdesc="Application crashed" action="crash" msg="Pid: 00178, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:26:43 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938802 logdesc="Application crashed" action="crash" msg="Pid: 00174, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32122730] [0x32121d4c] [0x32121fb8] [0x321200a8] [0x321276fc] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:26:32 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938792 logdesc="Application crashed" action="crash" msg="Pid: 00173, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32122730] [0x32121d4c] [0x32121fb8] [0x321200a8] [0x321276fc] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:26:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938789 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99" date=2018-06-02 time=13:26:21 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938781 logdesc="Application crashed" action="crash" msg="Pid: 00163, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:26:07 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938767 logdesc="Application crashed" action="crash" msg="Pid: 00162, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:25:52 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938752 logdesc="Application crashed" action="crash" msg="Pid: 00152, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:25:37 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938736 logdesc="Application crashed" action="crash" msg="Pid: 00151, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:25:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938729 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99" date=2018-06-02 time=13:25:28 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938727 logdesc="Application crashed" action="crash" msg="Pid: 00150, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:25:19 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938718 logdesc="Application crashed" action="crash" msg="Pid: 00144, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:25:07 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938706 logdesc="Application crashed" action="crash" msg="Pid: 00140, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864ca0] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:58 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938697 logdesc="Application crashed" action="crash" msg="Pid: 00139, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:50 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938688 logdesc="Application crashed" action="crash" msg="Pid: 00137, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:40 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938679 logdesc="Application crashed" action="crash" msg="Pid: 00136, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:31 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938670 logdesc="Application crashed" action="crash" msg="Pid: 00132, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938669 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99" date=2018-06-02 time=13:24:22 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938660 logdesc="Application crashed" action="crash" msg="Pid: 00131, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:24:12 logid="0100032546" type="event" subtype="system" level="warning" vd="root" eventtime=1527938651 logdesc="Application crashed" action="crash" msg="Pid: 00125, application: ipsengine 04.012, Firmware: FortiGate-90D-POE v6.0.0,build0076b0076,180329 (GA) (Release), Signal 11 received, Backtrace: [0x320f9c80] [0x32129598] [0x32121808] [0x3212055c] [0x32127488] [0x3212ccf0] [0x320d5ee4] [0x31f4f5d0] [0x31fd96a4] [0x31f4a32c] [0x00862abc] [0x00864bfc] [0x0086674c] [0x00029328] [0x0086757c] [0x00867bd4] [0x00029328] [0x0002eecc] [0x0002c304] [0x0002c654] [0x0002e1a0] [0x0002eabc] [0x301c3bc4]" date=2018-06-02 time=13:23:29 logid="0100040705" type="event" subtype="system" level="notice" vd="root" eventtime=1527938609 logdesc="CPU usage statistics" action="cpu-usage" cpu=99 msg="CPU usage reaches: 99"

 

 

Regards,

 

 

Best answer by RockIT

NBE-FW1 # get sys status | grep "build" Version: FortiGate-90D v6.0.0,build0076,180329 (GA)

 

NBE-FW1 # diagnose autoupdate versions | grep "IPS Attack Engine" -A2 IPS Attack Engine --------- Version: 4.00012 NBE-FW1 #

6 replies

Nicholas_Doropoulos
New Member
June 3, 2018

This issue seems to be a known one that can occur on several models and it's probably due to a new signature provided by FortiOS6. Could you do the following during a maintenance window:

 

1) Upgrade back to version 6.

 

2) Update AV & IPS Definitions under System >> Fortiguard. 

 

3) If the update doesn't fix the issue, run the following command:

 

diag test application ipsmonitor 99 

 

This command will restart the IPS engine.

 

4) Failing that, try $disabling the "use extended IPS signature package" option under System >> Fortiguard.

 

Let us know of the results.

heisenberg
New Member
June 4, 2018

Hi, 

I have a 6.0GA on a production 100E and yes....if are planning to do it...please DON'T.

Full of bug and abnormal behaviour.

sslvpnd takes an entire cpu core...you have to reboot the device periodically.

problems with ips, web filtering, certificates.

Gui is sluggish and filters works bad (they do not remember values if you change trough menus).

Hope a first (big) patch will be out there in little time.

 

darwin_FTNT
Staff
Staff
June 4, 2018

The latest internal build for ipsengine is 4.019 and I saw some commit for crash fixes (mainly affecting arm fsoc2 cpu but not x86 or arm fsoc3 cpu).  This is due to many features added in ipsengine specially for v6 build.  Hopefully the next v6 firmware update or the fortiguard updates will have fix for ipsengine libips.so as it stabilizes/optimized.

 

IPS engine daemon loads libips.so and handles all flow-based utm profiles (av, ips, app control, dlp,  webfilter, spamfilter, etc.).  All the utm profiles have flow-based (done in ipsengine daemon) or proxy-based (by wad daemon and likely others) modes.  The only utm profiles that don't have proxy-based counterpart are the ips and app-control utm.  These are ipsengine only feature.  Thus if using flow-based utm, the debugging commands are usually, diag ips debug * etc.

 

Due to flow-based operation, if IPS engine crashed, the packets in the queue are just flushed and there will be no packet loss.  Exceptions are when doing SSL deep inspection because this requires mitm processing (similar to proxy-based mode operation).  In this case, the sessions states are lost and affected when crash occurs. Also ips may bypass the sessions once done scanning.

 

RockIT
New Member
June 5, 2018

I got the same thing going on my 90D's 

ericli_FTNT
Staff
Staff
June 5, 2018

It seems this is a critical issue. Please leave your FortiOS build number, ipsengine number and device model. I will report to dev immediately.

You can find build number by

get sys status | grep "build"

ipsengine number

diagnose autoupdate versions | grep "IPS Attack Engine" -A2

RockIT
New Member
June 5, 2018

Version 6.0 on Fotigate 90D

ericli_FTNT
Staff
Staff
June 5, 2018

It seems this is a critical issue. Please leave your FortiOS build number, ipsengine number and device model. I will report to dev immediately.

You can find build number by

get sys status | grep "build"

ipsengine number

diagnose autoupdate versions | grep "IPS Attack Engine" -A2

Deepakkhw
New Member
June 17, 2018

Hi,

You have to book a ticket with FortiGate. The support team will go through the logs and get into the Root cause. As I know your CPU is going high. It may be due to IPS or IPS is going fail due to High CPU. 

 

Regards, Deepak Kumar

sviusa
sviusaAuthor
New Member
July 21, 2018

Hello All,

 

I have tried the update today with version 6.0.1, and CPU is not rising to the sky this time.

Seems to be related to BUG 482835.

I will post if the issue is happening again.

 

Regards,

 

 

 

robdeep
New Member
November 13, 2018

I am experiencing similar symptoms with 6.0.3 in a Hyper-V instance. If I apply an IPS policy to a single device on the LAN, the Firewall will lock up with 100% usage, and I have to force a reboot. It will do the same just by enabling "Block Malicious URLs" in the IPS sensor. Anyone else still on version 6 with a stable IPS policy?