When is 5.4.1 going to drop?

[ITGuy11]

Is there an ETA as to when 5.4.1 is going to drop?  I have a brand new 300D that I am waiting to put into production as soon as 5.4.1 is ready.

[FGTuser · Accepted solution]

by end of next week (April 15)

[seadave]

Typical feedback from support.  "Install 5.4.1".  I have it running on my backup unit.  I'm heading out of town for a week Saturday and I'm not that sadistic so I'll wait until I get back to see if I can swap over and see how it performs.  Based on other posts, SSLScanning is an issue with 5.4.1?

[seadave]

Here's my output from diag sys session stat:

misc info:       session_count=2800 setup_rate=30 exp_count=0 clash=35         memory_tension_drop=0 ephemeral=0/589824 removeable=0 delete=0, flush=0, dev_down=0/0 TCP sessions:          699 in ESTABLISHED state          2 in SYN_SENT state          7 in FIN_WAIT state          31 in TIME_WAIT state          37 in CLOSE state          56 in CLOSE_WAIT state firewall error stat: error1=00000000 error2=00000000 error3=00000000 error4=00000000 tt=00000000 cont=00808264 ids_recv=01611a87 url_recv=00000000 av_recv=02100bad fqdn_count=00000177 global: ses_limit=0 ses6_limit=0 rt_limit=0 rt6_limit=0

Doesn't seem off the chart, but not really educated on how to read this.  This is after-hours so many folks are not here at this point in the day.

[kallbrandt · Accepted solution]

That amount of clashes is nothing to worry about I'd say. On the LB-vdom I mentioned earlier the log shows 6-digit amounts of clashes. The clash counter is reset at reboot btw, and is not related to the current amount of sessions. It is just an ongoing counter.

To my knowledge, all restarts of applications with restart option 11 (segmentation fault) in FortiOS is seen as a crash. It doesn't have to mean anything bad per se. The OS recycles processes all the time using option 15 (graceful restart). When that doesn't work, it moves on to try to restart with option 11 wich will generate a log entry in the syslog. The recycle process continues all the time, buffers needs to be cleared etc etc. However, a constant restarting of the same application can also mean various problems - Memory leaks, buffer overflows etc.

I checked your log, but I can't see anything else then the PID and some weird ASCII-signs as application name. It does look kinda odd.

Check your logs and keep track of if the application crash log entries correlates with odd behaviour in the firewall, we're talking sudden reboots, functions and features stopping/not working.

What does "diagnose debug crashlog read" say?

Also, do a  "diagnose sys top", a few times during the day. Do you have processes in Z or D state?

[Holy]

Hello,

after Upgrading an 50E and 30E to 5.4.1 we now have no Botnet and C&C Lizenz anymore :(

its showin red and we cannot configure DNS Security Profile anymore either.

Had Fortinet made some License changes with 5.4.1 ?

we had licensed UTM Bundle + FortiSandbox cloud inspektion.

have someone the same issue?

Thanks

[kallbrandt]

Sounds very strange. Haven't seen anything like it on those I've upgraded to 5.4.1. But haven't touched 50E nor 30E. Waiting for 4*51E's to arrive...

[seadave]

kallbrandt wrote:

That amount of clashes is nothing to worry about I'd say. On the LB-vdom I mentioned earlier the log shows 6-digit amounts of clashes. The clash counter is reset at reboot btw, and is not related to the current amount of sessions. It is just an ongoing counter.

To my knowledge, all restarts of applications with restart option 11 (segmentation fault) in FortiOS is seen as a crash. It doesn't have to mean anything bad per se. The OS recycles processes all the time using option 15 (graceful restart). When that doesn't work, it moves on to try to restart with option 11 wich will generate a log entry in the syslog. The recycle process continues all the time, buffers needs to be cleared etc etc. However, a constant restarting of the same application can also mean various problems - Memory leaks, buffer overflows etc.

I checked your log, but I can't see anything else then the PID and some weird ASCII-signs as application name. It does look kinda odd.

Check your logs and keep track of if the application crash log entries correlates with odd behaviour in the firewall, we're talking sudden reboots, functions and features stopping/not working.

What does "diagnose debug crashlog read" say?

Also, do a  "diagnose sys top", a few times during the day. Do you have processes in Z or D state?

Thanks for taking the time to look.  I'm on "vacation" right now, but still have ticket with the TAC opened.  They may do a remote session with me today.  One thing I found strange with 5.4 was that my IPS logs dropped to virtually nil, whereas when I was on 5.2 I had all sorts of events from threat actors trying crap.  I did not attempt to reuse my 5.2 config when I migrated to 5.4.  I rebuilt it by hand, only importing address lists as I was trying to get as clean of a config as possible.  I did not and do not see any "config error log" results when I reboot.

But all of these issues do point to an IPS problem.  I just want to know what changed on 6/12.  My system was stable until that day.  I can't see how adding an address entry would be the cause, especially as I undid the change I made and the system still entered conserve mode after a period of ~30 hours.  The daily reboot has alleviated the unplanned conserve mode but isn't a good long term solution.  Once TAC has a look, I'll post results.

Here's a current "dia sys top"

Run Time:  0 days, 6 hours and 24 minutes 1U, 2N, 0S, 97I, 0WA, 0HI, 0SI, 0ST; 7996T, 5029F              wad     9357      S       2.7     0.6              wad     9358      S       1.9     0.7        ipsengine     9368      S <     0.7     1.6        ipsengine     9367      S <     0.7     1.5        scanunitd    10356      S <     0.7     0.3           proxyd    15993      S       0.5     3.8           proxyd    15658      S       0.5     3.6        ipsengine     9366      S <     0.5     1.5        scanunitd    10623      S <     0.5     0.3        scanunitd    10757      S <     0.5     0.3         dnsproxy     9322      S       0.3     0.3        urlfilter     9315      S       0.1     0.3          miglogd     9377      S       0.1     0.2           fnbamd     9291      S       0.1     0.1        scanunitd     9306      S <     0.0     0.4          miglogd     9283      S       0.0     0.3          cmdbsvr     6827      S       0.0     0.3          pyfcgid    23137      S       0.0     0.3        ipshelper     9330      S <     0.0     0.3           httpsd    21558      S       0.0     0.2

Looks normal to me.  But that is the issue.  System will be fine, using minimal RAM and CPU even at the busiest part of our work day and all of a sudden, CPU and MEM spike and then Conserve mode.

Here is part of my crash log for today (sorry this is so long):

fortigate # dia debug crashlog read 1: 2016-06-27 11:21:58 proxyworker - watchdog timeout 2: 2016-06-27 11:21:58 <29081> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 3: 2016-06-27 11:21:58 <29081> application proxyd 4: 2016-06-27 11:21:58 <29081> *** signal 6 (Aborted) received *** 5: 2016-06-27 11:21:58 <29081> Register dump: 6: 2016-06-27 11:21:58 <29081> RAX: 0000000000000000   RBX: 0000000000021ce7 7: 2016-06-27 11:21:58 <29081> RCX: ffffffffffffffff   RDX: 0000000000000006 8: 2016-06-27 11:21:58 <29081> R8:  0000000000007199   R9:  0000000000000006 9: 2016-06-27 11:21:58 <29081> R10: 0000000000000008   R11: 0000000000000246 10: 2016-06-27 11:21:58 <29081> R12: 000000000000001a   R13: 000000000499abc0 11: 2016-06-27 11:21:58 <29081> R14: 000000000499ac90   R15: 00007fff60aae850 12: 2016-06-27 11:21:58 <29081> RSI: 0000000000007199   RDI: 0000000000007199 13: 2016-06-27 11:21:58 <29081> RBP: 00007fff60aae1f0   RSP: 00007fff60aae0a8 14: 2016-06-27 11:21:58 <29081> RIP: 00007f5165648c4b   EFLAGS: 0000000000000246 15: 2016-06-27 11:21:58 <29081> CS:  0033   FS: 0000   GS: 0000 16: 2016-06-27 11:21:58 <29081> Trap: 0000000000000000   Error: 0000000000000000 17: 2016-06-27 11:21:58 <29081> OldMask: 0000000002000000 18: 2016-06-27 11:21:58 <29081> CR2: 0000000000000000 19: 2016-06-27 11:21:58 <29081> Backtrace: [size="3"]20: 2016-06-27 11:21:58 <29081> [0x7f5165648c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 21: 2016-06-27 11:21:58 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]22: 2016-06-27 11:21:58 <29081> [0x7f516564a108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 23: 2016-06-27 11:21:58 liboffset 00036108 [size="3"]24: 2016-06-27 11:21:58 <29081> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]25: 2016-06-27 11:21:58 <29081> [0x7f5165648d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]26: 2016-06-27 11:21:58 <29081> [0x01a15550] => /bin/proxyd  [/size] [size="3"]27: 2016-06-27 11:21:58 <29081> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]28: 2016-06-27 11:21:58 <29081> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]29: 2016-06-27 11:21:58 <29081> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]30: 2016-06-27 11:21:58 <29081> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]31: 2016-06-27 11:21:58 <29081> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]32: 2016-06-27 11:21:58 <29081> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]33: 2016-06-27 11:21:58 <29081> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]34: 2016-06-27 11:21:58 <29081> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]35: 2016-06-27 11:21:58 <29081> [0x7f5165635475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 36: 2016-06-27 11:21:58 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]37: 2016-06-27 11:21:58 <29081> [0x0043a9f1] => /bin/proxyd  [/size] 38: 2016-06-27 11:21:58 <09303> proxyapp=proxyd000 pid=29081 exittype=signal code=6 39: 2016-06-27 11:21:58 <09303> total=7996 free=5587 shmfs_total=6285 shmfs_free=6283 40: 2016-06-27 11:27:08 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 41: 2016-06-27 11:27:09 <09306> scanunit=manager str="Success loading anti-virus database." 42: 2016-06-27 11:57:58 the killed daemon is /bin/getty: status=0x0 43: 2016-06-27 11:58:39 <25433> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 44: 2016-06-27 11:58:39 <25433> application ipsengine 03.164 45: 2016-06-27 11:58:39 <25433> *** signal 11 (Segmentation fault) received *** 46: 2016-06-27 11:58:39 <25433> Register dump: 47: 2016-06-27 11:58:39 <25433> RAX: 0000000000000006   RBX: 00007fefe6e2306c 48: 2016-06-27 11:58:39 <25433> RCX: 0000000000000006   RDX: 0000000000000007 49: 2016-06-27 11:58:39 <25433> R8:  00007fefe6e2316c   R9:  00007fefe6ad7c80 50: 2016-06-27 11:58:39 <25433> R10: 00007fff7bfb1aa0   R11: 0000000000000000 51: 2016-06-27 11:58:39 <25433> R12: 0000000000000007   R13: 0000000000000006 52: 2016-06-27 11:58:39 <25433> R14: 00007fef9e89bb00   R15: 00007fefb27df000 53: 2016-06-27 11:58:39 <25433> RSI: 000000000006e17e   RDI: 00007fefb27ffffd 54: 2016-06-27 11:58:39 <25433> RBP: 0000000000000006   RSP: 00007fff7bfb19d8 55: 2016-06-27 11:58:39 <25433> RIP: 00007fefe68ec235   EFLAGS: 0000000000010293 56: 2016-06-27 11:58:39 <25433> CS:  0033   FS: 0000   GS: 0000 57: 2016-06-27 11:58:39 <25433> Trap: 000000000000000e   Error: 0000000000000004 58: 2016-06-27 11:58:39 <25433> OldMask: 0000000000000800 59: 2016-06-27 11:58:39 <25433> CR2: 00007fefb2800003 60: 2016-06-27 11:58:39 <25433> Backtrace: [size="3"]61: 2016-06-27 11:58:39 <25433> [0x7fefe68ec235] => /data/lib/libips.so  liboffset 00128235[/size] [size="3"]62: 2016-06-27 11:58:39 <25433> [0x7fefe68550a5] => /data/lib/libips.so  liboffset 000910a5[/size] [size="3"]63: 2016-06-27 11:58:39 <25433> [0x7fefe685b865] => /data/lib/libips.so  liboffset 00097865[/size] [size="3"]64: 2016-06-27 11:58:39 <25433> [0x7fefe6873239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]65: 2016-06-27 11:58:39 <25433> [0x7fefe687f0a3] => /data/lib/libips.so  liboffset 000bb0a3[/size] [size="3"]66: 2016-06-27 11:58:39 <25433> [0x7fefe687fdaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]67: 2016-06-27 11:58:39 <25433> [0x7fefe6825c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]68: 2016-06-27 11:58:39 <25433> [0x7fefe683813e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]69: 2016-06-27 11:58:39 <25433> [0x7fefe6804920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]70: 2016-06-27 11:58:39 <25433> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]71: 2016-06-27 11:58:39 <25433> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]72: 2016-06-27 11:58:39 <25433> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]73: 2016-06-27 11:58:39 <25433> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]74: 2016-06-27 11:58:39 <25433> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]75: 2016-06-27 11:58:39 <25433> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]76: 2016-06-27 11:58:39 <25433> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]77: 2016-06-27 11:58:39 <25433> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]78: 2016-06-27 11:58:39 <25433> [0x00443557] => /bin/ipsengine  [/size] [size="3"]79: 2016-06-27 11:58:39 <25433> [0x00441520] => /bin/ipsengine  [/size] [size="3"]80: 2016-06-27 11:58:39 <25433> [0x00443188] => /bin/ipsengine  [/size] [size="3"]81: 2016-06-27 11:58:39 <25433> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]82: 2016-06-27 11:58:39 <25433> [0x7fefe9a60475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 83: 2016-06-27 11:58:39 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]84: 2016-06-27 11:58:39 <25433> [0x0043a9f1] => /bin/ipsengine  [/size] 85: 2016-06-27 11:59:09 <04730> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 86: 2016-06-27 11:59:09 <04730> application ipsengine 03.164 87: 2016-06-27 11:59:09 <04730> *** signal 11 (Segmentation fault) received *** 88: 2016-06-27 11:59:09 <04730> Register dump: 89: 2016-06-27 11:59:09 <04730> RAX: 0000000000000006   RBX: 00007fefe6e2306c 90: 2016-06-27 11:59:09 <04730> RCX: 0000000000000006   RDX: 0000000000000007 91: 2016-06-27 11:59:09 <04730> R8:  00007fefe6e2316c   R9:  00007fefe6ad7c80 92: 2016-06-27 11:59:09 <04730> R10: 00007fff7bfb1aa0   R11: 0000000000000000 93: 2016-06-27 11:59:09 <04730> R12: 0000000000000007   R13: 0000000000000006 94: 2016-06-27 11:59:09 <04730> R14: 00007fefa38ed500   R15: 00007fefb27df000 95: 2016-06-27 11:59:09 <04730> RSI: 000000000003e01a   RDI: 00007fefb27ffffd 96: 2016-06-27 11:59:09 <04730> RBP: 0000000000000006   RSP: 00007fff7bfb19d8 97: 2016-06-27 11:59:09 <04730> RIP: 00007fefe68ec235   EFLAGS: 0000000000010297 98: 2016-06-27 11:59:09 <04730> CS:  0033   FS: 0000   GS: 0000 99: 2016-06-27 11:59:09 <04730> Trap: 000000000000000e   Error: 0000000000000004 100: 2016-06-27 11:59:09 <04730> OldMask: 0000000000000800 101: 2016-06-27 11:59:09 <04730> CR2: 00007fefb2800003 102: 2016-06-27 11:59:09 <04730> Backtrace: [size="3"]103: 2016-06-27 11:59:09 <04730> [0x7fefe68ec235] => /data/lib/libips.so  liboffset 00128235[/size] [size="3"]104: 2016-06-27 11:59:09 <04730> [0x7fefe68550a5] => /data/lib/libips.so  liboffset 000910a5[/size] [size="3"]105: 2016-06-27 11:59:09 <04730> [0x7fefe685b865] => /data/lib/libips.so  liboffset 00097865[/size] [size="3"]106: 2016-06-27 11:59:09 <04730> [0x7fefe6873239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]107: 2016-06-27 11:59:09 <04730> [0x7fefe687f0a3] => /data/lib/libips.so  liboffset 000bb0a3[/size] [size="3"]108: 2016-06-27 11:59:09 <04730> [0x7fefe687fdaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]109: 2016-06-27 11:59:09 <04730> [0x7fefe6825c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]110: 2016-06-27 11:59:09 <04730> [0x7fefe683813e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]111: 2016-06-27 11:59:09 <04730> [0x7fefe6804920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]112: 2016-06-27 11:59:09 <04730> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]113: 2016-06-27 11:59:09 <04730> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]114: 2016-06-27 11:59:09 <04730> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]115: 2016-06-27 11:59:09 <04730> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]116: 2016-06-27 11:59:09 <04730> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]117: 2016-06-27 11:59:09 <04730> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]118: 2016-06-27 11:59:09 <04730> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]119: 2016-06-27 11:59:09 <04730> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]120: 2016-06-27 11:59:09 <04730> [0x00443557] => /bin/ipsengine  [/size] [size="3"]121: 2016-06-27 11:59:09 <04730> [0x00441520] => /bin/ipsengine  [/size] [size="3"]122: 2016-06-27 11:59:09 <04730> [0x00443188] => /bin/ipsengine  [/size] [size="3"]123: 2016-06-27 11:59:09 <04730> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]124: 2016-06-27 11:59:09 <04730> [0x7fefe9a60475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 125: 2016-06-27 11:59:09 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]126: 2016-06-27 11:59:09 <04730> [0x0043a9f1] => /bin/ipsengine  [/size] 127: 2016-06-27 12:08:13 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 128: 2016-06-27 12:08:14 <09306> scanunit=manager str="Success loading anti-virus database." 129: 2016-06-27 12:44:41 the killed daemon is /bin/pyfcgid: status=0x0 130: 2016-06-27 12:57:22 proxyworker - watchdog timeout 131: 2016-06-27 12:57:22 <02681> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 132: 2016-06-27 12:57:22 <02681> application proxyd 133: 2016-06-27 12:57:22 <02681> *** signal 6 (Aborted) received *** 134: 2016-06-27 12:57:22 <02681> Register dump: 135: 2016-06-27 12:57:22 <02681> RAX: 0000000000000000   RBX: 000000000008afd5 136: 2016-06-27 12:57:22 <02681> RCX: ffffffffffffffff   RDX: 0000000000000006 137: 2016-06-27 12:57:22 <02681> R8:  0000000000000a79   R9:  0000000000000006 138: 2016-06-27 12:57:22 <02681> R10: 0000000000000008   R11: 0000000000000246 139: 2016-06-27 12:57:22 <02681> R12: 000000000000001a   R13: 0000000004893550 140: 2016-06-27 12:57:22 <02681> R14: 00000000048935e0   R15: 00007fff298024d0 141: 2016-06-27 12:57:22 <02681> RSI: 0000000000000a79   RDI: 0000000000000a79 142: 2016-06-27 12:57:22 <02681> RBP: 00007fff29801e30   RSP: 00007fff29801ce8 143: 2016-06-27 12:57:22 <02681> RIP: 00007f97dc75cc4b   EFLAGS: 0000000000000246 144: 2016-06-27 12:57:22 <02681> CS:  0033   FS: 0000   GS: 0000 145: 2016-06-27 12:57:22 <02681> Trap: 0000000000000000   Error: 0000000000000000 146: 2016-06-27 12:57:22 <02681> OldMask: 0000000002000000 147: 2016-06-27 12:57:22 <02681> CR2: 0000000000000000 148: 2016-06-27 12:57:22 <02681> Backtrace: [size="3"]149: 2016-06-27 12:57:22 <02681> [0x7f97dc75cc4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 150: 2016-06-27 12:57:22 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]151: 2016-06-27 12:57:22 <02681> [0x7f97dc75e108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 152: 2016-06-27 12:57:22 liboffset 00036108 [size="3"]153: 2016-06-27 12:57:22 <02681> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]154: 2016-06-27 12:57:22 <02681> [0x7f97dc75cd00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]155: 2016-06-27 12:57:22 <02681> [0x00f946d9] => /bin/proxyd  [/size] [size="3"]156: 2016-06-27 12:57:22 <02681> [0x00f95f11] => /bin/proxyd  [/size] [size="3"]157: 2016-06-27 12:57:22 <02681> [0x01a15493] => /bin/proxyd  [/size] [size="3"]158: 2016-06-27 12:57:22 <02681> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]159: 2016-06-27 12:57:22 <02681> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]160: 2016-06-27 12:57:22 <02681> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]161: 2016-06-27 12:57:22 <02681> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]162: 2016-06-27 12:57:22 <02681> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]163: 2016-06-27 12:57:22 <02681> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]164: 2016-06-27 12:57:22 <02681> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]165: 2016-06-27 12:57:22 <02681> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]166: 2016-06-27 12:57:22 <02681> [0x7f97dc749475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 167: 2016-06-27 12:57:22 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]168: 2016-06-27 12:57:22 <02681> [0x0043a9f1] => /bin/proxyd  [/size] 169: 2016-06-27 12:57:22 <09303> proxyapp=proxyd000 pid=2681 exittype=signal code=6 170: 2016-06-27 12:57:22 <09303> total=7996 free=5464 shmfs_total=6285 shmfs_free=6277 171: 2016-06-27 13:42:08 proxyworker - watchdog timeout 172: 2016-06-27 13:42:08 <29082> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 173: 2016-06-27 13:42:08 <29082> application proxyd 174: 2016-06-27 13:42:08 <29082> *** signal 6 (Aborted) received *** 175: 2016-06-27 13:42:08 <29082> Register dump: 176: 2016-06-27 13:42:08 <29082> RAX: 0000000000000000   RBX: 00000000000ef27e 177: 2016-06-27 13:42:08 <29082> RCX: ffffffffffffffff   RDX: 0000000000000006 178: 2016-06-27 13:42:08 <29082> R8:  000000000000719a   R9:  0000000000000006 179: 2016-06-27 13:42:08 <29082> R10: 0000000000000008   R11: 0000000000000246 180: 2016-06-27 13:42:08 <29082> R12: 000000000000001a   R13: 00000000046fb900 181: 2016-06-27 13:42:08 <29082> R14: 00000000046fb900   R15: 00007fff371dcee0 182: 2016-06-27 13:42:08 <29082> RSI: 000000000000719a   RDI: 000000000000719a 183: 2016-06-27 13:42:08 <29082> RBP: 00007fff371dc870   RSP: 00007fff371dc728 184: 2016-06-27 13:42:08 <29082> RIP: 00007f443eb12c4b   EFLAGS: 0000000000000246 185: 2016-06-27 13:42:08 <29082> CS:  0033   FS: 0000   GS: 0000 186: 2016-06-27 13:42:08 <29082> Trap: 0000000000000000   Error: 0000000000000000 187: 2016-06-27 13:42:08 <29082> OldMask: 0000000002000000 188: 2016-06-27 13:42:08 <29082> CR2: 0000000000000000 189: 2016-06-27 13:42:08 <29082> Backtrace: [size="3"]190: 2016-06-27 13:42:08 <29082> [0x7f443eb12c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 191: 2016-06-27 13:42:08 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]192: 2016-06-27 13:42:08 <29082> [0x7f443eb14108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 193: 2016-06-27 13:42:08 liboffset 00036108 [size="3"]194: 2016-06-27 13:42:08 <29082> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]195: 2016-06-27 13:42:08 <29082> [0x7f443eb12d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]196: 2016-06-27 13:42:08 <29082> [0x01a15563] => /bin/proxyd  [/size] [size="3"]197: 2016-06-27 13:42:08 <29082> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]198: 2016-06-27 13:42:08 <29082> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]199: 2016-06-27 13:42:08 <29082> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]200: 2016-06-27 13:42:08 <29082> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]201: 2016-06-27 13:42:08 <29082> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]202: 2016-06-27 13:42:08 <29082> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]203: 2016-06-27 13:42:08 <29082> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]204: 2016-06-27 13:42:08 <29082> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]205: 2016-06-27 13:42:08 <29082> [0x7f443eaff475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 206: 2016-06-27 13:42:08 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]207: 2016-06-27 13:42:08 <29082> [0x0043a9f1] => /bin/proxyd  [/size] 208: 2016-06-27 13:42:08 <09303> proxyapp=proxyd001 pid=29082 exittype=signal code=6 209: 2016-06-27 13:42:08 <09303> total=7996 free=5546 shmfs_total=6285 shmfs_free=6283 210: 2016-06-27 14:26:46 <13505> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 211: 2016-06-27 14:26:46 <13505> application ipsengine 03.164 212: 2016-06-27 14:26:46 <13505> *** signal 11 (Segmentation fault) received *** 213: 2016-06-27 14:26:46 <13505> Register dump: 214: 2016-06-27 14:26:46 <13505> RAX: 00007fefb2870369   RBX: 000000000009136a 215: 2016-06-27 14:26:46 <13505> RCX: 000000000009136a   RDX: 00007fefb27df000 216: 2016-06-27 14:26:46 <13505> R8:  0000000000001800   R9:  00007fefe9b75e60 217: 2016-06-27 14:26:46 <13505> R10: 00007fff7bfb1a30   R11: 00007fefe9baa280 218: 2016-06-27 14:26:46 <13505> R12: 00007fef9eb35570   R13: 00007fefb27df000 219: 2016-06-27 14:26:46 <13505> R14: 00007fefb27df000   R15: 00007fefa3500918 220: 2016-06-27 14:26:46 <13505> RSI: 00007fefb2791000   RDI: 00007fef9eb35570 221: 2016-06-27 14:26:46 <13505> RBP: 000000000009136a   RSP: 00007fff7bfb19d0 222: 2016-06-27 14:26:46 <13505> RIP: 00007fefe6855118   EFLAGS: 0000000000010206 223: 2016-06-27 14:26:46 <13505> CS:  0033   FS: 0000   GS: 0000 224: 2016-06-27 14:26:46 <13505> Trap: 000000000000000e   Error: 0000000000000004 225: 2016-06-27 14:26:46 <13505> OldMask: 0000000000000800 226: 2016-06-27 14:26:46 <13505> CR2: 00007fefb2870369 227: 2016-06-27 14:26:46 <13505> Backtrace: [size="3"]228: 2016-06-27 14:26:46 <13505> [0x7fefe6855118] => /data/lib/libips.so  liboffset 00091118[/size] [size="3"]229: 2016-06-27 14:26:46 <13505> [0x7fefe6873239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]230: 2016-06-27 14:26:46 <13505> [0x7fefe687ebb4] => /data/lib/libips.so  liboffset 000babb4[/size] [size="3"]231: 2016-06-27 14:26:46 <13505> [0x7fefe687ed51] => /data/lib/libips.so  liboffset 000bad51[/size] [size="3"]232: 2016-06-27 14:26:46 <13505> [0x7fefe687fdaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]233: 2016-06-27 14:26:46 <13505> [0x7fefe6825c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]234: 2016-06-27 14:26:46 <13505> [0x7fefe683813e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]235: 2016-06-27 14:26:46 <13505> [0x7fefe6804920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]236: 2016-06-27 14:26:46 <13505> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]237: 2016-06-27 14:26:46 <13505> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]238: 2016-06-27 14:26:46 <13505> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]239: 2016-06-27 14:26:46 <13505> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]240: 2016-06-27 14:26:46 <13505> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]241: 2016-06-27 14:26:46 <13505> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]242: 2016-06-27 14:26:46 <13505> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]243: 2016-06-27 14:26:46 <13505> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]244: 2016-06-27 14:26:46 <13505> [0x00443557] => /bin/ipsengine  [/size] [size="3"]245: 2016-06-27 14:26:46 <13505> [0x00441520] => /bin/ipsengine  [/size] [size="3"]246: 2016-06-27 14:26:46 <13505> [0x00443188] => /bin/ipsengine  [/size] [size="3"]247: 2016-06-27 14:26:46 <13505> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]248: 2016-06-27 14:26:46 <13505> [0x7fefe9a60475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 249: 2016-06-27 14:26:46 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]250: 2016-06-27 14:26:46 <13505> [0x0043a9f1] => /bin/ipsengine  [/size] 251: 2016-06-27 15:08:16 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 252: 2016-06-27 15:08:16 <09306> scanunit=manager str="Success loading anti-virus database." 253: 2016-06-27 15:10:01 proxyworker - watchdog timeout 254: 2016-06-27 15:10:01 <07214> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 255: 2016-06-27 15:10:01 <07214> application proxyd 256: 2016-06-27 15:10:01 <07214> *** signal 6 (Aborted) received *** 257: 2016-06-27 15:10:01 <07214> Register dump: 258: 2016-06-27 15:10:01 <07214> RAX: 0000000000000000   RBX: 000000000007fc67 259: 2016-06-27 15:10:01 <07214> RCX: ffffffffffffffff   RDX: 0000000000000006 260: 2016-06-27 15:10:01 <07214> R8:  0000000000001c2e   R9:  0000000000000006 261: 2016-06-27 15:10:01 <07214> R10: 0000000000000008   R11: 0000000000000246 262: 2016-06-27 15:10:01 <07214> R12: 000000000000001a   R13: 00000000048d6720 263: 2016-06-27 15:10:01 <07214> R14: 00000000048d67b0   R15: 00007fffa0a233c0 264: 2016-06-27 15:10:01 <07214> RSI: 0000000000001c2e   RDI: 0000000000001c2e 265: 2016-06-27 15:10:01 <07214> RBP: 00007fffa0a22cf0   RSP: 00007fffa0a22ba8 266: 2016-06-27 15:10:01 <07214> RIP: 00007f9852ad3c4b   EFLAGS: 0000000000000246 267: 2016-06-27 15:10:01 <07214> CS:  0033   FS: 0000   GS: 0000 268: 2016-06-27 15:10:01 <07214> Trap: 0000000000000000   Error: 0000000000000000 269: 2016-06-27 15:10:01 <07214> OldMask: 0000000002000000 270: 2016-06-27 15:10:01 <07214> CR2: 0000000000000000 271: 2016-06-27 15:10:01 <07214> Backtrace: [size="3"]272: 2016-06-27 15:10:01 <07214> [0x7f9852ad3c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 273: 2016-06-27 15:10:01 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]274: 2016-06-27 15:10:01 <07214> [0x7f9852ad5108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 275: 2016-06-27 15:10:01 liboffset 00036108 [size="3"]276: 2016-06-27 15:10:01 <07214> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]277: 2016-06-27 15:10:01 <07214> [0x7f9852ad3d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]278: 2016-06-27 15:10:01 <07214> [0x00f94692] => /bin/proxyd  [/size] [size="3"]279: 2016-06-27 15:10:01 <07214> [0x00f95f11] => /bin/proxyd  [/size] [size="3"]280: 2016-06-27 15:10:01 <07214> [0x01a15493] => /bin/proxyd  [/size] [size="3"]281: 2016-06-27 15:10:01 <07214> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]282: 2016-06-27 15:10:01 <07214> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]283: 2016-06-27 15:10:01 <07214> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]284: 2016-06-27 15:10:01 <07214> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]285: 2016-06-27 15:10:01 <07214> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]286: 2016-06-27 15:10:01 <07214> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]287: 2016-06-27 15:10:01 <07214> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]288: 2016-06-27 15:10:01 <07214> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]289: 2016-06-27 15:10:01 <07214> [0x7f9852ac0475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 290: 2016-06-27 15:10:01 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]291: 2016-06-27 15:10:01 <07214> [0x0043a9f1] => /bin/proxyd  [/size] 292: 2016-06-27 15:10:01 <09303> proxyapp=proxyd001 pid=7214 exittype=signal code=6 293: 2016-06-27 15:10:01 <09303> total=7996 free=5550 shmfs_total=6285 shmfs_free=6284 294: 2016-06-27 15:31:31 the killed daemon is /bin/pyfcgid: status=0x0 295: 2016-06-27 15:38:32 proxyworker - watchdog timeout 296: 2016-06-27 15:38:32 <28319> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 297: 2016-06-27 15:38:32 <28319> application proxyd 298: 2016-06-27 15:38:32 <28319> *** signal 6 (Aborted) received *** 299: 2016-06-27 15:38:32 <28319> Register dump: 300: 2016-06-27 15:38:32 <28319> RAX: 0000000000000000   RBX: 00000000000eaaed 301: 2016-06-27 15:38:32 <28319> RCX: ffffffffffffffff   RDX: 0000000000000006 302: 2016-06-27 15:38:32 <28319> R8:  0000000000006e9f   R9:  0000000000000006 303: 2016-06-27 15:38:32 <28319> R10: 0000000000000008   R11: 0000000000000246 304: 2016-06-27 15:38:32 <28319> R12: 000000000000001a   R13: 00000000048c08d8 305: 2016-06-27 15:38:32 <28319> R14: 00000000048c0898   R15: 00007fffe7e400a0 306: 2016-06-27 15:38:32 <28319> RSI: 0000000000006e9f   RDI: 0000000000006e9f 307: 2016-06-27 15:38:32 <28319> RBP: 00007fffe7e3f9b0   RSP: 00007fffe7e3f868 308: 2016-06-27 15:38:32 <28319> RIP: 00007f71fc188c4b   EFLAGS: 0000000000000246 309: 2016-06-27 15:38:32 <28319> CS:  0033   FS: 0000   GS: 0000 310: 2016-06-27 15:38:32 <28319> Trap: 0000000000000000   Error: 0000000000000000 311: 2016-06-27 15:38:32 <28319> OldMask: 0000000002000000 312: 2016-06-27 15:38:32 <28319> CR2: 0000000000000000 313: 2016-06-27 15:38:32 <28319> Backtrace: [size="3"]314: 2016-06-27 15:38:32 <28319> [0x7f71fc188c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 315: 2016-06-27 15:38:32 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]316: 2016-06-27 15:38:32 <28319> [0x7f71fc18a108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 317: 2016-06-27 15:38:32 liboffset 00036108 [size="3"]318: 2016-06-27 15:38:32 <28319> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]319: 2016-06-27 15:38:32 <28319> [0x7f71fc188d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]320: 2016-06-27 15:38:32 <28319> [0x019f9200] => /bin/proxyd  [/size] [size="3"]321: 2016-06-27 15:38:32 <28319> [0x01a15493] => /bin/proxyd  [/size] [size="3"]322: 2016-06-27 15:38:32 <28319> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]323: 2016-06-27 15:38:32 <28319> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]324: 2016-06-27 15:38:32 <28319> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]325: 2016-06-27 15:38:32 <28319> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]326: 2016-06-27 15:38:32 <28319> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]327: 2016-06-27 15:38:32 <28319> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]328: 2016-06-27 15:38:32 <28319> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]329: 2016-06-27 15:38:32 <28319> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]330: 2016-06-27 15:38:32 <28319> [0x7f71fc175475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 331: 2016-06-27 15:38:32 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]332: 2016-06-27 15:38:32 <28319> [0x0043a9f1] => /bin/proxyd  [/size] 333: 2016-06-27 15:38:32 <09303> proxyapp=proxyd000 pid=28319 exittype=signal code=6 334: 2016-06-27 15:38:32 <09303> total=7996 free=5487 shmfs_total=6285 shmfs_free=6284 335: 2016-06-27 15:53:51 the killed daemon is /bin/getty: status=0x0 336: 2016-06-27 17:06:56 the killed daemon is /bin/pyfcgid: status=0x100 337: 2016-06-27 17:17:34 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 338: 2016-06-27 17:17:34 <09306> scanunit=manager str="Success loading anti-virus database." 339: 2016-06-27 17:49:43 the killed daemon is /bin/pyfcgid: status=0x0 340: 2016-06-27 18:08:16 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 341: 2016-06-27 18:08:16 <09306> scanunit=manager str="Success loading anti-virus database." 342: 2016-06-27 21:08:18 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 343: 2016-06-27 21:08:18 <09306> scanunit=manager str="Success loading anti-virus database." 344: 2016-06-27 21:46:06 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 345: 2016-06-27 21:46:06 <09306> scanunit=manager str="Success loading anti-virus database." 346: 2016-06-28 00:05:59 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 347: 2016-06-28 00:05:59 <09306> scanunit=manager str="Success loading anti-virus database." 348: 2016-06-28 01:03:13 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 349: 2016-06-28 01:03:14 <09306> scanunit=manager str="Success loading anti-virus database." 350: 2016-06-28 03:08:16 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 351: 2016-06-28 03:08:17 <09306> scanunit=manager str="Success loading anti-virus database." 352: 2016-06-28 03:46:13 <09309> scanunit=manager str="Success loading anti-virus database." 353: 2016-06-28 05:00:51 scanunit=manager pid=9309 str="AV database changed (1); restarting workers" 354: 2016-06-28 05:00:51 <09309> scanunit=manager str="Success loading anti-virus database." 355: 2016-06-28 05:45:32 proxyworker - watchdog timeout 356: 2016-06-28 05:45:32 <09305> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 357: 2016-06-28 05:45:32 <09305> application proxyd 358: 2016-06-28 05:45:32 <09305> *** signal 6 (Aborted) received *** 359: 2016-06-28 05:45:32 <09305> Register dump: 360: 2016-06-28 05:45:32 <09305> RAX: 0000000000000000   RBX: 00000000000ad794 361: 2016-06-28 05:45:32 <09305> RCX: ffffffffffffffff   RDX: 0000000000000006 362: 2016-06-28 05:45:32 <09305> R8:  0000000000002459   R9:  0000000000000006 363: 2016-06-28 05:45:32 <09305> R10: 0000000000000008   R11: 0000000000000246 364: 2016-06-28 05:45:32 <09305> R12: 000000000000001a   R13: 000000000594d098 365: 2016-06-28 05:45:32 <09305> R14: 000000000594d058   R15: 00007fff5180af30 366: 2016-06-28 05:45:32 <09305> RSI: 0000000000002459   RDI: 0000000000002459 367: 2016-06-28 05:45:32 <09305> RBP: 00007fff5180a830   RSP: 00007fff5180a6e8 368: 2016-06-28 05:45:32 <09305> RIP: 00007f3605665c4b   EFLAGS: 0000000000000246 369: 2016-06-28 05:45:32 <09305> CS:  0033   FS: 0000   GS: 0000 370: 2016-06-28 05:45:32 <09305> Trap: 0000000000000000   Error: 0000000000000000 371: 2016-06-28 05:45:32 <09305> OldMask: 0000000002000000 372: 2016-06-28 05:45:32 <09305> CR2: 0000000000000000 373: 2016-06-28 05:45:32 <09305> Backtrace: [size="3"]374: 2016-06-28 05:45:32 <09305> [0x7f3605665c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 375: 2016-06-28 05:45:32 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]376: 2016-06-28 05:45:32 <09305> [0x7f3605667108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 377: 2016-06-28 05:45:32 liboffset 00036108 [size="3"]378: 2016-06-28 05:45:32 <09305> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]379: 2016-06-28 05:45:32 <09305> [0x7f3605665d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]380: 2016-06-28 05:45:32 <09305> [0x019f9218] => /bin/proxyd  [/size] [size="3"]381: 2016-06-28 05:45:32 <09305> [0x01a15493] => /bin/proxyd  [/size] [size="3"]382: 2016-06-28 05:45:32 <09305> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]383: 2016-06-28 05:45:32 <09305> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]384: 2016-06-28 05:45:32 <09305> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]385: 2016-06-28 05:45:32 <09305> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]386: 2016-06-28 05:45:32 <09305> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]387: 2016-06-28 05:45:32 <09305> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]388: 2016-06-28 05:45:32 <09305> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]389: 2016-06-28 05:45:32 <09305> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]390: 2016-06-28 05:45:32 <09305> [0x7f3605652475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 391: 2016-06-28 05:45:32 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]392: 2016-06-28 05:45:32 <09305> [0x0043a9f1] => /bin/proxyd  [/size] 393: 2016-06-28 05:45:32 <09303> proxyapp=proxyd000 pid=9305 exittype=signal code=6 394: 2016-06-28 05:45:32 <09303> total=7996 free=5734 shmfs_total=6285 shmfs_free=6284 395: 2016-06-28 06:02:53 scanunit=manager pid=9309 str="AV database changed (1); restarting workers" 396: 2016-06-28 06:02:53 <09309> scanunit=manager str="Success loading anti-virus database." 397: 2016-06-28 06:42:09 scanunit=manager pid=9309 str="AV database changed (1); restarting workers" 398: 2016-06-28 06:42:10 <09309> scanunit=manager str="Success loading anti-virus database." 399: 2016-06-28 08:36:27 proxyworker - watchdog timeout 400: 2016-06-28 08:36:27 <07549> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 401: 2016-06-28 08:36:27 <07549> application proxyd 402: 2016-06-28 08:36:27 <07549> *** signal 6 (Aborted) received *** 403: 2016-06-28 08:36:27 <07549> Register dump: 404: 2016-06-28 08:36:27 <07549> RAX: 0000000000000000   RBX: 00000000000f97ed 405: 2016-06-28 08:36:27 <07549> RCX: ffffffffffffffff   RDX: 0000000000000006 406: 2016-06-28 08:36:27 <07549> R8:  0000000000001d7d   R9:  0000000000000006 407: 2016-06-28 08:36:27 <07549> R10: 0000000000000008   R11: 0000000000000246 408: 2016-06-28 08:36:27 <07549> R12: 000000000000001a   R13: 000000000560a460 409: 2016-06-28 08:36:27 <07549> R14: 000000000560a460   R15: 00007fffa90586d0 410: 2016-06-28 08:36:27 <07549> RSI: 0000000000001d7d   RDI: 0000000000001d7d 411: 2016-06-28 08:36:27 <07549> RBP: 00007fffa9058070   RSP: 00007fffa9057f28 412: 2016-06-28 08:36:27 <07549> RIP: 00007f40b7fb4c4b   EFLAGS: 0000000000000246 413: 2016-06-28 08:36:27 <07549> CS:  0033   FS: 0000   GS: 0000 414: 2016-06-28 08:36:27 <07549> Trap: 0000000000000000   Error: 0000000000000000 415: 2016-06-28 08:36:27 <07549> OldMask: 0000000002000000 416: 2016-06-28 08:36:27 <07549> CR2: 0000000000000000 417: 2016-06-28 08:36:27 <07549> Backtrace: [size="3"]418: 2016-06-28 08:36:27 <07549> [0x7f40b7fb4c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 419: 2016-06-28 08:36:27 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]420: 2016-06-28 08:36:27 <07549> [0x7f40b7fb6108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 421: 2016-06-28 08:36:27 liboffset 00036108 [size="3"]422: 2016-06-28 08:36:27 <07549> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]423: 2016-06-28 08:36:27 <07549> [0x7f40b7fb4d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]424: 2016-06-28 08:36:27 <07549> [0x019f9000] => /bin/proxyd  [/size] [size="3"]425: 2016-06-28 08:36:27 <07549> [0x01a15493] => /bin/proxyd  [/size] [size="3"]426: 2016-06-28 08:36:27 <07549> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]427: 2016-06-28 08:36:27 <07549> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]428: 2016-06-28 08:36:27 <07549> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]429: 2016-06-28 08:36:27 <07549> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]430: 2016-06-28 08:36:27 <07549> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]431: 2016-06-28 08:36:27 <07549> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]432: 2016-06-28 08:36:27 <07549> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]433: 2016-06-28 08:36:27 <07549> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]434: 2016-06-28 08:36:27 <07549> [0x7f40b7fa1475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 435: 2016-06-28 08:36:27 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]436: 2016-06-28 08:36:27 <07549> [0x0043a9f1] => /bin/proxyd  [/size] 437: 2016-06-28 08:36:27 <09303> proxyapp=proxyd000 pid=7549 exittype=signal code=6 438: 2016-06-28 08:36:27 <09303> total=7996 free=5376 shmfs_total=6285 shmfs_free=6282 439: 2016-06-28 09:05:41 <09370> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 440: 2016-06-28 09:05:41 <09370> application ipsengine 03.164 441: 2016-06-28 09:05:41 <09370> *** signal 11 (Segmentation fault) received *** 442: 2016-06-28 09:05:41 <09370> Register dump: 443: 2016-06-28 09:05:41 <09370> RAX: 00007fc6240211c8   RBX: 00000000000421c9 444: 2016-06-28 09:05:41 <09370> RCX: 00000000000421c9   RDX: 00007fc623fdf000 445: 2016-06-28 09:05:41 <09370> R8:  0000000000002249   R9:  00000000000021da 446: 2016-06-28 09:05:41 <09370> R10: 00007fff5666aef0   R11: 00007fc65b3f8280 447: 2016-06-28 09:05:41 <09370> R12: 00007fc6155018f0   R13: 00007fc623fdf000 448: 2016-06-28 09:05:41 <09370> R14: 00007fc623fdf000   R15: 00007fc6150fc418 449: 2016-06-28 09:05:41 <09370> RSI: 00007fc623f91000   RDI: 00007fc6155018f0 450: 2016-06-28 09:05:41 <09370> RBP: 00000000000421c9   RSP: 00007fff5666ae90 451: 2016-06-28 09:05:41 <09370> RIP: 00007fc6580a3118   EFLAGS: 0000000000010206 452: 2016-06-28 09:05:41 <09370> CS:  0033   FS: 0000   GS: 0000 453: 2016-06-28 09:05:41 <09370> Trap: 000000000000000e   Error: 0000000000000004 454: 2016-06-28 09:05:41 <09370> OldMask: 0000000000000800 455: 2016-06-28 09:05:41 <09370> CR2: 00007fc6240211c8 456: 2016-06-28 09:05:41 <09370> Backtrace: [size="3"]457: 2016-06-28 09:05:41 <09370> [0x7fc6580a3118] => /data/lib/libips.so  liboffset 00091118[/size] [size="3"]458: 2016-06-28 09:05:41 <09370> [0x7fc6580c1239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]459: 2016-06-28 09:05:41 <09370> [0x7fc6580ccbb4] => /data/lib/libips.so  liboffset 000babb4[/size] [size="3"]460: 2016-06-28 09:05:41 <09370> [0x7fc6580ccd51] => /data/lib/libips.so  liboffset 000bad51[/size] [size="3"]461: 2016-06-28 09:05:41 <09370> [0x7fc6580cddaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]462: 2016-06-28 09:05:41 <09370> [0x7fc658073c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]463: 2016-06-28 09:05:41 <09370> [0x7fc65808613e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]464: 2016-06-28 09:05:41 <09370> [0x7fc658052920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]465: 2016-06-28 09:05:41 <09370> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]466: 2016-06-28 09:05:41 <09370> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]467: 2016-06-28 09:05:41 <09370> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]468: 2016-06-28 09:05:41 <09370> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]469: 2016-06-28 09:05:41 <09370> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]470: 2016-06-28 09:05:41 <09370> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]471: 2016-06-28 09:05:41 <09370> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]472: 2016-06-28 09:05:41 <09370> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]473: 2016-06-28 09:05:41 <09370> [0x00443557] => /bin/ipsengine  [/size] [size="3"]474: 2016-06-28 09:05:41 <09370> [0x00441520] => /bin/ipsengine  [/size] [size="3"]475: 2016-06-28 09:05:41 <09370> [0x00443188] => /bin/ipsengine  [/size] [size="3"]476: 2016-06-28 09:05:41 <09370> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]477: 2016-06-28 09:05:41 <09370> [0x7fc65b2ae475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 478: 2016-06-28 09:05:41 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]479: 2016-06-28 09:05:41 <09370> [0x0043a9f1] => /bin/ipsengine  [/size] 480: 2016-06-28 09:40:22 proxyworker - watchdog timeout 481: 2016-06-28 09:40:22 <19180> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 482: 2016-06-28 09:40:22 <19180> application proxyd 483: 2016-06-28 09:40:22 <19180> *** signal 6 (Aborted) received *** 484: 2016-06-28 09:40:22 <19180> Register dump: 485: 2016-06-28 09:40:22 <19180> RAX: 0000000000000000   RBX: 000000000005c982 486: 2016-06-28 09:40:22 <19180> RCX: ffffffffffffffff   RDX: 0000000000000006 487: 2016-06-28 09:40:22 <19180> R8:  0000000000004aec   R9:  0000000000000006 488: 2016-06-28 09:40:22 <19180> R10: 0000000000000008   R11: 0000000000000246 489: 2016-06-28 09:40:22 <19180> R12: 000000000000001a   R13: 0000000000000001 490: 2016-06-28 09:40:22 <19180> R14: 0000000005a7e7d0   R15: 00007fff9d2cea20 491: 2016-06-28 09:40:22 <19180> RSI: 0000000000004aec   RDI: 0000000000004aec 492: 2016-06-28 09:40:22 <19180> RBP: 00007fff9d2ce370   RSP: 00007fff9d2ce228 493: 2016-06-28 09:40:22 <19180> RIP: 00007fd0d78f9c4b   EFLAGS: 0000000000000246 494: 2016-06-28 09:40:22 <19180> CS:  0033   FS: 0000   GS: 0000 495: 2016-06-28 09:40:22 <19180> Trap: 0000000000000000   Error: 0000000000000000 496: 2016-06-28 09:40:22 <19180> OldMask: 0000000002000000 497: 2016-06-28 09:40:22 <19180> CR2: 0000000000000000 498: 2016-06-28 09:40:22 <19180> Backtrace: [size="3"]499: 2016-06-28 09:40:22 <19180> [0x7fd0d78f9c4b] => /fortidev4-x86_64/lib/libc.so.6 [/size] 500: 2016-06-28 09:40:22 (gsignal+0x0000003b) liboffset 00034c4b [size="3"]501: 2016-06-28 09:40:22 <19180> [0x7fd0d78fb108] => /fortidev4-x86_64/lib/libc.so.6 (abort+0x00000148) [/size] 502: 2016-06-28 09:40:22 liboffset 00036108 [size="3"]503: 2016-06-28 09:40:22 <19180> [0x0188bccf] => /bin/proxyd  [/size] [size="3"]504: 2016-06-28 09:40:22 <19180> [0x7fd0d78f9d00] => /fortidev4-x86_64/lib/libc.so.6  liboffset 00034d00[/size] [size="3"]505: 2016-06-28 09:40:22 <19180> [0x019f9a52] => /bin/proxyd  [/size] [size="3"]506: 2016-06-28 09:40:22 <19180> [0x01a15493] => /bin/proxyd  [/size] [size="3"]507: 2016-06-28 09:40:22 <19180> [0x019fa3d1] => /bin/proxyd  [/size] [size="3"]508: 2016-06-28 09:40:22 <19180> [0x00caa80d] => /bin/proxyd  [/size] [size="3"]509: 2016-06-28 09:40:22 <19180> [0x00caaa89] => /bin/proxyd  [/size] [size="3"]510: 2016-06-28 09:40:22 <19180> [0x00ca8a2c] => /bin/proxyd  [/size] [size="3"]511: 2016-06-28 09:40:22 <19180> [0x019fe93e] => /bin/proxyd  [/size] [size="3"]512: 2016-06-28 09:40:22 <19180> [0x00f98d28] => /bin/proxyd  [/size] [size="3"]513: 2016-06-28 09:40:22 <19180> [0x0043d21c] => /bin/proxyd  [/size] [size="3"]514: 2016-06-28 09:40:22 <19180> [0x0043a42f] => /bin/proxyd  [/size] [size="3"]515: 2016-06-28 09:40:22 <19180> [0x7fd0d78e6475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 516: 2016-06-28 09:40:22 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]517: 2016-06-28 09:40:22 <19180> [0x0043a9f1] => /bin/proxyd  [/size] 518: 2016-06-28 09:40:22 <09303> proxyapp=proxyd000 pid=19180 exittype=signal code=6 519: 2016-06-28 09:40:22 <09303> total=7996 free=5354 shmfs_total=6285 shmfs_free=6281 520: 2016-06-28 09:42:14 scanunit=manager pid=9309 str="AV database changed (1); restarting workers" 521: 2016-06-28 09:42:14 <09309> scanunit=manager str="Success loading anti-virus database." 522: 2016-06-28 10:04:51 <26690> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 523: 2016-06-28 10:04:51 <26690> application ipsengine 03.164 524: 2016-06-28 10:04:51 <26690> *** signal 11 (Segmentation fault) received *** 525: 2016-06-28 10:04:51 <26690> Register dump: 526: 2016-06-28 10:04:51 <26690> RAX: 00007fc62403645e   RBX: 000000000005745f 527: 2016-06-28 10:04:51 <26690> RCX: 000000000005745f   RDX: 00007fc623fdf000 528: 2016-06-28 10:04:51 <26690> R8:  00000000000074df   R9:  000000000eb0ef80 529: 2016-06-28 10:04:51 <26690> R10: 0000000000000000   R11: 00007fc65b3f8280 530: 2016-06-28 10:04:51 <26690> R12: 00007fc61058c1f0   R13: 00007fc623fdf000 531: 2016-06-28 10:04:51 <26690> R14: 00007fc623fdf000   R15: 00007fc61505ca18 532: 2016-06-28 10:04:51 <26690> RSI: 00007fc623f91000   RDI: 00007fc61058c1f0 533: 2016-06-28 10:04:51 <26690> RBP: 000000000005745f   RSP: 00007fff5666ae90 534: 2016-06-28 10:04:51 <26690> RIP: 00007fc6580a3118   EFLAGS: 0000000000010206 535: 2016-06-28 10:04:51 <26690> CS:  0033   FS: 0000   GS: 0000 536: 2016-06-28 10:04:51 <26690> Trap: 000000000000000e   Error: 0000000000000004 537: 2016-06-28 10:04:51 <26690> OldMask: 0000000000000800 538: 2016-06-28 10:04:51 <26690> CR2: 00007fc62403645e 539: 2016-06-28 10:04:51 <26690> Backtrace: [size="3"]540: 2016-06-28 10:04:51 <26690> [0x7fc6580a3118] => /data/lib/libips.so  liboffset 00091118[/size] [size="3"]541: 2016-06-28 10:04:51 <26690> [0x7fc6580c1239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]542: 2016-06-28 10:04:51 <26690> [0x7fc6580ccc82] => /data/lib/libips.so  liboffset 000bac82[/size] [size="3"]543: 2016-06-28 10:04:51 <26690> [0x7fc6580ccd51] => /data/lib/libips.so  liboffset 000bad51[/size] [size="3"]544: 2016-06-28 10:04:51 <26690> [0x7fc6580cddaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]545: 2016-06-28 10:04:51 <26690> [0x7fc658073c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]546: 2016-06-28 10:04:51 <26690> [0x7fc65808613e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]547: 2016-06-28 10:04:51 <26690> [0x7fc658052920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]548: 2016-06-28 10:04:51 <26690> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]549: 2016-06-28 10:04:51 <26690> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]550: 2016-06-28 10:04:51 <26690> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]551: 2016-06-28 10:04:51 <26690> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]552: 2016-06-28 10:04:51 <26690> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]553: 2016-06-28 10:04:51 <26690> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]554: 2016-06-28 10:04:51 <26690> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]555: 2016-06-28 10:04:51 <26690> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]556: 2016-06-28 10:04:51 <26690> [0x00443557] => /bin/ipsengine  [/size] [size="3"]557: 2016-06-28 10:04:51 <26690> [0x00441520] => /bin/ipsengine  [/size] [size="3"]558: 2016-06-28 10:04:51 <26690> [0x00443188] => /bin/ipsengine  [/size] [size="3"]559: 2016-06-28 10:04:51 <26690> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]560: 2016-06-28 10:04:51 <26690> [0x7fc65b2ae475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 561: 2016-06-28 10:04:51 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]562: 2016-06-28 10:04:51 <26690> [0x0043a9f1] => /bin/ipsengine  [/size] 563: 2016-06-28 10:31:16 <09371> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 564: 2016-06-28 10:31:16 <09371> application ipsengine 03.164 565: 2016-06-28 10:31:16 <09371> *** signal 11 (Segmentation fault) received *** 566: 2016-06-28 10:31:16 <09371> Register dump: 567: 2016-06-28 10:31:16 <09371> RAX: 0000000000000006   RBX: 00007fc65867106c 568: 2016-06-28 10:31:16 <09371> RCX: 0000000000000006   RDX: 0000000000000007 569: 2016-06-28 10:31:16 <09371> R8:  00007fc65867116c   R9:  00007fc658325c80 570: 2016-06-28 10:31:16 <09371> R10: 00007fff5666af60   R11: 0000000000000000 571: 2016-06-28 10:31:16 <09371> R12: 0000000000000007   R13: 0000000000000006 572: 2016-06-28 10:31:16 <09371> R14: 00007fc615496000   R15: 00007fc623fdf000 573: 2016-06-28 10:31:16 <09371> RSI: 00000000000031b3   RDI: 00007fc623fffffd 574: 2016-06-28 10:31:16 <09371> RBP: 0000000000000006   RSP: 00007fff5666ae98 575: 2016-06-28 10:31:16 <09371> RIP: 00007fc65813a235   EFLAGS: 0000000000010283 576: 2016-06-28 10:31:16 <09371> CS:  0033   FS: 0000   GS: 0000 577: 2016-06-28 10:31:16 <09371> Trap: 000000000000000e   Error: 0000000000000004 578: 2016-06-28 10:31:16 <09371> OldMask: 0000000000000800 579: 2016-06-28 10:31:16 <09371> CR2: 00007fc624000003 580: 2016-06-28 10:31:16 <09371> Backtrace: [size="3"]581: 2016-06-28 10:31:16 <09371> [0x7fc65813a235] => /data/lib/libips.so  liboffset 00128235[/size] [size="3"]582: 2016-06-28 10:31:16 <09371> [0x7fc6580a30a5] => /data/lib/libips.so  liboffset 000910a5[/size] [size="3"]583: 2016-06-28 10:31:16 <09371> [0x7fc6580a9865] => /data/lib/libips.so  liboffset 00097865[/size] [size="3"]584: 2016-06-28 10:31:16 <09371> [0x7fc6580c1239] => /data/lib/libips.so  liboffset 000af239[/size] [size="3"]585: 2016-06-28 10:31:16 <09371> [0x7fc6580cd0a3] => /data/lib/libips.so  liboffset 000bb0a3[/size] [size="3"]586: 2016-06-28 10:31:16 <09371> [0x7fc6580cddaa] => /data/lib/libips.so  liboffset 000bbdaa[/size] [size="3"]587: 2016-06-28 10:31:16 <09371> [0x7fc658073c3e] => /data/lib/libips.so  liboffset 00061c3e[/size] [size="3"]588: 2016-06-28 10:31:16 <09371> [0x7fc65808613e] => /data/lib/libips.so  liboffset 0007413e[/size] [size="3"]589: 2016-06-28 10:31:16 <09371> [0x7fc658052920] => /data/lib/libips.so  liboffset 00040920[/size] [size="3"]590: 2016-06-28 10:31:16 <09371> [0x00cc1eb6] => /bin/ipsengine  [/size] [size="3"]591: 2016-06-28 10:31:16 <09371> [0x00cc2397] => /bin/ipsengine  [/size] [size="3"]592: 2016-06-28 10:31:16 <09371> [0x00cc2650] => /bin/ipsengine  [/size] [size="3"]593: 2016-06-28 10:31:16 <09371> [0x00cc4845] => /bin/ipsengine  [/size] [size="3"]594: 2016-06-28 10:31:16 <09371> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]595: 2016-06-28 10:31:16 <09371> [0x00cb119a] => /bin/ipsengine  [/size] [size="3"]596: 2016-06-28 10:31:16 <09371> [0x00cb1e48] => /bin/ipsengine  [/size] [size="3"]597: 2016-06-28 10:31:16 <09371> [0x0043d3f0] => /bin/ipsengine  [/size] [size="3"]598: 2016-06-28 10:31:16 <09371> [0x00443557] => /bin/ipsengine  [/size] [size="3"]599: 2016-06-28 10:31:16 <09371> [0x00441520] => /bin/ipsengine  [/size] [size="3"]600: 2016-06-28 10:31:16 <09371> [0x00443188] => /bin/ipsengine  [/size] [size="3"]601: 2016-06-28 10:31:16 <09371> [0x0043a917] => /bin/ipsengine  [/size] [size="3"]602: 2016-06-28 10:31:16 <09371> [0x7fc65b2ae475] => /fortidev4-x86_64/lib/libc.so.6 [/size] 603: 2016-06-28 10:31:16 (__libc_start_main+0x000000f5) liboffset 00021475 [size="3"]604: 2016-06-28 10:31:16 <09371> [0x0043a9f1] => /bin/ipsengine  [/size] 605: 2016-06-28 10:43:41 the killed daemon is /bin/pyfcgid: status=0x0 606: 2016-06-28 11:00:57 proxyworker - watchdog timeout

[seadave]

Just got off the phone with the TAC engineer. He suggested the issue is I have my Certificate and Deep Scanning settings configured to "Scan All Ports" instead of the traditional SSL/TLS, S/SMTP, etc ports.  Makes sense, but I think we all know that TLS is flowing over all sorts of ports these days.  I will disable the "all ports" setting to see if that resolves the issue.  I think this might be a default setting?  Can't remember if I modified.

Security Profiles...SSL/SSH Inspection...Protocol Port Mapping...Inspect All Ports (Set to disable), then enable specific ports.  It does appear you can enter multiple ports for SSL for example (443, 8443).

[seadave]

Looks like I should change SSH Inspection options to "22 only" also which is further down that config page.

TAC said leaving these to "all ports" overtasks the proxy worker daemon.  Still doesn't explain why it worked until 6/12, but perhaps this setting in combination of IPS engine update is the cause.

[kallbrandt]

Hmmm, the crash log you attached looks a bit worrying to me. That doesn't look like the normal restarting of processes. The watchdog timeouts and the memory references gives me the impression that FortiOS can't restart the processes as it should be able to (really - just me guessing). This corresponds to several issues - Faulty RAM for example, or something simpler.

I'm suspecting corrupted databases. Just a hunch, but easy enough to fix if that's the case. A manual update overwrites the old databases. The autoupdate that runs automatically (if set) only update the delta. So, try a manual update of all the av/ips-stuff, either in the gui or in the cli:

execute update-av

execute update-ips

Or just update everything with one command:

execute update-now

Check the updates after a while with "diagnose autoupdate versions", post the output here.

You should have databases created a while back (it follows the firmware), and update that are from today.

Another way to probably fix this is to backup your config and do a clean install of the firmware you are running, and reload your config.

But try the database stuff first.

[seadave]

It has only been a few hours, but my crashlog is not showing the previous crash reports as it did when I had the "scan all ports" function enabled in the SSL/SSH Scanning option.

# dia debu crashlog read 1: 2016-07-06 11:36:54 the killed daemon is /bin/pyfcgid: status=0x0 2: 2016-07-06 11:53:10 the killed daemon is /bin/pyfcgid: status=0x0 3: 2016-07-06 12:01:13 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 4: 2016-07-06 12:01:13 <09306> scanunit=manager str="Success loading anti-virus database." 5: 2016-07-06 12:37:35 the killed daemon is /bin/pyfcgid: status=0x0 6: 2016-07-06 12:48:20 the killed daemon is /bin/pyfcgid: status=0x100

TAC was pretty confident the Scan All Ports option was the cause of this.  I'll give it 12 hours and run another crashlog read.  The real test will be if the unit enters conserve mode or not.  I've disabled the scheduled reboot.  I have a remote power reset, so will bounce the firewall if it hangs.