When is 5.4.1 going to drop?

ITGuy11 · ‎03-28-2016

Is there an ETA as to when 5.4.1 is going to drop? I have a brand new 300D that I am waiting to put into production as soon as 5.4.1 is ready.

FGTuser · ‎04-04-2016

by end of next week (April 15)

View solution in original post

kallbrandt · ‎06-30-2016

That amount of clashes is nothing to worry about I'd say. On the LB-vdom I mentioned earlier the log shows 6-digit amounts of clashes. The clash counter is reset at reboot btw, and is not related to the current amount of sessions. It is just an ongoing counter.

To my knowledge, all restarts of applications with restart option 11 (segmentation fault) in FortiOS is seen as a crash. It doesn't have to mean anything bad per se. The OS recycles processes all the time using option 15 (graceful restart). When that doesn't work, it moves on to try to restart with option 11 wich will generate a log entry in the syslog. The recycle process continues all the time, buffers needs to be cleared etc etc. However, a constant restarting of the same application can also mean various problems - Memory leaks, buffer overflows etc.

I checked your log, but I can't see anything else then the PID and some weird ASCII-signs as application name. It does look kinda odd.

Check your logs and keep track of if the application crash log entries correlates with odd behaviour in the firewall, we're talking sudden reboots, functions and features stopping/not working.

What does "diagnose debug crashlog read" say?

Also, do a "diagnose sys top", a few times during the day. Do you have processes in Z or D state?

Richie

NSE7

View solution in original post

kallbrandt · ‎07-06-2016

That log looks perfectly ok - Just hope it stays that way!

But this is kind of alarming news, since this is a serious bug. However, there are several fixes in 5.4.1, for example BUG 372706, wich is about how FortiOS handles the abort/close calls for the proxyworker...

Did the TAC suggest an upgrade?

It's good to know, since your issue might not be fixed in 5.4.1...

EDIT: Scrolled back and saw your earlier post about TAC suggesting that...

Richie

NSE7

seadave · ‎07-07-2016

Looking better this AM.

1: 2016-07-06 11:36:54 the killed daemon is /bin/pyfcgid: status=0x0 2: 2016-07-06 11:53:10 the killed daemon is /bin/pyfcgid: status=0x0 3: 2016-07-06 12:01:13 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 4: 2016-07-06 12:01:13 <09306> scanunit=manager str="Success loading anti-virus database." 5: 2016-07-06 12:37:35 the killed daemon is /bin/pyfcgid: status=0x0 6: 2016-07-06 12:48:20 the killed daemon is /bin/pyfcgid: status=0x100 7: 2016-07-06 13:18:19 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 8: 2016-07-06 13:18:19 <09306> scanunit=manager str="Success loading anti-virus database." 9: 2016-07-06 13:30:21 <09358> firmware FortiGate-500D v5.4.0,build1011b1011,151221 (GA) (Release) 10: 2016-07-06 13:30:21 <09358> application wad 11: 2016-07-06 13:30:21 <09358> *** signal 11 (Segmentation fault) received *** 12: 2016-07-06 13:30:21 <09358> Register dump: 13: 2016-07-06 13:30:21 <09358> RAX: 0000000000000003 RBX: 00007f570039c7d0 14: 2016-07-06 13:30:21 <09358> RCX: 00000000000000f8 RDX: 00007fffb9a90c10 15: 2016-07-06 13:30:21 <09358> R8: 0000000001359ea0 R9: 00000000000000bd 16: 2016-07-06 13:30:21 <09358> R10: 0000000000000003 R11: 00007f5714865640 17: 2016-07-06 13:30:21 <09358> R12: 0000000000000000 R13: 00007fffb9a90c10 18: 2016-07-06 13:30:21 <09358> R14: 00007f570039c810 R15: 0000000000000003 19: 2016-07-06 13:30:21 <09358> RSI: 00007f5700e49fb0 RDI: 00007fffb9a90c10 20: 2016-07-06 13:30:21 <09358> RBP: 00007fffb9a90bb0 RSP: 00007fffb9a90930 21: 2016-07-06 13:30:21 <09358> RIP: 00000000013865d0 EFLAGS: 0000000000010246 22: 2016-07-06 13:30:21 <09358> CS: 0033 FS: 0000 GS: 0000 23: 2016-07-06 13:30:21 <09358> Trap: 000000000000000e Error: 0000000000000004 24: 2016-07-06 13:30:21 <09358> OldMask: 0000000000000000 25: 2016-07-06 13:30:21 <09358> CR2: 0000000000000098 26: 2016-07-06 13:30:21 <09358> Backtrace: 27: 2016-07-06 13:30:21 <09358> [0x013865d0] => /bin/wad 28: 2016-07-06 13:30:21 <09358> [0x019168af] => /bin/wad 29: 2016-07-06 13:30:21 <09358> [0x019071c1] => /bin/wad 30: 2016-07-06 13:30:21 <09358> [0x0138f48d] => /bin/wad 31: 2016-07-06 13:30:21 <09358> [0x0138fd06] => /bin/wad 32: 2016-07-06 13:30:21 <09358> [0x0139d9c8] => /bin/wad 33: 2016-07-06 13:30:21 <09358> [0x01373b92] => /bin/wad 34: 2016-07-06 13:30:21 <09358> [0x013aee1c] => /bin/wad 35: 2016-07-06 13:30:21 <09358> [0x0043d21c] => /bin/wad 36: 2016-07-06 13:30:21 <09358> [0x0043a42f] => /bin/wad 37: 2016-07-06 13:30:21 <09358> [0x7f571471c475] => /fortidev4-x86_64/lib/libc.so.6 38: 2016-07-06 13:30:21 (__libc_start_main+0x000000f5) liboffset 00021475 39: 2016-07-06 13:30:21 <09358> [0x0043a9f1] => /bin/wad 40: 2016-07-06 13:30:21 <09358> proess=wad type=3 pid=9358 total=7996 free=4955 mmu=173221657 41: 2016-07-06 13:30:21 mu=101865407 m=78131451 f=78032774 r=0 42: 2016-07-06 13:30:21 <09358> current tcp(0x7f5705c2d910) vf=0 session-id=901622 app_type=1 43: 2016-07-06 13:30:21 dyn_type=0 state=2 cur_bank=0x33d30e0 cur_tl=0x37c8480 cur_tm=0x7f5711becd00 44: 2016-07-06 13:32:50 the killed daemon is /bin/pyfcgid: status=0x100 45: 2016-07-06 14:05:24 the killed daemon is /bin/pyfcgid: status=0x100 46: 2016-07-06 14:15:24 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 47: 2016-07-06 14:15:25 <09306> scanunit=manager str="Success loading anti-virus database." 48: 2016-07-06 14:16:56 the killed daemon is /bin/pyfcgid: status=0x100 49: 2016-07-06 15:01:17 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 50: 2016-07-06 15:01:17 <09306> scanunit=manager str="Success loading anti-virus database." 51: 2016-07-06 18:01:16 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 52: 2016-07-06 18:01:17 <09306> scanunit=manager str="Success loading anti-virus database." 53: 2016-07-06 21:01:18 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 54: 2016-07-06 21:01:19 <09306> scanunit=manager str="Success loading anti-virus database." 55: 2016-07-06 22:57:56 scanunit=manager pid=9306 str="AV database changed (1); restarting workers" 56: 2016-07-06 22:57:56 <09306> scanunit=manager str="Success loading anti-virus database." 57: 2016-07-07 03:46:11 <09308> scanunit=manager str="Success loading anti-virus database." 58: 2016-07-07 05:18:33 scanunit=manager pid=9308 str="AV database changed (1); restarting workers" 59: 2016-07-07 05:18:34 <09308> scanunit=manager str="Success loading anti-virus database." 60: 2016-07-07 06:38:12 scanunit=manager pid=9308 str="AV database changed (1); restarting workers" 61: 2016-07-07 06:38:13 <09308> scanunit=manager str="Success loading anti-virus database."

Previously, I'd have seen thousands of app crashes after 12 hours. Still wondering why there was a segmentation fault, but thinking that seeing these occasionally is not out of the norm.

seadave · ‎07-07-2016

I like a lot of the newer features in 5.4.1, but the aforementioned SSL Scanning bug in this and other posts is giving me pause. In fact the TAC engineer said he would "never suggest" upgrading to a firmware release only two weeks old.

kallbrandt · ‎07-07-2016

Yeah, looks a lot cleaner then before.

Segmentation fault is just the name of the (restart)signal 11, so the cause can be everything from FortiOS restarting the service to an actual segmentation fault. All restarts are handled and logged as crashes. As long as the crash log doesn't look like it did before you're probably alright.

Well, tons of bug fixes in 5.4.1, and I suspect some of the new features are there since they obviously had to rewrite a lot of code anyway. But... You know what you have, but not what you'll get...

I will try 5.4.1 out on a A-A HA-pair, 2*600D in a few weeks, will post the result. If it's not stable or too much stuff isn't working as expected I will of course roll back. Those summer weeks in late July is a good time to create a mess since nobody is there to complain (24/7 activity otherwise...).

:)

Richie

NSE7

When is 5.4.1 going to drop?

Nominate a Forum Post for Knowledge Article Creation