HA issue with interfaces showing 'mondev down'

[bigkeoni64]

Anyone seen an issue with the primary system showing 'mondev down' in the "get system ha status" - but - the interfaces are actually up?

Looks like it flipped the priority on HA2 to make it the new priority, and higher than HA1.

I'm on 6.4.9 and have not seen this before.

Any feedback is welcomed.

AAA-FGT201F-HA1 $ get system ha status
HA Health Status:
WARNING: FG201FT920900152 has mondev down;
Model: FortiGate-201F
Mode: HA A-P
Group: 0
Debug: 0
Cluster Uptime: 536 days 9:57:26
Cluster state change time: 2022-08-20 01:42:16
Primary selected using:
<2022/08/20 01:42:16> FG201FT920900345 is selected as the primary because it has the least value 0 of link-failure + pingsvr-failure.
<2022/08/20 01:40:01> FG201FT920900345 is selected as the primary because it's the only member in the cluster.
<2022/08/20 01:39:55> FG201FT920900345 is selected as the primary because the peer member FG201FT920900152 has SET_AS_SECONDARY flag set.
<2022/08/20 01:29:11> FG201FT920900152 is selected as the primary because it has the least value 0 of link-failure + pingsvr-failure.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
FG201FT920900345(updated 1 seconds ago): in-sync
FG201FT920900152(updated 4 seconds ago): in-sync
System Usage stats:
FG201FT920900345(updated 1 seconds ago):
sessions=15987, average-cpu-user/nice/system/idle=0%/0%/0%/99%, memory=51%
FG201FT920900152(updated 4 seconds ago):
sessions=4147, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=41%
HBDEV stats:
FG201FT920900345(updated 1 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=14919245663/44810738/0/0, tx=38071942386/47265131/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2592513160/4900743/0/0, tx=2563159207/4900915/0/0
FG201FT920900152(updated 4 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=38062597741/47240406/0/0, tx=14904793039/44797196/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2561364781/4897447/0/0, tx=2590724071/4897399/0/0
MONDEV stats:
FG201FT920900345(updated 1 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=2049223312830/3255875202/0/0, tx=4697197062932/5048560250/0/0
LAN_AGG: aggregate/00, up, rx-bytes/packets/dropped/errors=4009558459998/3680181906/0/0, tx=1996165561501/2540571701/0/0
FG201FT920900152(updated 4 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=480914958/7845384/0/0, tx=0/0/0/0
LAN_AGG: aggregate/00, down, rx-bytes/packets/dropped/errors=734559086/8467248/0/0, tx=36564788/261171/0/0
Primary : AAA-FGT201F-HA1 , FG201FT920900345, HA cluster index = 0
Secondary : AAA-FGT201F-HA2 , FG201FT920900152, HA cluster index = 1
number of vcluster: 1
vcluster 1: work 169.254.0.1
Primary: FG201FT920900345, HA operating index = 0
Secondary: FG201FT920900152, HA operating index = 1

AAA-FGT201F-HA1 $
AAA-FGT201F-HA1 $ execute ha manage 1 private
private@169.254.0.2's password:

AAA-FGT201F-HA2 $ get system ha status
HA Health Status:
WARNING: FG201FT920900152 has mondev down;
Model: FortiGate-201F
Mode: HA A-P
Group: 0
Debug: 0
Cluster Uptime: 536 days 10:2:3
Cluster state change time: 2022-08-20 01:42:15
Primary selected using:
<2022/08/20 01:42:15> FG201FT920900345 is selected as the primary because it has the least value 0 of link-failure + pingsvr-failure.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
FG201FT920900152(updated 1 seconds ago): in-sync
FG201FT920900345(updated 3 seconds ago): in-sync
System Usage stats:
FG201FT920900152(updated 1 seconds ago):
sessions=3993, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=41%
FG201FT920900345(updated 3 seconds ago):
sessions=14212, average-cpu-user/nice/system/idle=0%/0%/0%/99%, memory=51%
HBDEV stats:
FG201FT920900152(updated 1 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=38075015599/47252068/0/0, tx=14906927019/44803506/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2562083906/4898822/0/0, tx=2591451446/4898774/0/0
FG201FT920900345(updated 3 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=14921329268/44816836/0/0, tx=38084151442/47276602/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2593227310/4902093/0/0, tx=2563865257/4902265/0/0
MONDEV stats:
FG201FT920900152(updated 1 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=481113636/7848650/0/0, tx=0/0/0/0
LAN_AGG: aggregate/00, down, rx-bytes/packets/dropped/errors=734767142/8469632/0/0, tx=36574868/261243/0/0
FG201FT920900345(updated 3 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=2049999068931/3257458758/0/0, tx=4697880616664/5050218272/0/0
LAN_AGG: aggregate/00, up, rx-bytes/packets/dropped/errors=4009778553216/3680728595/0/0, tx=1996778544419/2541370555/0/0
Secondary : AAA-FGT201F-HA2 , FG201FT920900152, HA cluster index = 1
Primary : AAA-FGT201F-HA1 , FG201FT920900345, HA cluster index = 0
number of vcluster: 1
vcluster 1: standby 169.254.0.1
Secondary: FG201FT920900152, HA operating index = 1
Primary: FG201FT920900345, HA operating index = 0

AAA-FGT201F-HA2 $ private@169.254.0.2's password:
AAA-FGT201F-HA2 $ get system ha status
HA Health Status:
WARNING: FG201FT920900152 has mondev down;
Model: FortiGate-201F
Mode: HA A-P
Group: 0
Debug: 0
Cluster Uptime: 536 days 10:2:3
Cluster state change time: 2022-08-20 01:42:15
Primary selected using:
<2022/08/20 01:42:15> FG201FT920900345 is selected as the primary because it has the least value 0 of link-failure + pingsvr-failure.
ses_pickup: enable, ses_pickup_delay=disable
override: disable
Configuration Status:
FG201FT920900152(updated 1 seconds ago): in-sync
FG201FT920900345(updated 3 seconds ago): in-sync
System Usage stats:
FG201FT920900152(updated 1 seconds ago):
sessions=3993, average-cpu-user/nice/system/idle=0%/0%/0%/100%, memory=41%
FG201FT920900345(updated 3 seconds ago):
sessions=14212, average-cpu-user/nice/system/idle=0%/0%/0%/99%, memory=51%
HBDEV stats:
FG201FT920900152(updated 1 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=38075015599/47252068/0/0, tx=14906927019/44803506/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2562083906/4898822/0/0, tx=2591451446/4898774/0/0
FG201FT920900345(updated 3 seconds ago):
port1: physical/1000auto, up, rx-bytes/packets/dropped/errors=14921329268/44816836/0/0, tx=38084151442/47276602/0/0
port2: physical/1000auto, up, rx-bytes/packets/dropped/errors=2593227310/4902093/0/0, tx=2563865257/4902265/0/0
MONDEV stats:
FG201FT920900152(updated 1 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=481113636/7848650/0/0, tx=0/0/0/0
LAN_AGG: aggregate/00, down, rx-bytes/packets/dropped/errors=734767142/8469632/0/0, tx=36574868/261243/0/0
FG201FT920900345(updated 3 seconds ago):
port9: physical/1000auto, up, rx-bytes/packets/dropped/errors=2049999068931/3257458758/0/0, tx=4697880616664/5050218272/0/0
LAN_AGG: aggregate/00, up, rx-bytes/packets/dropped/errors=4009778553216/3680728595/0/0, tx=1996778544419/2541370555/0/0
Secondary : AAA-FGT201F-HA2 , FG201FT920900152, HA cluster index = 1
Primary : AAA-FGT201F-HA1 , FG201FT920900345, HA cluster index = 0
number of vcluster: 1
vcluster 1: standby 169.254.0.1
Secondary: FG201FT920900152, HA operating index = 1
Primary: FG201FT920900345, HA operating index = 0

AAA-FGT201F-HA2 $

Thank you...

[vsahu]

Hello,

Can you share the output of the below command from both devices.

get system ha status

Also, check if there's any crash for the HA process

diag debug crashlog read

Regards,
Vishal

[bigkeoni64]

Is there an issue with FNT community at the moment - it seems as though my paste will not stick when I hit reply

[bigkeoni64]

Sent the output of 'get system ha status' as a message

[bigkeoni64]

415: 2022-05-20 14:40:12 <00369> CR2: 0000000000000000
416: 2022-05-20 14:40:12 <00369> stack: 0x7fffad9c7990 - 0x7fffad9c8c30
417: 2022-05-20 14:40:12 <00369> Backtrace:
418: 2022-05-20 14:40:12 <00369> [0x7fe26ed36630] => /usr/lib/x86_64-linux-gnu/libc.so.6
419: 2022-05-20 14:40:12 (gsignal+0x00000110) liboffset 00034630
420: 2022-05-20 14:40:12 <00369> [0x7fe26ed37c87] => /usr/lib/x86_64-linux-gnu/libc.so.6
421: 2022-05-20 14:40:12 (abort+0x000001c7) liboffset 00035c87
422: 2022-05-20 14:40:12 <00369> [0x7fe26ed7f5da] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
423: 2022-05-20 14:40:12 0007d5da
424: 2022-05-20 14:40:12 <00369> [0x7fe26ed80ea9] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
425: 2022-05-20 14:40:12 0007eea9
426: 2022-05-20 14:40:12 <00369> [0x00e07048] => /bin/ipsengine
427: 2022-05-20 14:40:12 <00369> [0x7fe26835ef93] => /data/lib/libips.so
428: 2022-05-20 14:40:12 <00369> [0x7fe26835f118] => /data/lib/libips.so
429: 2022-05-20 14:40:12 <00369> [0x7fe268351fae] => /data/lib/libips.so
430: 2022-05-20 14:40:12 <00369> [0x00e01943] => /bin/ipsengine
431: 2022-05-20 14:40:12 <00369> [0x004362e0] => /bin/ipsengine
432: 2022-05-20 14:40:12 <00369> [0x00e030d7] => /bin/ipsengine
433: 2022-05-20 14:40:12 <00369> [0x00e02545] => /bin/ipsengine
434: 2022-05-20 14:40:12 <00369> [0x004362e0] => /bin/ipsengine
435: 2022-05-20 14:40:12 <00369> [0x0043d2af] => /bin/ipsengine
436: 2022-05-20 14:40:12 <00369> [0x00439f38] => /bin/ipsengine
437: 2022-05-20 14:40:12 <00369> [0x0043a44e] => /bin/ipsengine
438: 2022-05-20 14:40:12 <00369> [0x0043cd29] => /bin/ipsengine
439: 2022-05-20 14:40:12 <00369> [0x7fe26ed22eaa] => /usr/lib/x86_64-linux-gnu/libc.so.6
440: 2022-05-20 14:40:12 (__libc_start_main+0x000000ea) liboffset 00020eaa
441: 2022-05-20 14:40:12 <00369> [0x00432cea] => /bin/ipsengine
442: 2022-05-20 14:40:17 ipsengine 05.002.245 previously crashed 4 times. The last crash was at
443: 2022-05-20 14:40:17 2022-05-20 14:40:16.
444: 2022-05-20 14:40:17 <03804> firmware FortiGate-201F v6.2.9,build7131b1234,210601 (GA) (Release)
445: 2022-05-20 14:40:17 <03804> application ipsengine 05.002.245
446: 2022-05-20 14:40:17 <03804> *** signal 6 (Aborted) received ***
447: 2022-05-20 14:40:17 <03804> Register dump:
448: 2022-05-20 14:40:17 <03804> RAX: 0000000000000000 RBX: 0000000000000006
449: 2022-05-20 14:40:17 <03804> RCX: ffffffffffffffff RDX: 0000000000000000
450: 2022-05-20 14:40:17 <03804> R08: 0000000000000000 R09: 00007fffad9a9980
451: 2022-05-20 14:40:17 <03804> R10: 0000000000000008 R11: 0000000000000246
452: 2022-05-20 14:40:17 <03804> R12: 00007fe26ee7b990 R13: 0000000000000001
453: 2022-05-20 14:40:17 <03804> R14: 000000001b35a790 R15: 6770736f3d715df8
454: 2022-05-20 14:40:17 <03804> RSI: 00007fffad9a9980 RDI: 0000000000000002
455: 2022-05-20 14:40:17 <03804> RBP: 00007fe26f0adc20 RSP: 00007fffad9a9980
456: 2022-05-20 14:40:17 <03804> RIP: 00007fe26ed36630 EFLAGS: 0000000000000246
457: 2022-05-20 14:40:17 <03804> CS: 0033 FS: 0000 GS: 0000
458: 2022-05-20 14:40:17 <03804> Trap: 0000000000000000 Error: 0000000000000000
459: 2022-05-20 14:40:17 <03804> OldMask: 0000000000004206
460: 2022-05-20 14:40:17 <03804> CR2: 0000000000000000
461: 2022-05-20 14:40:17 <03804> stack: 0x7fffad9a9980 - 0x7fffad9c8c30
462: 2022-05-20 14:40:17 <03804> Backtrace:
463: 2022-05-20 14:40:17 <03804> [0x7fe26ed36630] => /usr/lib/x86_64-linux-gnu/libc.so.6
464: 2022-05-20 14:40:17 (gsignal+0x00000110) liboffset 00034630
465: 2022-05-20 14:40:17 <03804> [0x7fe26ed37c87] => /usr/lib/x86_64-linux-gnu/libc.so.6
466: 2022-05-20 14:40:17 (abort+0x000001c7) liboffset 00035c87
467: 2022-05-20 14:40:17 <03804> [0x7fe26ed7f5da] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
468: 2022-05-20 14:40:17 0007d5da
469: 2022-05-20 14:40:17 <03804> [0x7fe26ed80ea9] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
470: 2022-05-20 14:40:17 0007eea9
471: 2022-05-20 14:40:17 <03804> [0x0228e93a] => /bin/ipsengine
472: 2022-05-20 14:40:17 <03804> [0x0228eba6] => /bin/ipsengine
473: 2022-05-20 14:40:17 <03804> [0x0228f094] => /bin/ipsengine
474: 2022-05-20 14:40:17 <03804> [0x0229184e] => /bin/ipsengine
475: 2022-05-20 14:40:17 <03804> [0x00df42b8] => /bin/ipsengine
476: 2022-05-20 14:40:17 <03804> [0x00de8c6e] => /bin/ipsengine
477: 2022-05-20 14:40:17 <03804> [0x7fe2684042c2] => /data/lib/libips.so
478: 2022-05-20 14:40:17 <03804> [0x7fe268404c4c] => /data/lib/libips.so
479: 2022-05-20 14:40:17 <03804> [0x7fe267f83cd6] => /data/lib/libips.so
480: 2022-05-20 14:40:17 <03804> [0x7fe267f84d94] => /data/lib/libips.so
481: 2022-05-20 14:40:17 <03804> [0x7fe267f85aac] => /data/lib/libips.so
482: 2022-05-20 14:40:17 <03804> [0x7fe267fef8f4] => /data/lib/libips.so
483: 2022-05-20 14:40:17 <03804> [0x7fe26840736b] => /data/lib/libips.so
484: 2022-05-20 14:40:17 <03804> [0x7fe267f6489d] => /data/lib/libips.so
485: 2022-05-20 14:40:17 <03804> [0x7fe267f274a4] => /data/lib/libips.so
486: 2022-05-20 14:40:17 <03804> [0x00e04885] => /bin/ipsengine
487: 2022-05-20 14:40:17 <03804> [0x00dfd21f] => /bin/ipsengine
488: 2022-05-20 14:40:17 <03804> [0x00de5fe9] => /bin/ipsengine
489: 2022-05-20 14:40:17 <03804> [0x7fe26835c1b9] => /data/lib/libips.so
490: 2022-05-20 14:40:17 <03804> [0x7fe268352111] => /data/lib/libips.so
491: 2022-05-20 14:40:17 <03804> [0x00e01943] => /bin/ipsengine
492: 2022-05-20 14:40:17 <03804> [0x004362e0] => /bin/ipsengine
493: 2022-05-20 14:40:17 <03804> [0x00e030d7] => /bin/ipsengine
494: 2022-05-20 14:40:17 <03804> [0x00e02545] => /bin/ipsengine
495: 2022-05-20 14:40:17 <03804> [0x004362e0] => /bin/ipsengine
496: 2022-05-20 14:40:17 <03804> [0x0043d2af] => /bin/ipsengine
497: 2022-05-20 14:40:17 <03804> [0x00439f38] => /bin/ipsengine
498: 2022-05-20 14:40:17 <03804> [0x0043a44e] => /bin/ipsengine
499: 2022-05-20 14:40:17 <03804> [0x0043cd29] => /bin/ipsengine
500: 2022-05-20 14:40:17 <03804> [0x7fe26ed22eaa] => /usr/lib/x86_64-linux-gnu/libc.so.6
501: 2022-05-20 14:40:17 (__libc_start_main+0x000000ea) liboffset 00020eaa
502: 2022-05-20 14:40:17 <03804> [0x00432cea] => /bin/ipsengine
503: 2022-05-20 14:40:29 ipsengine 05.002.245 previously crashed 6 times. The last crash was at
504: 2022-05-20 14:40:29 2022-05-20 14:40:22.
505: 2022-05-20 14:40:29 <00371> firmware FortiGate-201F v6.2.9,build7131b1234,210601 (GA) (Release)
506: 2022-05-20 14:40:29 <00371> application ipsengine 05.002.245
507: 2022-05-20 14:40:29 <00371> *** signal 6 (Aborted) received ***
508: 2022-05-20 14:40:29 <00371> Register dump:
509: 2022-05-20 14:40:29 <00371> RAX: 0000000000000000 RBX: 0000000000000006
510: 2022-05-20 14:40:29 <00371> RCX: ffffffffffffffff RDX: 0000000000000000
511: 2022-05-20 14:40:29 <00371> R08: 0000000000000000 R09: 00007fffad9c7990
512: 2022-05-20 14:40:29 <00371> R10: 0000000000000008 R11: 0000000000000246
513: 2022-05-20 14:40:29 <00371> R12: 00007fe26ee7b990 R13: 0000000000000001
514: 2022-05-20 14:40:29 <00371> R14: 000000001b300f30 R15: 3632353225443320
515: 2022-05-20 14:40:29 <00371> RSI: 00007fffad9c7990 RDI: 0000000000000002
516: 2022-05-20 14:40:29 <00371> RBP: 00007fe26f0adc20 RSP: 00007fffad9c7990
517: 2022-05-20 14:40:29 <00371> RIP: 00007fe26ed36630 EFLAGS: 0000000000000246
518: 2022-05-20 14:40:29 <00371> CS: 0033 FS: 0000 GS: 0000
519: 2022-05-20 14:40:29 <00371> Trap: 0000000000000000 Error: 0000000000000000
520: 2022-05-20 14:40:29 <00371> OldMask: 0000000000004206
521: 2022-05-20 14:40:29 <00371> CR2: 0000000000000000
522: 2022-05-20 14:40:29 <00371> stack: 0x7fffad9c7990 - 0x7fffad9c8c30
523: 2022-05-20 14:40:29 <00371> Backtrace:
524: 2022-05-20 14:40:29 <00371> [0x7fe26ed36630] => /usr/lib/x86_64-linux-gnu/libc.so.6
525: 2022-05-20 14:40:29 (gsignal+0x00000110) liboffset 00034630
526: 2022-05-20 14:40:29 <00371> [0x7fe26ed37c87] => /usr/lib/x86_64-linux-gnu/libc.so.6
527: 2022-05-20 14:40:29 (abort+0x000001c7) liboffset 00035c87
528: 2022-05-20 14:40:29 <00371> [0x7fe26ed7f5da] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
529: 2022-05-20 14:40:29 0007d5da
530: 2022-05-20 14:40:29 <00371> [0x7fe26ed80ea9] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
531: 2022-05-20 14:40:29 0007eea9
532: 2022-05-20 14:40:29 <00371> [0x00e07048] => /bin/ipsengine
533: 2022-05-20 14:40:29 <00371> [0x7fe26835ef93] => /data/lib/libips.so
534: 2022-05-20 14:40:29 <00371> [0x7fe26835f118] => /data/lib/libips.so
535: 2022-05-20 14:40:29 <00371> [0x7fe268351fae] => /data/lib/libips.so
536: 2022-05-20 14:40:29 <00371> [0x00e01943] => /bin/ipsengine
537: 2022-05-20 14:40:29 <00371> [0x004362e0] => /bin/ipsengine
538: 2022-05-20 14:40:29 <00371> [0x00e030d7] => /bin/ipsengine
539: 2022-05-20 14:40:29 <00371> [0x00e02545] => /bin/ipsengine
540: 2022-05-20 14:40:29 <00371> [0x004362e0] => /bin/ipsengine
541: 2022-05-20 14:40:29 <00371> [0x0043d2af] => /bin/ipsengine
542: 2022-05-20 14:40:29 <00371> [0x00439f38] => /bin/ipsengine
543: 2022-05-20 14:40:29 <00371> [0x0043a44e] => /bin/ipsengine
544: 2022-05-20 14:40:29 <00371> [0x0043cd29] => /bin/ipsengine
545: 2022-05-20 14:40:29 <00371> [0x7fe26ed22eaa] => /usr/lib/x86_64-linux-gnu/libc.so.6
546: 2022-05-20 14:40:29 (__libc_start_main+0x000000ea) liboffset 00020eaa
547: 2022-05-20 14:40:29 <00371> [0x00432cea] => /bin/ipsengine
548: 2022-05-20 14:53:11 ipsengine 05.002.245 previously crashed 7 times. The last crash was at
549: 2022-05-20 14:53:11 2022-05-20 14:40:29.
550: 2022-05-20 14:53:11 <03810> firmware FortiGate-201F v6.2.9,build7131b1234,210601 (GA) (Release)
551: 2022-05-20 14:53:11 <03810> application ipsengine 05.002.245
552: 2022-05-20 14:53:11 <03810> *** signal 6 (Aborted) received ***
553: 2022-05-20 14:53:11 <03810> Register dump:
554: 2022-05-20 14:53:11 <03810> RAX: 0000000000000000 RBX: 0000000000000006
555: 2022-05-20 14:53:11 <03810> RCX: ffffffffffffffff RDX: 0000000000000000
556: 2022-05-20 14:53:11 <03810> R08: 0000000000000000 R09: 00007fffad9c7990
557: 2022-05-20 14:53:11 <03810> R10: 0000000000000008 R11: 0000000000000246
558: 2022-05-20 14:53:11 <03810> R12: 00007fe26ee7b990 R13: 0000000000000001
559: 2022-05-20 14:53:11 <03810> R14: 000000001b3a77d0 R15: 6553656c79706d60
560: 2022-05-20 14:53:11 <03810> RSI: 00007fffad9c7990 RDI: 0000000000000002
561: 2022-05-20 14:53:11 <03810> RBP: 00007fe26f0adc20 RSP: 00007fffad9c7990
562: 2022-05-20 14:53:11 <03810> RIP: 00007fe26ed36630 EFLAGS: 0000000000000246
563: 2022-05-20 14:53:11 <03810> CS: 0033 FS: 0000 GS: 0000
564: 2022-05-20 14:53:11 <03810> Trap: 0000000000000000 Error: 0000000000000000
565: 2022-05-20 14:53:11 <03810> OldMask: 0000000000004206
566: 2022-05-20 14:53:11 <03810> CR2: 0000000000000000
567: 2022-05-20 14:53:11 <03810> stack: 0x7fffad9c7990 - 0x7fffad9c8c30
568: 2022-05-20 14:53:11 <03810> Backtrace:
569: 2022-05-20 14:53:11 <03810> [0x7fe26ed36630] => /usr/lib/x86_64-linux-gnu/libc.so.6
570: 2022-05-20 14:53:11 (gsignal+0x00000110) liboffset 00034630
571: 2022-05-20 14:53:11 <03810> [0x7fe26ed37c87] => /usr/lib/x86_64-linux-gnu/libc.so.6
572: 2022-05-20 14:53:11 (abort+0x000001c7) liboffset 00035c87
573: 2022-05-20 14:53:11 <03810> [0x7fe26ed7f5da] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
574: 2022-05-20 14:53:11 0007d5da
575: 2022-05-20 14:53:11 <03810> [0x7fe26ed80ea9] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
576: 2022-05-20 14:53:11 0007eea9
577: 2022-05-20 14:53:11 <03810> [0x00e07048] => /bin/ipsengine
578: 2022-05-20 14:53:11 <03810> [0x7fe26835ef93] => /data/lib/libips.so
579: 2022-05-20 14:53:11 <03810> [0x7fe26835f118] => /data/lib/libips.so
580: 2022-05-20 14:53:11 <03810> [0x7fe268351fae] => /data/lib/libips.so
581: 2022-05-20 14:53:11 <03810> [0x00e01943] => /bin/ipsengine
582: 2022-05-20 14:53:11 <03810> [0x004362e0] => /bin/ipsengine
583: 2022-05-20 14:53:11 <03810> [0x00e030d7] => /bin/ipsengine
584: 2022-05-20 14:53:11 <03810> [0x00e02545] => /bin/ipsengine
585: 2022-05-20 14:53:11 <03810> [0x004362e0] => /bin/ipsengine
586: 2022-05-20 14:53:11 <03810> [0x0043d2af] => /bin/ipsengine
587: 2022-05-20 14:53:11 <03810> [0x00439f38] => /bin/ipsengine
588: 2022-05-20 14:53:11 <03810> [0x0043a44e] => /bin/ipsengine
589: 2022-05-20 14:53:11 <03810> [0x0043cd29] => /bin/ipsengine
590: 2022-05-20 14:53:11 <03810> [0x7fe26ed22eaa] => /usr/lib/x86_64-linux-gnu/libc.so.6
591: 2022-05-20 14:53:11 (__libc_start_main+0x000000ea) liboffset 00020eaa
592: 2022-05-20 14:53:11 <03810> [0x00432cea] => /bin/ipsengine
593: 2022-05-20 14:53:12 ipsengine 05.002.245 previously crashed 8 times. The last crash was at
594: 2022-05-20 14:53:12 2022-05-20 14:53:11.
595: 2022-05-20 14:53:12 <03807> firmware FortiGate-201F v6.2.9,build7131b1234,210601 (GA) (Release)
596: 2022-05-20 14:53:12 <03807> application ipsengine 05.002.245
597: 2022-05-20 14:53:12 <03807> *** signal 6 (Aborted) received ***
598: 2022-05-20 14:53:12 <03807> Register dump:
599: 2022-05-20 14:53:12 <03807> RAX: 0000000000000000 RBX: 0000000000000006
600: 2022-05-20 14:53:12 <03807> RCX: ffffffffffffffff RDX: 0000000000000000
601: 2022-05-20 14:53:12 <03807> R08: 0000000000000000 R09: 00007fffad9c7990
602: 2022-05-20 14:53:12 <03807> R10: 0000000000000008 R11: 0000000000000246
603: 2022-05-20 14:53:12 <03807> R12: 00007fe26ee7b990 R13: 0000000000000001
604: 2022-05-20 14:53:12 <03807> R14: 000000001b3a4da0 R15: 6d616b203b746360
605: 2022-05-20 14:53:12 <03807> RSI: 00007fffad9c7990 RDI: 0000000000000002
606: 2022-05-20 14:53:12 <03807> RBP: 00007fe26f0adc20 RSP: 00007fffad9c7990
607: 2022-05-20 14:53:12 <03807> RIP: 00007fe26ed36630 EFLAGS: 0000000000000246
608: 2022-05-20 14:53:12 <03807> CS: 0033 FS: 0000 GS: 0000
609: 2022-05-20 14:53:12 <03807> Trap: 0000000000000000 Error: 0000000000000000
610: 2022-05-20 14:53:12 <03807> OldMask: 0000000000004206
611: 2022-05-20 14:53:12 <03807> CR2: 0000000000000000
612: 2022-05-20 14:53:12 <03807> stack: 0x7fffad9c7990 - 0x7fffad9c8c30
613: 2022-05-20 14:53:12 <03807> Backtrace:
614: 2022-05-20 14:53:12 <03807> [0x7fe26ed36630] => /usr/lib/x86_64-linux-gnu/libc.so.6
615: 2022-05-20 14:53:12 (gsignal+0x00000110) liboffset 00034630
616: 2022-05-20 14:53:12 <03807> [0x7fe26ed37c87] => /usr/lib/x86_64-linux-gnu/libc.so.6
617: 2022-05-20 14:53:12 (abort+0x000001c7) liboffset 00035c87
618: 2022-05-20 14:53:12 <03807> [0x7fe26ed7f5da] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
619: 2022-05-20 14:53:12 0007d5da
620: 2022-05-20 14:53:12 <03807> [0x7fe26ed80ea9] => /usr/lib/x86_64-linux-gnu/libc.so.6 liboffset
621: 2022-05-20 14:53:12 0007eea9
622: 2022-05-20 14:53:12 <03807> [0x00e07048] => /bin/ipsengine
623: 2022-05-20 14:53:12 <03807> [0x7fe26835ef93] => /data/lib/libips.so
624: 2022-05-20 14:53:12 <03807> [0x7fe26835f118] => /data/lib/libips.so
625: 2022-05-20 14:53:12 <03807> [0x7fe268351fae] => /data/lib/libips.so
626: 2022-05-20 14:53:12 <03807> [0x00e01943] => /bin/ipsengine
627: 2022-05-20 14:53:12 <03807> [0x004362e0] => /bin/ipsengine
628: 2022-05-20 14:53:12 <03807> [0x00e030d7] => /bin/ipsengine
629: 2022-05-20 14:53:12 <03807> [0x00e02545] => /bin/ipsengine
630: 2022-05-20 14:53:12 <03807> [0x004362e0] => /bin/ipsengine
631: 2022-05-20 14:53:12 <03807> [0x0043d2af] => /bin/ipsengine
632: 2022-05-20 14:53:12 <03807> [0x00439f38] => /bin/ipsengine
633: 2022-05-20 14:53:12 <03807> [0x0043a44e] => /bin/ipsengine
634: 2022-05-20 14:53:12 <03807> [0x0043cd29] => /bin/ipsengine
635: 2022-05-20 14:53:12 <03807> [0x7fe26ed22eaa] => /usr/lib/x86_64-linux-gnu/libc.so.6
636: 2022-05-20 14:53:12 (__libc_start_main+0x000000ea) liboffset 00020eaa
637: 2022-05-20 14:53:12 <03807> [0x00432cea] => /bin/ipsengine
638: 2022-05-20 15:53:18 ipsengine 05.002.245 crashed 35 times. The latest crash was at 2022-05-20
639: 2022-05-20 15:53:18 15:18:03.
640: 2022-08-16 20:42:31 the killed daemon is /bin/eap_proxy: status=0x0
641: 2022-08-19 23:33:34 the killed daemon is /bin/updated: status=0x0
642: 2022-08-20 00:35:04 the killed daemon is /bin/eap_proxy: status=0x0
643: 2022-08-20 01:10:37 the killed daemon is /bin/sshd: status=0x100
644: 2022-08-20 01:26:21 the killed daemon is /bin/updated: status=0x0
645: 2022-08-20 01:26:22 the killed daemon is /bin/updated: status=0x0
646: 2022-08-20 02:36:51 the killed daemon is /bin/eap_proxy: status=0x0
Crash log interval is 3600 seconds
Max crash log line number: 16384