Technical Tip: Frequent 'fgtlogd' Daemon Crashes on FortiGate when FortiAnalyzer Cloud Logging is Enabled

After upgrading FortiGate to v7.6.0, 'fgtlogd' daemon crashes are observed when FortiGate is configured with FortiAnalyzer Cloud logging.

diagnose debug crashlog read
8521: 2024-10-29 17:22:04 <02341> firmware FortiGate-61F v7.6.0,build3401b3401,240724 (GA.F) (Release)
8522: 2024-10-29 17:22:04 <02341> application fgtlogd
8523: 2024-10-29 17:22:04 <02341> *** signal 11 (Segmentation fault) received ***
8524: 2024-10-29 17:22:04 <02341> Register dump:
8525: 2024-10-29 17:22:04 <02341> R0: 00000000099e2f60 R1: 000000000000032f R2: 0000000000000000
8526: 2024-10-29 17:22:04 <02341> R3: 000000000000032e R4: 0000007fa39a9c81 R5: 0000000009a760af
8527: 2024-10-29 17:22:04 <02341> R6: 000000000000007d R7: 656361667265746e XR: 000000000000007d
8528: 2024-10-29 17:22:04 <02341> R9: 0000000004a13b80 R10: 000000000000007d R11: 0000007ff472b858
8529: 2024-10-29 17:22:04 <02341> R12: 6e695f656e696c6e R13: 7365636166726574 R14: 0000000000000000
8530: 2024-10-29 17:22:04 <02341> R15: 0000000000000050 IP0: 0000000000000000 IP1: 0000007fa4c47540
8531: 2024-10-29 17:22:04 <02341> PR: 0000000000000000 R19: 00000000099e1108 R20: 0000000009a61eb0
8532: 2024-10-29 17:22:04 <02341> R21: 0000000009a75d80 R22: 00000000099e2f60 R23: 0000000000000362
8533: 2024-10-29 17:22:04 <02341> R24: 0000007ff472bcc4 R25: 000000000000a83a R26: 0000000009af3168
8534: 2024-10-29 17:22:04 <02341> R27: 00000000099e0d28 R28: 000000000229c648 FP: 0000007ff472bc10
8535: 2024-10-29 17:22:04 <02341> fault_address: 0000000000000000 sp: 0000007ff472bc10
8536: 2024-10-29 17:22:04 <02341> pc: 0000000000000000 lr: 0000000000b64214
8537: 2024-10-29 17:22:04 <02341> pstate: 80000000 (Nzcv daif -PAN -UAO)
8538: 2024-10-29 17:22:04 <02341> Call to function with invalid address, attempting manual stack walk.
8539: 2024-10-29 17:22:04 <02341> Backtrace:
8540: 2024-10-29 17:22:04 <02341> [0x00000000]
8541: 2024-10-29 17:22:04 <02341> [0x00764214] => /bin/fgtlogd
8542: 2024-10-29 17:22:04 <02341> [0x01ea2b94] => /bin/fgtlogd
8543: 2024-10-29 17:22:04 <02341> [0x0076447c] => /bin/fgtlogd
8544: 2024-10-29 17:22:04 <02341> [0x026dc5b8] => /bin/fgtlogd
8545: 2024-10-29 17:22:04 <02341> [0x026dc758] => /bin/fgtlogd
8546: 2024-10-29 17:22:04 <02341> [0x02776d5c] => /bin/fgtlogd
8547: 2024-10-29 17:22:04 <02341> [0x0269f52c] => /bin/fgtlogd
8548: 2024-10-29 17:22:04 <02341> [0x0004f314] => /bin/fgtlogd
8549: 2024-10-29 17:22:04 <02341> [0x0005525c] => /bin/fgtlogd
8550: 2024-10-29 17:22:04 <02341> [0x00055aa0] => /bin/fgtlogd
8551: 2024-10-29 17:22:04 <02341> [0x0005868c] => /bin/fgtlogd
8552: 2024-10-29 17:22:04 <02341> [0x0005916c] => /bin/fgtlogd
8553: 2024-10-29 17:22:04 <02341> [0x00020f24] => /usr/lib/aarch64-linux-gnu/libc.so.6
8554: 2024-10-29 17:22:04 (__libc_start_main)
8555: 2024-10-29 17:22:04 <02341> [0x0004ae00] => /bin/fgtlogd
8556: 2024-10-29 17:22:04 <02341> fortidev 6.0.1.0005
8557: 2024-10-29 17:22:04 the killed daemon is /bin/fgtlogd: status=0xb
8558: 2024-10-29 17:37:05 fgtlogd previously crashed 9 times. The last crash was at 2024-10-29 17:22:04.

This issue has been resolved in v7.6.1.

Logs required by FortiGate TAC for investigation.

1. Debugs:

diagnose debug application fgtlogd -1
diagnose debug console timestamp enable
diagnose debug enable
<wait for 2-3mins>
diagnose debug reset
diagnose user-device-store device memory list

diagnose sniffer migsock timestamp enable
diagnose sniffer migsock filter name=devicequery
diagnose sniffer migsock start
<wait for the output until it stops printing, it may take around 15 to 30 minutes>

diagnose test application fgtlogd 4
diagnose test app fgtlogd 20
diagnose test app forticldd 3

2. TAC Report:

execute tac report

3. Configuration file of the FortiGate.