Technical Tip: FortiAnalyzer lag-behind showing in logs

[Nour]

Description	The following article describes the potential causes of FortiAnalyzer's high lag-behind issue when logs are being sent from FortiAnalyzer to a syslog server.
Scope	FortiAnalyzer sends logs to a syslog server.
Solution	Behavior and Symptoms: Logs were not being sent from the FortiAnalyzer to the syslog server. This issue persisted intermittently even after a failover. Checking CPU and other resource utilization of FortiAnalyzer, nothing is out of the norm.  Log forwarding from the FortiAnalyzer showed a high lag rate, and the logs were not received by the syslog server. Check the lag rate with the following command 'diag test app logfwd 4', the output of the command would show a high Lag rate: diag test app logfwd 4 ** Loader: <name of syslog server> lag-behind=99.95% Lag-behind is high or close to 100%, which means logs were queued up in FortiAnalyzer, indicating they were not being received by the syslog server. Taking a packet capture of syslog traffic on the FortiAnalyzer (default port 514), TCP Zero Window errors can be observed in the packet captures sent from the syslog server to the FortiAnalyzer, as can be seen below: This means the syslog server was not able to accept additional packets due to TCP buffer issues.  Check the below settings on the syslog server: Number of parallel TCP connections. TCP receives buffer size. Other TCP settings. Conclusion: The rate of sending logs from FortiAnalyzer to the syslog server was high , which seemed to overwhelm the syslog server, adjusting settings on the syslog server can help in fixing the issue. Furthermore, the RTT delay between FortiAnalyzer and the Syslog server can impact the number of logs sent over TCP.