Technical Tip: Minimizing logging from FortiGate to FortiAnalyzer

[rakanda]

Description

This article describes how to limit logs from the FortiGate. Such reductions in logging may be caused, for example, by exceeding the licensed daily log limit of a FortiAnalyzer.

Scope

FortiGate, FortiAnalyzer.

Solution

FortiAnalyzer displays the message 'You have exceeded your daily GB Logs/Day within 7 days' when, within the last 7 day period, connected FortiGates have exceeded the daily logging allowance specified by the licensing. FortiAnalyzer will continue to display this warning message as long as that limit is being exceeded.

If the FortiAnalyzer's licensed log limit is being routinely exceeded over a 7 day period, it may be a good time to consider a license upgrade, or adjusting system resources to meet the logging demand (see also: Minimum system requirements). If system resources are insufficient, the FortiAnalyzer-VM may drop incoming logs. Consistently running over-capacity will eventually lead to unexpected behavior. 

It is also important to note that the license state of the FortiAnalyzer affects technical support entitlements (though it does not impact logging functionality on the unit itself). For example, Fortinet technical support teams will not be able to investigate any issues while the FortiAnalyzer-VM is in a license-exceeded state (GB/day), which means that any ongoing incidents/issues will face delays in resolution until the licensing issues are resolved.

With that in mind, there are several ways to limit the volume of logs sent from the FortiGate to FortiAnalyzer:

1. If possible, disable or reduce logging on Firewall Policies for internal/trusted traffic
   • For example, consider switching Log allowed traffic from All sessions to Security Events (only logs traffic blocked/monitored by UTM inspection profiles) or even disabling logging entirely.
   • Note: when the FortiGate is configured in a Security Fabric then this cannot be modified, as the fabric functionality relies on logging of all sessions to FortiAnalyzer.

2. Reduce logs generated by UTM profiles (Antivirus, Web Filter, Application Control, etc.)
   • It is possible to disable logging functionality within various UTM profiles, though generally this can only be done from the CLI
   • Refer to the FortiGate CLI Reference documentation for more information, as well as Knowledge Base Articles on the Fortinet Community site:
     • FortiOS CLI Reference - config antivirus profile
     • FortiOS CLI Reference - config application list
     • FortiOS CLI Reference - config webfilter profile
     • Technical Tip: Configure web filter and URL filter via CLI
     • Technical Tip: Disable specific IPS Signature logging

3. Limit logs using the log filter functionality. This allows administrators to specify log severities/categories that should be logged to FortiAnalyzer.
   • For a more fine-grained approach, create free-style filters to further filter based on particular log fields/values. Refer to the following documents for more information:
     • Technical Tip: Using syslog free-style filters
     • FortiOS CLI Reference - config log fortianalyzer filter

config log fortianalyzer filter

set severity [emergency|alert|...]
set forward-traffic [enable|disable]
set local-traffic [enable|disable]
set multicast-traffic [enable|disable]
set sniffer-traffic [enable|disable]
set ztna-traffic [enable|disable]
set anomaly [enable|disable]

[...]

config free-style

edit <id>

set category [traffic|event|...]

set filter {string}

set filter-type [include|exclude]

next

end

4. Limit logging of local-in traffic using the log setting CLI section (available on a per-VDOM basis when using VDOMs).
   • See also: FortiOS CLI Reference - config log setting.

config log setting

[...]

set fwpolicy-implicit-log [enable|disable]
set log-invalid-packet [enable|disable]
set local-in-allow [enable|disable]
set local-in-deny-unicast [enable|disable]
set local-in-deny-broadcast [enable|disable]
set deamon-log [enable|disable]

[...]

end

Note: In FortiOS 7.4.0 and later, the command set log-invalid-packet shown above has been replaced with the following:

config log setting

set extended-log [enable|disable]

end

5. Use the integrated log shaping function to impose a maximum logging rate on the FortiGate (measured in megabytes per second, aka MBps, not megabits per second). Note that logs generated in-excess of the maximum rate will be dropped/lost, so take caution when implementing this option.

   • This feature is also supported for syslog and FortiGate Cloud logging.

config log fortianalyzer setting

set priority [default | low] <--- Define socket priority of log traffic vs. user traffic over interface
set max-log-rate <0 - 100000, default = 0 (unlimited MBps)>

end

Note: If logs are dropped due to max-log-rate then an event log is generated every hour to indicate the number of logs dropped. It is also possible to check the total number of dropped logs using the following test command:

diagnose test application miglogd 40

diagnose test application fgtlogd 5 <--- FortiOS 7.2 and later.

Related article:

Technical Tip: How to troubleshoot the 'daily logs GB/day limit is exceeded' warning on FortiAnalyze....