Description
This article describes how to estimate disk space needed for Archive and Analytics Logs, based on number of days retention requirements.
Scope
FortiAnalyzer after version 5.4.
Solution
When setting up disk space on FortiAnalyzer VM, it is important to size the LVM properly even if increasing is possible and easy to perform (see related article), it is not possible to reduce it and therefore it is good to size it properly when setting the FortiAnalyzer up.
Here is a formula to estimate the minimum disk/quota size, required for retaining the logs and log databases:
HDD=LR*(RA/5+3*RR)*1.1
Where:
HDD - Approximate required total disk/quota size [GB].
LR - Average log rate [GB/day] - Take the average of the weekly log rate statistic under System Settings -> Dashboard -> 'License Information' widget -> GB/Day -> Details.
RA - Retention period for archive/raw logs [days].
RR - Retention period for reporting/analysis [days].
'5' - When the raw logs are archived, their file size is reduced approx 5-8 times. 5 is a bit conservative, and can be replaced with up to 8 for less strict retention policies.
'3' - Multiplier - When the raw logs are inserted, the SQL db files are approximately 3 times bigger than the original log size.
'1.1' - 10% extra, as the disk space cannot be completely utilized. There is some space reserved for cache, temporary tables, etc..
Example:
- If theFortiAnalyzer is receiving in average 8 GB/day.
- If it is necessary to retain the reporting data for 90 days back.
- And also store archived raw logs for strictly 356 days, then:
HDD = 8 * (365/5 + 3*90) * 1.1 = 8 * (73 + 270) * 1.1 = 3018 GB
To allow handling spikes in the log rate, it's always better to have more than the calculated minimum space.
Version 6.4.3 introduced SQL table compression which reduces disk usage:
# config system sq
set compress-table-min-age <----- Minimum age of the log tables in days
end
The estimation formula does not consider this compression factor.
Related KB article:
https://community.fortinet.com/t5/FortiAnalyzer/Technical-Tip-Extending-disk-space-in-FortiAnalyzer-...
