Description
This article describes how to estimate the disk space needed for Archive and Analytics logs based on the number of retention days required.
Scope
FortiAnalyzer after version 5.4.
Solution
When setting up disk space on FortiAnalyzer-VM, it is important to scale the LVM properly. Even if increasing the size is possible and easy to perform (see the related article), it is not possible to reduce VM size. It is therefore good to pick a proper size when setting up the FortiAnalyzer.
Below is a formula to estimate the minimum disk/quota size required for retaining the logs and log databases:
HDD=LR*(RA/5+3*RR)*1.1
Where:
HDD - Approximate required total disk/quota size [GB].
LR - Average log rate [GB/day] - Take the average of the weekly log rate statistic under System Settings -> Dashboard -> 'License Information' widget -> GB/Day -> Details.
RA - Retention period for archive/raw logs [days].
RR - Retention period for reporting/analysis [days].
'5' - When the raw logs are archived, their file size is reduced approx 5-8 times. 5 is a bit conservative, and can be replaced with up to 8 for less strict retention policies.
'3' - Multiplier - When the raw logs are inserted, the SQL db files are approximately 3 times bigger than the original log size.
'1.1' - 10% extra, as the disk space cannot be completely utilized. There is some space reserved for cache, temporary tables, etc..
For example:
- The FortiAnalyzer is receiving 8 GB per day on average.
- Itis necessary to retain the reporting data for 90 days at a time.
- It is also necessary to store archived raw logs for strictly 356 days.
HDD = 8 * (365/5 + 3*90) * 1.1 = 8 * (73 + 270) * 1.1 = 3018 GB
To allow for handling spikes in the log rate, it is always better to have more than the calculated minimum space.
Version 6.4.3 introduced SQL table compression,which reduces disk usage:
config system sql
set compress-table-min-age <----- Minimum age of the log tables in days.
end
The estimation formula does not consider this compression factor.
Related articles:
Technical Tip: Extending disk space in FortiAnalyzer VM.
Technical Tip: How to troubleshoot the 'daily logs GB/day limit is exceeded' warning on FortiAnalyze...
