Description
This article describes how to estimate the disk space needed for Archive and Analytics logs based on the number of retention days required.
Scope
FortiAnalyzer after version 5.4.
Solution
When setting up disk space on FortiAnalyzer-VM, it is important to scale the LVM properly. Even if increasing the size is possible and easy to perform (see the related article), it is not possible to reduce VM size. It is therefore advised to pick a suitable size when first setting up the FortiAnalyzer.
Hard disk settings information mentioned on VMWare ESXI as below:
Configuring hardware settings
Below is a formula to estimate the minimum disk/quota size required for retaining the logs and log databases:
HDD=LR*(RA/5 + 3*RR)*1.1
Where:
HDD - Approximate required total disk/quota size [GB].
LR - Average log rate [GB/day] - Take the average of the weekly log rate statistic under System Settings -> Dashboard -> 'License Information' widget -> GB/Day -> Details.
RA - Retention period for archive/raw logs [days].
RR - Retention period for reporting/analysis/realtime [days].
'5' - When the raw logs are archived, their file size is reduced approx 5-8 times. 5 is a bit conservative, and can be replaced with up to 8 for less strict retention policies.
'3' - Multiplier - When the raw logs are inserted, the SQL db files are approximately 3 times bigger than the original log size.
'1.1' - 10% extra, as the disk space cannot be completely utilized. There is some space reserved for cache, temporary tables, etc..
For example:
- The FortiAnalyzer is receiving 8 GB per day on average.
- Itis necessary to retain the reporting data for 90 days at a time.
- It is also necessary to store archived raw logs for strictly 356 days.
HDD = 8 * (365/5 + 3*90) * 1.1 = 8 * (73 + 270) * 1.1 = 3018 GB
To allow for handling spikes in the log rate, it is always better to have more than the calculated minimum space.
In case Administrative Domain [ADOM] is enabled, then to calculate how much disk space is needed to be allocated to which ADOM it can be used the same Formula, but the Average log rate should be per-ADOM.
- Value of GB/day by ADOM can be defined only via CLI:
diagnose fortilogd logvol-adom <adom_name>
The command above will print weekly statistics about Lograte GB/Day for all ADOM and the average value.
Version 6.4.3 introduced SQL table compression,which reduces disk usage:
config system sql
set compress-table-min-age <----- Minimum age of the log tables in days.
end
The estimation formula does not consider this compression factor.
Related articles:
