Technical tip: How to estimate the required disk space for a FortiAnalyzer BigData storage pool that handles traffic logs only
Description
This article describes how to estimate the amount of disk space required when configuring a storage pool in FortiAnalyzer BigData based on traffic logs only.
Scope
FortiAnalyzer BigData.
Solution
FortiAnalyzer BigData uses Storage Pools to manage and organize disk usage.
Storage pools helps to organize log data efficiently, control overall storage utilization, define log retention policies, and maintain system performance by managing data aging.
When a defined storage threshold is reached - either based on retention time or available disk space - the system automatically removes the oldest table partitions within the storage group to free up space.
Log size assumption:
After replication and compression, FortiGate traffic logs typically average ~150 bytes per log entry.
Note: This value is an estimate based on FortiGate traffic logs. Other log types may vary in size (~300 bytes including traffic logs).
The following formula can be used to estimate the required disk space:
<Logs per second> × <86,400 seconds> × <Average log size> × <Retention days>
Example:
- Sustained log rate: 300,000 logs per second
- Retention period: 30 days
- Average compressed log size: 150 bytes
Calculate daily disk usage:
300,000 logs/second × 86,400 seconds/day × 150 bytes = 3.89 TB per day
Calculate total disk required for 30 days:
3.89 TB × 30 days = 117 TB
This calculation assumes a constant sustained log rate.
By performing this estimation in advance, FortiAnalyzer BigData storage pool size can be properly configured to ensure that logs retention requirements are met without impacting system performance.
Related document:
