Description
This article describes about the log rolling, trimming process and HDD space optimization in FortiAnalyzer.
The FortiAnalyzer allows to log system events to disk.
It is possible to control 'device log file size' and the use of the FortiAnalyzer unit’s disk space by configuring log rolling.
Solution
Introduction.
Whenever new logs enter into FortiAnalyzer, the log type is determined from its header, and depending on that it's saved in the respective files on the disk.
Example 'e.log',' t.log'.
It is possible to view under:
LogView -> Log browse.
These files on the disk are called the 'archive'.
The SQL daemons pick the log that was just saved in the archive, and insert it in the log database.
These logs in database are known as 'analytic' log.
The log files ('e.log', 't.log') are rolled as per the configuration done under:
System Settings -> Advanced -> Device log settings and roll log file when size exceeds -> Value.
When a log file reaches its maximum size configured, FortiAnalyzer rolls the active log file by renaming the file.
The file name is in the form of xlog.N.log, where x is a letter indicating the log type, and N is a unique number, corresponding to the time the first log entry was received example: 'elog.1611593395.log.gz'.
Automatic deletion.
Logs and files are automatically deleted from the FortiAnalyzer unit according to the following settings:
- Global: automatic file deletion file management settings specify when to delete the oldest archive logs, quarantined files, reports, and archived files from disks, regardless of the log storage settings.
- Data policy: data policies specify how long to store analytics and archive logs for each unit. When the specified length of time expires, Archive logs for the device are automatically deleted from the FortiAnalyzer unit's disks.
- Disk utilization: disk utilization settings delete the oldest archive logs for each unit when the allotted disk space is filled. The allotted disk space is defined by the log storage.
Configured percentage.
It is possible to run this command to check that unit and ADOM disk quota
weepy-fmg-esx41 # diag log deviceGeneral Problem Summary: Archive logs, where it shows 1128 of 365.
Device Name Device ID Used Space(logs / quarantine / content / IPS) Allocated Space Used%
Total: 0 log devices, used=0.0KB quota=unlimited
AdomName AdomOID Type Logs Database
[Retention Quota Used( logs/quaranti/ content/ IPS) Used%] [Retention Quota Used( SiemDB/ hcache) Used%]
FortiAnalyzer 128 FAZ 365days 300.0MB 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) 0.0% 60days 700.0MB 0.0KB( 0.0KB/ 0.0KB) 0.0%
FortiAuthenticator 144 FAC 365days 300.0MB 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) 0.0% 60days 700.0MB 0.0KB( 0.0KB/ 0.0KB) 0.0%
FortiClient 134 FCT 365days 300.0MB 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) 0.0% 60days 700.0MB 0.0KB( 0.0KB/ 0.0KB) 0.0%
Unmanaged_Devices 148 FSF 365days 300.0MB 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) 0.0% 60days 700.0MB 0.0KB( 0.0KB/ 0.0KB) 0.0%
fgg 194 FGT 365days unlimited 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) n/a 60days unlimited 0.0KB( 0.0KB/ 0.0KB) n/a
root 3 FSF 365days 15.0GB 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) 0.0% 60days 35.0GB 1.8MB( 0.0KB/ 0.0KB) 0.0%
test 192 FCT 365days unlimited 0.0KB( 0.0KB/ 0.0KB/ 0.0KB/ 0.0KB) n/a 60days unlimited 0.0KB( 0.0KB/ 0.0KB) n/a
Total usage: 19 ADOMs, logs=0.0KB database=690.1MB(ADOMs usage:1.8MB + Internal Usage:688.3MB)
Total Quota Summary:
*** Warning: Total Allocated Quota is bigger than Total Quota! Please check the quota configuration of ADOMs!
Total Quota Allocated Available Allocate%
63.2GB 65.6GB 0.0KB 103.8%
System Storage Summary:
Total Used Available Use%
78.2GB 6.7GB 71.5GB 8.6 %
Reserved space: 15.0GB (19.2% of total space).
This is likely caused by log files from low lograte units (or low volume log types) which have not yet reached the configured rolling size.
The active log files that are being still used cannot be deleted, hence preventing the retention enforcement.
If this is case, it is necessary to configure daily rolling of the log files.
The active log files that are being still used cannot be deleted, hence preventing the retention enforcement.
If this is case, it is necessary to configure daily rolling of the log files.
This is under System Settings -> Advanced -> Device Log Settings and 'roll log files at scheduled time'.
Roll logs when they reach a specific size.
Use the following CLI commands to specify the size, in MB, at which a log file is rolled.
To roll logs when they reach a specific size:
Use the following CLI commands to configure rolling logs on a set schedule, or never.
To disable log rolling.
Enabling 'Roll log files at scheduled time' will roll these old files but they would not be deleted automatically until their 'To' date becomes 365 days old.
When the unit logs are older than the Keep Logs for analytics setting, there are automatically deleted.
Use the following CLI commands to specify the size, in MB, at which a log file is rolled.
To roll logs when they reach a specific size:
# config system log settingsRoll logs on a schedule.
# config rolling-regular
set file-size <integer>
end
Use the following CLI commands to configure rolling logs on a set schedule, or never.
To disable log rolling.
# config system log settingsTo enable daily log rolling.
# config rolling-regular
set when none
end
# config system log settingsIf the daily rolling setting is disabled, FortiAnalyzer would wait until the files reach the specified size before rolling them.
# config rolling-regular
set upload enable
set when daily
set hour <integer>
set min <integer>
end
Enabling 'Roll log files at scheduled time' will roll these old files but they would not be deleted automatically until their 'To' date becomes 365 days old.
When the unit logs are older than the Keep Logs for analytics setting, there are automatically deleted.
Also, when analytic logs exceed their disk quota, the SQL database trimming process is started with the oldest database tables.
Troubleshooting section:
Wait for the supposed rolling event to happen.
Troubleshooting section:
# Exe tac report
# Diag debug application logfiled 255
# Diag debug enable
Wait for the supposed rolling event to happen.
# Diag debug disableCollect and Send TAC the event system log and the output of the command # diag debug command
Related articles:
https://community.fortinet.com/t5/FortiAnalyzer/Technical-Note-Minimizing-logging-from-FortiGate-to/...
https://community.fortinet.com/t5/FortiAnalyzer/Technical-Tip-How-to-calculate-disk-space-needed-for...
Technical Tip: Why analytics log size decreased after firmware upgrade 6.4.x version
