Description
FortiGates with a SSD disk have a configurable log buffer.
When the connection to FortiAnalyzer is unreachable, the FortiGate is able to buffer logs on disk if the memory log buffer is full.
The logs queued on the disk buffer can be sent successfully once the connection to FortiAnalyzer is restored.
Solution
The number of logs queued on the disk buffer is visible under Log & Report -> Log Settings page:
FortiGates with a SSD disk have a configurable log buffer.
When the connection to FortiAnalyzer is unreachable, the FortiGate is able to buffer logs on disk if the memory log buffer is full.
The logs queued on the disk buffer can be sent successfully once the connection to FortiAnalyzer is restored.
Solution
The number of logs queued on the disk buffer is visible under Log & Report -> Log Settings page:
The queued logs are buffered to the memory first and then disk. Main miglogd handles the disk buffering job, while miglogd-children handles the memory buffering. Disk buffer statistics only appear under Main miglogd, and memory buffer statistics only appears under miglogd-children. If the total buffer is full, new logs will overwrite the old logs.
To configure the log buffer:
Allocate disk space (MB) to temporarily store logs to FortiAnalyzer:
To configure the log buffer:
Allocate disk space (MB) to temporarily store logs to FortiAnalyzer:
# config system globalCheck the Main miglogd and miglogd-children statistics. The 200 MB disk buffer has been set, and there are currently no logs buffered in memory or on disk when FortiAnalyzer is reachable:
set faz-disk-buffer-size 200
end
# diagnose test application miglogd 41 0Disable the connection between the FortiGate and FortiAnalyzer.
cache maximum: 106100940(101MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
VDOM:root
Queue for: global-faz
memory queue:
num:0 size:0(0MB) max:101906636(97MB) logs:0
disk max queue size:200MB total:0MB
totol items:0
disk queue agents:
devid:-1-10-0-1
buffer path:/var/log/qbuf/10.0/1
saved size:0MB cached size:0
save roll:0 restore roll:0
restore id:0 space:0MB
# diagnose test application miglogd 41 1
cache maximum: 106100940(101MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
VDOM:root
Queue for: global-faz
memory queue:
num:0 size:0(0MB) max:101906636(97MB) logs:0
disk queue client:
devid:-1-10-0-1 status:buffering
Total in cache:0 size:0(0MB) max:4MB logs:0
For example, delete the FortiGate from the FortiAnalyzer authorized unit list.
Assuming a massive number of logs (~ 300000) are recorded during this downtime, the logs will be queued in the memory buffer first.
If the memory buffer is full, then the remaining logs will be queued on the disk buffer.
Check the Main miglogd and miglogd-children statistics again.
All 97 MB of the memory buffer is occupied, and 76 of the 200 MB has been taken from the disk buffer:
After a while, check the miglogd statistics to confirm that all buffered logs are being sent to FortiAnalyzer successfully:
# diagnose test application miglogd 41 0The overall miglogd statistics shows the total cached logs is the sum of the logs buffered in memory and on disk:
cache maximum: 106100940(101MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
VDOM:root
Queue for: global-faz
memory queue:
num:0 size:0(0MB) max:101906636(97MB) logs:0
disk max queue size:200MB total:76MB
totol items:128917
disk queue agents:
devid:-1-10-0-1
buffer path:/var/log/qbuf/10.0/1
saved size:76MB cached size:3324984
save roll:19 restore roll:0
restore id:0 space:0MB
# diagnose test application miglogd 41 1
cache maximum: 106100940(101MB) objects: 165721 used: 101908358(97MB) allocated: 106449280(101MB)
VDOM:root
Queue for: global-faz
memory queue:
num:165718 size:101906500(97MB) max:101906636(97MB) logs:165718
disk queue client:
devid:-1-10-0-1 status:restoring
restore id:1267 space:0MB
Total in cache:3 size:1858(0MB) max:4MB logs:3
# diagnose test application miglogd 6Enable the connection between FortiAnalyzer and the FortiGate.
mem=0, disk=11, alert=0, alarm=0, sys=0, faz=300053, faz-cloud=0, webt=0, fds=0
interface-missed=44
Queues in all miglogds: cur:165718 total-so-far:165718
global log dev statistics:
faz 0: sent=0, failed=0, cached=300053, dropped=0 , relayed=0
Num of REST URLs: 0
After a while, check the miglogd statistics to confirm that all buffered logs are being sent to FortiAnalyzer successfully:
# diagnose test application miglogd 6
mem=0, disk=11, alert=0, alarm=0, sys=0, faz=300058, faz-cloud=0, webt=0, fds=0
interface-missed=44
Queues in all miglogds: cur:4294832957 total-so-far:165726
global log dev statistics:
faz 0: sent=300058, failed=0, cached=0, dropped=0 , relayed=0
Num of REST URLs: 15
# diagnose test application miglogd 41 0
cache maximum: 106100940(101MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
VDOM:root
Queue for: global-faz
memory queue:
num:0 size:0(0MB) max:101906636(97MB) logs:0
disk max queue size:200MB total:0MB
totol items:0
disk queue agents:
devid:-1-10-0-1
buffer path:/var/log/qbuf/10.0/1
saved size:0MB cached size:0
save roll:20 restore roll:20
restore id:1267 space:0MB
# diagnose test application miglogd 41 1
cache maximum: 106100940(101MB) objects: 0 used: 0(0MB) allocated: 0(0MB)
VDOM:root
Queue for: global-faz
memory queue:
num:0 size:0(0MB) max:101906636(97MB) logs:0
disk queue client:
devid:-1-10-0-1 status:buffering
Total in cache:0 size:0(0MB) max:4MB logs:0
