|
There are scenarios in which the user is using a disk for logging, and if disk rollover is not happening as per the settings.
Log_se process may go high as below.
diagnose sys top 1 10
Run Time: 89 days, 15 hours and 11 minutes
0U, 46N, 5S, 4I, 44WA, 1HI, 0SI, 0ST; 24140T, 11438F
log_se 11066 R N 38.0 0.1 8
log_se 11061 R N 37.5 0.1 10
log_se 10998 D N 37.5 0.1 3
log_se 11003 D N 37.5 0.1 11
log_se 11072 D N 37.5 0.1 14
log_se 11075 D N 37.5 0.1 3
log_se 11000 D N 37.0 0.1 1
log_se 11097 R N 37.0 0.1 15
log_se 11025 D N 36.5 0.1 2
log_se 11016 R N 36.0 0.1 4
Run the command 'get sys perf status', 'dia sys mpstat', and 'dia sys top 1 10' to see in which area the load is present consistently. The output will be something like below (depending on the number of cores the unit has):
The first line, 'CPU states:' shows the average load across all CPU cores.
get sys perf stat
CPU states: 20% user 60% system 0% nice 20% idle 0% iowait 0% irq 0% softirq
CPU0 states: 21% user 62% system 0% nice 17% idle 0% iowait 0% irq 0% softirq
CPU1 states: 44% user 56% system 0% nice 0% idle 0% iowait 0% irq 0% softirq
CPU2 states: 32% user 58% system 0% nice 10% idle 0% iowait 0% irq 0% softirq
...
In some cases ,the CPU goes high do to iowait and can relate to the log_se issue based on the below logs:
CPU states: 0% user 0% system 7% nice 2% idle 91% iowait 0% irq 0% softirq
CPU0 states: 0% user 0% system 5% nice 3% idle 91% iowait 1% irq 0% softirq
CPU1 states: 0% user 0% system 8% nice 1% idle 91% iowait 0% irq 0% softirq
CPU2 states: 0% user 0% system 8% nice 2% idle 89% iowait 1% irq 0% softirq
- Process log_SE is used by the log disk. It is the process of handling the log search in the GUI.
- All 'log_se' processes are running with low priority (process state N) so it shouldn't affect other running processes.
diagnose sys top 1 10
Run Time: 89 days, 15 hours and 11 minutes
0U, 46N, 5S, 4I, 44WA, 1HI, 0SI, 0ST; 24140T, 11438F
log_se 11066 R N 38.0 0.1 8
log_se 11061 R N 37.5 0.1 10
Related article:
Technical Tip: Using the 'diagnose sys top' CLI command
There is a chance that the firewall disk is not functioning correctly if log_se is high. Try disabling the log_se as a temporary workaround.
config log disk setting
set status disable
end
If below is the case, then try a workaround by reducing the log-age:
config log disk setting
set status enable
set maximum-log-age 3650
end
Then reduce the log age to 7.
config log disk setting
set status enable
set maximum-log-age 7
end
execute formatlogdisk
It is possible to perform a failover and see if another firewall disk is healthy, or to format the disk for the fix if suspected issue with the disk malfunctioning.
Log to be collected:
CLI Session (1)
get sys perf status
diag sys top 2 50
diag sys mpstat 2
For disk logging-related debug:
diagnose debug reset
diagnose debug enable
diagnose debug console timestamp enable
diag debug app miglogd 0x1000
diag deb app httpsd -1
Monitor CPU utilization of 'log_se' processes from FortiGate CLI Session (1) and then stop debugging once the processes are gone and the CPU is back in a normal state.
Stop the debug:
diagnose debug disable
diagnose debug console timestamp disable
diagnose debug app miglogd 0
diagnose debug app httpsd 0
diagnose debug reset
Collect the logs below to verify the disk log storage.
fnsysctl ls -l /var/log/log/root
diagnose hardware deviceinfo disk
fnsysctl df
fnsysctl ls /var/log -l
|
