Description
This article describes how to use the 'diagnose sys top' command from the CLI.
Scope
FortiGate.
Solution
Use the 'diagnose sys top' command from the CLI to list the processes running on the FortiGate.
The command also displays information about each process.
Example output (up to 6.4):
diagnose sys top
Run Time: 13 days, 13 hours and 58 minutes
0U, 0S, 98I; 123T, 25F, 32KF
newcli 903 R 0.5 5.5
sshd 901 S< 0.5 4.0
Example output (from 7.0):
Run Time: 0 days, 18 hours and 6 minutes
0U, 0N, 0S, 100I, 0WA, 0HI, 0SI, 0ST; 3039T, 1950F
bcm.user 97 S < 3.4 0.4 0
snmpd 192 S 0.9 0.2 1
forticron 173 S 0.4 0.6 0
fcnacd 181 S 0.4 0.3 1
newcli 4488 R < 0.4 0.2 1
Here, the codes displayed on the second output line mean the following:
U is the percentage of user space applications using CPU. In the example, 0U means 0% of the user space applications are using the CPU.
S is the percentage of system processes (or kernel processes) using CPU. In the example, 0S means 0% of the system processes are using the CPU.
I is the percentage of idle CPU. In the example, 98I means the CPU is 98% idle.
T is the total FortiOS system memory in Mb. In the example, 123T means there are 123 Mb of system memory.
F is free memory in Mb. In the example, 25F means there is 25 Mb of free memory.
KF is the total shared memory pages used. In the example, 32KF means the system is using 32 shared memory pages.
Each additional line of the command output displays information for each of the processes running on the FortiGate.
For example, the third line of the output is:
newcli 4488 R < 0.4 0.2 1
In this instance, newcli is the process name.
Other process names can include ipsengine, sshd, cmdbsrv, httpsd, scanunitd, and miglogd.
4488 is the process ID. The process ID can be any number.
R is the state that the process is running in. The process state can be:
R running.
S sleep.
Z zombie.
D disk sleep.
T stopped.
< on a process means that it is a process with higher priority compared to remaining ones (is not nice to all remaining processes).
N on a process means that it is a process with lower priority compared to the remaining ones (is nice to all remaining processes).
The D state is particularly important, as it implies that something is wrong with the disk IO, meaning the process can therefore not continue running because it cannot read from or write to the flash disk.
0.4 is the amount of CPU that the process is using. CPU usage can range from 0.0 for a process that is sleeping to higher values for a process that is taking a lot of CPU time (This utilization is per core on which the process is running).
0.2 is the amount of memory that the process is using.
1 (last column, newly added in 7.0) is the CPU core on which this process is running.
Below are some interactive 'diagnose sys top' commands.
Enter the following single-key commands when 'diagnose sys top' is running to sort by columns.
‘M’ to sort by memory usage
‘P’ to sort by CPU usage
‘N’ to sort by process ID
‘T’ to sort by the running time
‘Q’ to quit
By default, the 'diag sys top' command refreshes every 5 seconds.
If the commands need to be run for any specific duration with a fixed number of lines, the following options are available:
diagnose sys top <Delay_in_seconds> <Maximum_lines_to_display> <Iterations_to_run>
Delay in seconds (default 5).
Maximum lines to display (default 20). Show all the running processes if larger than its total number.
Iterations to run (default unlimited).
The only difference is that the latter command can be run in certain iterations as specified in the last argument.
Other options to filter the output of this command can be useful to focus on specific aspect of the reported processes:
top-all Show top threads information.
top-fd Display processes with the most active file descriptors (default 5 processes).
top-mem Display processes with the most used memory (default 5 processes).
top-sockmem Display processes with the most used socket memory (default 5 processes).
Example:
diagnose sys top-fd 20
wad (7688): 442
ipshelper (7631): 103
wad (7694): 74
wad (7697): 74
wad (7699): 74
wad (7689): 73
wad (7690): 73
wad (7691): 73
wad (7692): 73
wad (7693): 73
wad (7695): 73
wad (7698): 73
wad (7696): 72
authd (7627): 54
miglogd (7637): 46
dnsproxy (7670): 46
miglogd (7761): 40
miglogd (7760): 39
miglogd (7762): 39
miglogd (7763): 39
An additional argument can be inserted as well to sort the output of the command as the following:
- n number of top processes to show.
- i interval.
- s sort by cpu|mem|fds|pid.
- dump to the file (saved in /tmp/top).
Example:
diagnose sys top -i 20 1
Run Time: 0 days, 5 hours and 38 minutes
0U, 0N, 0S, 100I, 0WA, 0HI, 0SI, 0ST; 16010T, 13377F
appDemo 2247 S 2.5 0.5 11
node 7618 S 0.0 0.4 10
ipshelper 7631 S < 0.0 0.4 6
miglogd 7637 S 0.0 0.3 11
wad 7699 S 0.0 0.2 1
wad 7689 S 0.0 0.2 4
wad 7697 S 0.0 0.2 7
wad 7695 S 0.0 0.2 6
wad 7692 S 0.0 0.2 11
wad 7694 S 0.0 0.2 9
wad 7698 S 0.0 0.2 8
wad 7690 S 0.0 0.2 11
wad 7696 S 0.0 0.2 5
wad 7693 S 0.0 0.2 10
wad 7691 S 0.0 0.2 0
miglogd 7764 S 0.0 0.2 6
miglogd 7765 S 0.0 0.2 5
miglogd 7763 S 0.0 0.2 7
miglogd 7762 S 0.0 0.2 8
miglogd 7761 S 0.0 0.2 9
Stopping running processes:
Use the following command to stop running processes:
diagnose sys kill <signal> <process id>
In this example:
- <signal> can be any number but 11 is preferred because this signal sends output to the crashlog which can be used by Fortinet Support to troubleshoot problems.
- <process id> is the process ID listed by the diagnose sys top command.
To find the process IDs of a single daemon:
diagnose sys process pidof <process name>
For example, to stop the process with process ID 903, enter the following command:
diagnose sys kill 11 903
To kill/restart all the process IDs using the single daemon, use the command below:
fnsysctl killall <process name>
Note: Super Admin privileges are necessary to run the 'fnsysctl' command. Otherwise, FortiGate will return an error. See this article.
To monitor the resource usage by any daemon, grep may be used:
diagnose sys top 5 99 | grep wad
wad 185 S 0.4 0.4 0
wad 191 S 0.4 0.3 3
wad 184 S 0.0 0.4 1
wad 173 S 0.0 0.4 1
wad 183 S 0.0 0.3 1
wad 190 S 0.0 0.3 0
Related document:
CLI command reference - 'system performance top'.
