FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Wallerson
Staff
Staff
Article Id 332364
Description This article describes how to use 'diagnose sys profile report'.
Scope FortiGate v7.0 and above.
Solution

The 'diagnose sys profile report' command is a hidden command that may be highly useful to troubleshoot high CPU issues, mainly during events where the high CPU is due to 'softirq'. For more information about 'softirq', see Troubleshooting Tip: Check SoftIrq increments (recommended when experiencing high CPU usage).

 

As mentioned above, the 'report' parameter is hidden:

 

diagnose sys profile
start           start kernel profiling data
stop           copy kernel profiling data
show         show kernel profiling result
sysmap      show kernel sysmap
cpumask    profile which CPUs [Take 0-10 arg(s)]
step           set profile step
module     show kernel module

 

The command will display the following outputs if the CPU usage is low:

 

get system performance status
CPU0 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq

 

diagnose sys profile report
CPU Kernel Percentages:
0: 1% (1 of 100). Not profiling.
No busy CPUs found.

 

If the FortiGate has more CPUs, a line will be displayed for each one:

 

diag sys profile report
CPU Kernel Percentages:
0: 44% (44 of 100). Not profiling.
1: 37% (37 of 100). Not profiling.
2: 45% (45 of 100). Not profiling.
3: 29% (29 of 100). Not profiling.
4: 42% (42 of 100). Not profiling.
5: 39% (39 of 99). Not profiling.
6: 43% (43 of 100). Not profiling.
7: 35% (35 of 100). Not profiling.
8: 28% (28 of 100). Not profiling.

No busy CPUs found.

 

The profiling will start if any CPU goes above 50%, and the kernel outputs will be displayed. More than one CPU can be profiled simultaneously.

 

diag sys profile report
CPU Kernel Percentages:
0: 27% (28 of 101). Not profiling.
1: 25% (25 of 99). Not profiling.
2: 52% (52 of 100). Will profile.
3: 28% (29 of 101). Not profiling.
4: 36% (36 of 100). Not profiling.
5: 37% (37 of 99). Not profiling.
6: 28% (28 of 100). Not profiling.
7: 26% (27 of 102). Not profiling.
8: 20% (21 of 101). Not profiling.
Kernel=0xffffffff80200000-0xffffffff80c021d1, module-gap=0x1f3fde2f
Profile buffer: profile step=4, sz=15741624-7870812, last-addr=0xffffffffa140439f
warning: functions cannot be profiled properly, __entry_SYSCALL_64_trampoline-pGDT32
warning: functions cannot be profiled properly, early_idt_handler_array-br_deinit
diagnose vpn tunnel stat
0xffffffff809e0624: 54 cpu_idle_poll.isra.0+0x24/0x60
0xffffffffa013e78f: 13 np7_hpe_exit+0x2cf/0x340
0xffffffffa000bde7: 12 ip_session_ha_scan_walker+0x657/0x8a0
0xffffffffa0016fbb: 8 ip_session_tree_walker+0x1e8b/0x2760
0xffffffffa0140ddf: 3 np7_hw_dsw_init+0x3af/0x520
0xffffffff802f8764: 3 __local_bh_enable_ip+0x34/0x60
0xffffffffa0140fbb: 2 np7_hhtbl_free+0x6b/0x300
0xffffffffa00e9bdb: 2 ip6_session_tree_walker+0x42b/0x2280
0xffffffff80876c60: 1 neigh_resolve_output.part.0+0x0/0x100
0xffffffff809e000c: 1 schedule_hrtimeout_range_clock+0x9c/0x130
0xffffffffa003d157: 1 detect_unknown_spi6+0x397/0x7f0

 

These outputs will be useful for the support and development team to understand the reason of the CPU usage.

Contributors