Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
fox
New Contributor

Fortigate VM esxi high CPU usage

Hi, when I enable DPDK, the CPU always 100% usage, even I enable sleep-on-idle, still one core was 100%.

This was normal? Are there any setting can reduce the CPU usage?

config:
config dpdk global
set status enable
set interface "port1" "port2" "port3"
set multiqueue enable
set sleep-on-idle enable
set elasticbuffer enable
set per-session-accounting traffic-log-only
set ipsec-offload disable
set hugepage-percentage 30
set mbufpool-percentage 25
end

config dpdk cpus
set rx-cpus "all"
set vnp-cpus "all"
set ips-cpus "all"
set tx-cpus "all"
end

log:
diagnose dpdk performance show
----------------------------------------
CPU usages
----------------------------------------
Average Engine 0 Engine 1
2023:04:10 16:58:29 rx: 0.0 0.1 0.0
2023:04:10 16:58:29 vnp: 0.3 0.0 0.6
2023:04:10 16:58:29 ips: 0.0 0.0 0.0
2023:04:10 16:58:29 tx: 0.0 0.0 0.0
2023:04:10 16:58:29 idle: 99.6 99.8 99.4

diagnose sys mpstat 2 3
Gathering data, wait 2 sec, press any key to quit.
..0..1
TIME CPU %usr %nice %sys %iowait %irq %soft %steal %idle
04:57:50 PM all 36.66 0.00 13.97 0.00 0.00 0.25 0.00 49.13
0 73.13 0.00 26.87 0.00 0.00 0.00 0.00 0.00
1 0.00 0.00 1.00 0.00 0.00 0.50 0.00 98.51

TIME CPU %usr %nice %sys %iowait %irq %soft %steal %idle
04:57:52 PM all 38.44 0.00 12.06 0.00 0.00 0.00 0.00 49.50
0 76.00 0.00 24.00 0.00 0.00 0.00 0.00 0.00
1 0.51 0.00 0.00 0.00 0.00 0.00 0.00 99.49

TIME CPU %usr %nice %sys %iowait %irq %soft %steal %idle
04:57:54 PM all 37.22 0.00 13.40 0.00 0.25 0.00 0.00 49.13
0 73.00 0.00 27.00 0.00 0.00 0.00 0.00 0.00
1 1.98 0.00 0.00 0.00 0.00 0.00 0.00 98.02

6 REPLIES 6
rosatechnocrat
Contributor

Enabling DPDK in polling mode results in high CPU usage.

Regards
ROSA Technocrat - Also on youtube




Please Subscribe
kvimaladevi
Staff
Staff

Hi Fox,

 

DPDK is enabled on the Firewall which does the load balancing. To understand when process is utilizing high CPU, please provide the below outputs:


diag dpdk performance show
diag sys top-summary
diag sys top 1 20

Also, please check if you are seeing any crash logs in the diag debug crashlog read output.

Regards,

Vimala

fox

Hi Vimala, thanks for repling, please check below log;

 

diag dpdk performance show
----------------------------------------
CPU usages
----------------------------------------
Average Engine 0 Engine 1
2023:04:11 20:52:56 rx: 0.0 0.1 0.0
2023:04:11 20:52:56 vnp: 0.3 0.1 0.4
2023:04:11 20:52:56 ips: 0.0 0.0 0.0
2023:04:11 20:52:56 tx: 0.0 0.0 0.0
2023:04:11 20:52:56 idle: 99.6 99.7 99.6
-----------------------------------------

diag sys top 1 20

Run Time: 29 days, 21 hours and 34 minutes
40U, 0N, 10S, 50I, 0WA, 0HI, 0SI, 0ST; 3921T, 544F
ipsengine 3845 R < 99.9 7.5 0
ipsengine 3846 S < 0.0 7.5 1
node 3619 S 0.0 3.1 1
ipshelper 3678 S < 0.0 2.7 0
cmdbsvr 3577 S 0.0 1.8 0
scanunitd 128744 S < 0.0 1.5 1
scanunitd 130583 S < 0.0 1.5 1
wad 3709 S 0.0 1.4 1
wad 3708 S 0.0 1.4 1
miglogd 3637 S 0.0 1.3 1
reportd 3639 S 0.0 1.2 1
cid 3666 S 0.0 1.2 1
miglogd 3783 S 0.0 1.1 1
cw_acd 3687 S 0.0 1.1 1
forticron 3628 S 0.0 1.0 1
scanunitd 3642 S < 0.0 1.0 1
csfd 3690 S 0.0 1.0 1
wad 3698 S 0.0 0.9 1
forticldd 3629 S 0.0 0.9 1
fgfmd 3686 S 0.0 0.9 1

 

fox
New Contributor

thanks for the repling, please see the log:

diag dpdk performance show
----------------------------------------
CPU usages
----------------------------------------
Average Engine 0 Engine 1
2023:04:11 22:20:03 rx: 0.9 1.6 0.1
2023:04:11 22:20:03 vnp: 2.4 0.1 4.7
2023:04:11 22:20:03 ips: 0.0 0.0 0.0
2023:04:11 22:20:03 tx: 0.7 1.0 0.4
2023:04:11 22:20:03 idle: 96.0 97.2 94.8
-----------------------------------------

diag sys top 1 20

Run Time: 29 days, 23 hours and 1 minutes
30U, 0N, 21S, 47I, 0WA, 0HI, 2SI, 0ST; 3921T, 541F
ipsengine 3845 R < 99.0 7.5 0
ipsengine 3846 R < 3.0 7.5 1
node 3619 S 1.0 3.0 1
ipshelper 3678 S < 1.0 2.7 0
cmdbsvr 3577 S 0.0 1.8 0
scanunitd 128744 S < 0.0 1.5 1
scanunitd 130583 S < 0.0 1.5 1
wad 3709 S 0.0 1.4 1
wad 3708 S 0.0 1.4 1
miglogd 3637 S 0.0 1.3 1
reportd 3639 S 0.0 1.2 1
cid 3666 S 0.0 1.2 1
miglogd 3783 S 0.0 1.1 1
cw_acd 3687 S 0.0 1.1 1
forticron 3628 S 0.0 1.0 1
scanunitd 3642 S < 0.0 1.0 1
csfd 3690 S 0.0 1.0 1
wad 3698 S 0.0 0.9 1
forticldd 3629 S 0.0 0.9 1
fgfmd 3686 S 0.0 0.9 1

 

rosatechnocrat

Seems issue is caused by ipsengine. 

 

ipsengine 3845 R < 99.0 7.5 0

 

what is the software version. Provide output of get system status

Regards
ROSA Technocrat - Also on youtube




Please Subscribe
fox
New Contributor

V7.2.4

 

get system status
Version: FortiGate-VM64 v7.2.4,build1396,230131 (GA.F)
Virus-DB: 91.01261(2023-03-09 03:26)
Extended DB: 91.01261(2023-03-09 03:25)
Extreme DB: 1.00000(2018-04-09 18:07)
AV AI/ML Model: 0.00000(2001-01-01 00:00)
IPS-DB: 23.00508(2023-03-08 01:36)
IPS-ETDB: 23.00508(2023-03-08 01:36)
APP-DB: 6.00741(2015-12-01 02:30)
INDUSTRIAL-DB: 6.00741(2015-12-01 02:30)
IPS Malicious URL Database: 1.00001(2015-01-01 01:01)
IoT-Detect: 0.00000(2022-08-17 17:31)
Serial-Number:
License Status:
VM Resources: 2 CPU/2 allowed, 3921 MB RAM
Log hard disk: Available
Hostname:
Private Encryption: Disable
Operation Mode: NAT
Current virtual domain: root
Max number of virtual domains: 2
Virtual domains status: 1 in NAT mode, 0 in TP mode
Virtual domain configuration: disable
FIPS-CC mode: disable
Current HA mode: standalone
Branch point: 1396
Release Version Information: GA
FortiOS x86-64: Yes
System time: Wed Apr 12 16:56:30 2023
Last reboot reason: warm reboot

Top Kudoed Authors