FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
This article describes high CPU usage in the 'user' space every two hours, causing outages. The issue is triggered after upgrading to v7.6.2 and v7.6.3 and is caused by a software issue.
Scope
FortiGate v7.6.2, and v7.6.3.
Solution
Symptoms:
High CPU utilization on one or more cores by a user space.
The Ipsengine process consumes nearly 100% CPU.
In the command get system performance status, the user field shows abnormally high values.
Repeated IPS engine crashes with 'segmentation fault (signal 11)'. To verify, execute 'diag debug crashlog read' and upload it to the TAC case for confirmation.
Example output showing the condition:
FortiGate # get system performance status CPU states: 12% user 0% system 0% nice 88% idle 0% iowait 0% irq 0% softirq CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU3 states: 100% user 0% system 0% nice 0% idle 0% iowait 0% irq 0% softirq <<< 100% high CPU CPU4 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq CPU5 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU6 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU7 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
This indicates that the CPU core is fully consumed by a user process such as ipsengine.
Resolution:
The fix will be included in v7.6.5, and has already been addressed in IPS engine builds v7.4.9:0586 and v7.6.3:1151/1154. Tracked with Bug ID 1140846.
