Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
Miata
New Contributor II

High CPU and Memory Usage

Hi guys

 

So my FG-60D running 5.2.3 has been at 100% CPU and about 90% memory recently so I thought I would run the diag sys top command as shown below. 

 

From this command I can see that the scanunitd and IPS engine it taking most of my CPU usage. I don't have vulnerability scanner but I have AV enabled on 17 different policies. I think the box is being overworked, but can I restart any processes or do you guys have any other advice?

 

Run Time: 42 days, 19 hours and 54 minutes 62U, 0N, 37S, 1I; 439T, 40F, 189KF scanunitd 7079 R < 68.4 3.7 ipsengine 602 S < 19.2 13.1 httpsd 7717 S 2.3 4.2 httpsd 7718 S 1.9 4.2 httpsd 7737 S 1.7 4.2

 

I also ran get sys performance - Output below

 

CPU states: 75% user 25% system 0% nice 0% idle CPU0 states: 75% user 25% system 0% nice 0% idle Memory states: 93% used Average network usage: 6282 kbps in 1 minute, 2754 kbps in 10 minutes, 2200 kbps in 30 minutes Average sessions: 1995 sessions in 1 minute, 2178 sessions in 10 minutes, 1824 sessions in 30 minutes

 

If you have any form of advice in terms of how to manage this more successfully or anything to restart/kill then please let me know, would be greatly appreciated.

 

Kind regards

Miata

1 Solution
frajico

Finally, we realized that some interfaces of Fortigate unit that were configured as trunk interfaces (multiple vlans), were receiving more traffic than they have to (have to receive only 1 vlan traffic, and was receiving 10 vlan traffic), so interface got oversubscribed and CPU of Fortigate raised almos al 100%. Allowing only the 1 vlan on the switch, solved the issue.

Check for overloaded / oversubscribed interfaces traffic.

View solution in original post

31 REPLIES 31
BarryM

Update:

The latest firmware, 5.2.10,  fixed the CPU utilization problem. My 300C is running optimally now, as it was before some of the last firmware updates. It was bugs in the firmware that were the cause of the problems. My unit is nowhere near maxed out in features and settings. The CPU barely spikes now.

 

 

MikePruett
Valued Contributor

Probably had memory leaks and misc other things causing that. Good that Fortinet fixed it for you!

Mike Pruett Fortinet GURU | Fortinet Training Videos
jpastor
New Contributor

Newest versions need more CPU and more memory. D family souldn´t be affected in performance by an upgrade to 5.4 versions. Open a ticket if it´s your case.

 

virtualj

Hello,

I'm also interested in this function to test the traffic between two fortigates. I see two interesting thigs:

the command "diag traffictest run" is like a script. It launchs two times the iperf, as you can see with this:

FW1_xxx_xx # diagnose traffictest show server-intf:    wan1 client-intf:    wan2 port:   9999 proto:  TCP

FW1_xxx_xx # diagnose traffictest run (running in another CLI)

FW1_xxx_xx # fnsysctl ps 8931      0       0       R       /bin/iperf -s -B xx.xx.xx.xx -m root -p 9999 8932      0       0       R       /bin/iperf -c xx.xx.xx.xx -B yy.yy.yy.yy -m root -p 9999

 

You can see also that there is a listening connection on port 9999:

FW1_xxx_xx # diagnose sys tcpsock | grep 9999 xx.xx.xx.xx:9999->0.0.0.0:0->state=listen err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:5241->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:19686->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=774144 tma=0 yy.yy.yy.yy:19686->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=133120 fma=309248 tma=0 yy.yy.yy.yy:5241->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0

 

I tried to do the same command on another fortigate with -c xx.xx.xx.xx so it should connect to the other fortigate iperf. Diag sniffer confirm that traffic arrives to the server fortigate, but the syns are dropped. Debug flow shows "iprope_check_failed", like when you are trying to manage the firewall, but don't have trusted hosts or management service enabled.

I tried also a firewall local-in policy to accept anything with no result.

 

Some other ideas? :)

NSE 7

NSE 7
Paul_S

virtualj wrote:

Hello,

I'm also interested in this function to test the traffic between two fortigates. I see two interesting thigs:

the command "diag traffictest run" is like a script. It launchs two times the iperf, as you can see with this:

FW1_xxx_xx # diagnose traffictest show server-intf:    wan1 client-intf:    wan2 port:   9999 proto:  TCP

FW1_xxx_xx # diagnose traffictest run (running in another CLI)

FW1_xxx_xx # fnsysctl ps 8931      0       0       R       /bin/iperf -s -B xx.xx.xx.xx -m root -p 9999 8932      0       0       R       /bin/iperf -c xx.xx.xx.xx -B yy.yy.yy.yy -m root -p 9999

 

You can see also that there is a listening connection on port 9999:

FW1_xxx_xx # diagnose sys tcpsock | grep 9999 xx.xx.xx.xx:9999->0.0.0.0:0->state=listen err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:5241->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:19686->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=774144 tma=0 yy.yy.yy.yy:19686->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=133120 fma=309248 tma=0 yy.yy.yy.yy:5241->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0

 

I tried to do the same command on another fortigate with -c xx.xx.xx.xx so it should connect to the other fortigate iperf. Diag sniffer confirm that traffic arrives to the server fortigate, but the syns are dropped. Debug flow shows "iprope_check_failed", like when you are trying to manage the firewall, but don't have trusted hosts or management service enabled.

I tried also a firewall local-in policy to accept anything with no result.

 

Some other ideas? :)

according to the forum thread here: https://forum.fortinet.com/tm.aspx?m=146386

you cannot run traffictest in server mode on a fortigate. connect to a computer and try the -r flag if you want to have the traffic flow to the fortigate instead of from it.

FG200D 5.6.5 (HA) - primary [size="1"]FWF50B' s 4.3.x, FG60D's 5.2.x, FG60E's 5.4.x                   [Did my post help you? Please rate my post.][/size] FAZ-VM 5.6.5  |  Fortimail 5.3.11 Network+, Security+

FG200D 5.6.5 (HA) - primary [size="1"]FWF50B' s 4.3.x, FG60D's 5.2.x, FG60E's 5.4.x [Did my post help you? Please rate my post.][/size] FAZ-VM 5.6.5 | Fortimail 5.3.11 Network+, Security+
virtualj

Sorry Paul, I want to answer to that thread!

NSE 7

NSE 7
Qhamani_Zendile

Good Day All

 

Client of mine experienced a spike of 99% in CPU Usage on a Fortigate 200E Model. The spike was due to High CPU Usage on the ipsengine process. I restarted the process via CLI and it seemed to resolve the issue. What could possibly be causing the spike on the ipsengine process and how can be prevented from happening again?

boneyard

next time just start a new question, lots of answer about specific models or versions.

 

if the IPS process is high on CPU then probably it is not handling certain traffic well. it might be something very specific that doesn't happen often or just this once. if it happens again raise a ticket with Fortinet support and provide the information they ask for, version, some debugs, ... they might be able to trace where it went wrong.

Qhamani_Zendile

Thank you Boneyard

 

Noted. Much appreciated for the feedback.

SCSIraidGURU
Contributor

I have worked with Cisco, Fortinet, Checkpoint and other firewalls over 35 years.  17 policies on a 60D?   Seems like overkill.   Can you consolidate the policies and processes down?     I would not try that on my 800C in my data center.   I limited policies to default that covers 85% of users.  A custom policy for 10% of the users.  A third policy for 5% of users.   With most firewalls.  You want to create policies to cover most of your users.    Can you give more details on why you have so many policies on a 60D? 

Announcements

Select Forum Responses to become Knowledge Articles!

Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.

Labels
Top Kudoed Authors