Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
gcraenen
New Contributor

Several problems high memory and cpu usage blocking WAN connection after upgrade to 6.2

Hi,

 

After upgrading from 6.0.4 to 6.2 I have problems with WAN connectivity falling out. I'm getting this message in de Fortios GUI:

 

Conserve mode activated due to high memory usage

 

I have tried to downgrade to 6.0.4 but can't with the error message that it failed beacause it cannot download the file from fortiguard. Help.

5 Solutions
SMabille
Contributor

Hi,

 

Download the image from the Fortinet support website and upload/apply it from your browser.

 

Best regards,

Stephane

View solution in original post

Dave_Hall
Honored Contributor

This is by no means a fix, but a work-around is to have the fgt perform a daily reboot. 

 

config system global set daily-restart enable set restart-time <time value> end

NSE4/FMG-VM64/FortiAnalyzer-VM/6.0 (FWF30E/FW92D/FGT200D/FGT101E/FGT81E)/ FAP220B/221C

View solution in original post

NSE4/FMG-VM64/FortiAnalyzer-VM/6.0 (FWF30E/FW92D/FGT200D/FGT101E/FGT81E)/ FAP220B/221C
Adrian_Lewis

Vinicius wrote:

Do you know when will the 6.2.2 be released?

No specific dates are available only targeted dates which can slip if any issues are identified. Currently target though I hear is this week.

View solution in original post

ISOffice

Hi all,

 

We upgraded our 100D appliances to 6.2.2 a week ago and noticed a slight improvement in GUI performance when viewing logs in Log & Report. However, when filters were applied the CPU once again spiked to 90+% with multiple instances of the 'log_se' process running.

I have an ongoing support call logged with Fortinet and their TAC Engineer (cheers Kevin!!) suggested that I fail over to the 'slave' appliance to see if the issue could be replicated there. To my surprise, it wasn't. The GUI was responsive and I could apply filters to viewing logs and the CPU didn't spike. The only difference between the appliances was that the 'master' appliance had used 75% of it's disk for log storage and the 'slave' appliance only 2% (we are running active-passive). When viewing logs (with filters applied) on the 'slave' appliance the 'log_se' process didn't even appear when I ran 'diag sys top'.

Long story short, on failing back to the 'master' appliance I deleted all logs on the local disk (execute log delete-all) and can now view logs, with multiple filters, quite easily and without overly stressing the CPU. We syslog traffic logs in real-time to a third party product, so it isn't critical for us to retain logs on the appliance's hard disk for extended periods of time.

Querying local disk logs, particularly if there are many of them, possibly results in high CPU usage due to the high i/o requirements to do so.

Whilst this may not be relevant to many others in this discussion, I thought I would share our experience anyway.

 

Best regards,

 

John P

View solution in original post

tanr
Valued Contributor II

@gcraenen, did you open a support ticket with TAC?  I known they've fixed a number of bugs between 6.2.0 and 6.2.3, some specific to ipsengine, but if your specific issue and repro case hasn't been reported them then it's unlikely to have gotten fixed.

 

Not saying that 6.2.3 is stable enough, though!  We usually wait till the .4 releases to start testing them for possible production use.  I'm reasonably hopeful

View solution in original post

79 REPLIES 79
ISOffice

Hi all,

 

We upgraded our 100D appliances to 6.2.2 a week ago and noticed a slight improvement in GUI performance when viewing logs in Log & Report. However, when filters were applied the CPU once again spiked to 90+% with multiple instances of the 'log_se' process running.

I have an ongoing support call logged with Fortinet and their TAC Engineer (cheers Kevin!!) suggested that I fail over to the 'slave' appliance to see if the issue could be replicated there. To my surprise, it wasn't. The GUI was responsive and I could apply filters to viewing logs and the CPU didn't spike. The only difference between the appliances was that the 'master' appliance had used 75% of it's disk for log storage and the 'slave' appliance only 2% (we are running active-passive). When viewing logs (with filters applied) on the 'slave' appliance the 'log_se' process didn't even appear when I ran 'diag sys top'.

Long story short, on failing back to the 'master' appliance I deleted all logs on the local disk (execute log delete-all) and can now view logs, with multiple filters, quite easily and without overly stressing the CPU. We syslog traffic logs in real-time to a third party product, so it isn't critical for us to retain logs on the appliance's hard disk for extended periods of time.

Querying local disk logs, particularly if there are many of them, possibly results in high CPU usage due to the high i/o requirements to do so.

Whilst this may not be relevant to many others in this discussion, I thought I would share our experience anyway.

 

Best regards,

 

John P

Aron1
New Contributor

Vinicius wrote:

Hello,

Anyone having this issue on 6.2.1?

Every week i must restart those cause they run into conserve mode.

Already set-memory-threshold-green to 89 and red to 90.

 

I'm running 6.2.1 on a 30E and on 60E.

 

I have a client with 12 forti FW's. Upgraded 4 of them to 6.2.1. 2 were 5.6.7 up to 6.2.1, the other 2 6.2.0 to 6.2.1. The 2 that were running 6.2.0 and having memory issues no longer are having memory issues. On those two, the applincation that was causing the mem issue was wad on one unit and ipsmonitor on the other. Restarting those apps with the diag test app **** 99 command is only a temporary fix from what I have seen.

 

While they are not seeing the memory issue again, I ran into an SSL inspection issue with 6.2.1 on an online application they are running. I may be opening a Case with Fortinet on that one soon.

Ignotum per ignotius...

Ignotum per ignotius...
FortiET
New Contributor

We have Fortigate 200 E  - cluster.  Today I ran into the Conserve mode.  diagnose test application wad 99 fixed the problem at once.  But Fortinet - any updates about fix for this problem ?

zaphod
New Contributor III

FortiET wrote:

We have Fortigate 200 E  - cluster.  Today I ran into the Conserve mode.  diagnose test application wad 99 fixed the problem at once.  But Fortinet - any updates about fix for this problem ?

have you updated to 6.2.1 ?

 

FortiET
New Contributor

Sorry, Yes, I forgot to write our version. We have run 6.2.1  Build 0932 for quite some weeks.

 

scan
New Contributor

Hi

 

Did you check why you Fortigate is in the conserve Mode?

Here some ideas to figure it out why your Fortigate is in conserve Mode:

diagnose sys top-summary #Check which process use a lot of memory oder cpu

diagnose test application <Application-Name> #Restart Application

 

In the most situation a kill of the process do not solve the issue. Often you get other issues sometimes a couple of days or weeks later!

 

dogan_md
New Contributor

I'm using 90E, I had memory problems in version 6.2, then I switched to 6.2.1. But I'm having the same problem even more often. I couldn't restart the device from the menu when this last problem occurred, I had to unplug the power cord and plug it again.

aroch
New Contributor

Hello,

using 200E Cluster with 6.2.1, uptime 36 days and today we had the same issue.

diagnose test application wad 99 fixed it for the first time.

I have seen that 6.2.2 is out.

Can anyone confirm the problem is fixed with this ?

BR

gweeby
New Contributor

Hello,

 

unfortunately we still have the memory conserve problem in 6.2.2

thuynh_FTNT

Hi gweeby, can you provide more details about your environment? You can pm me if needed.

- What is your topology and traffic load?

get system status

get system session-info full-stat

get system session-info statistics

 

- Can you provide the output of the following commands? Which process is using up the memory?

get sys perf stat

get sys perf top

diagnose sys top

diagnose sys top-summary "-s mem"

 

- Get more ips debug if it's confirmed the process is IPS

diagnose ips memory status

diagnose ips session list by-mem 10

diagnose ips session status

diagnose ips packet status 

 

- Any relevant crash log?

diagnose  debug crashlog read

 

- Does this command fix your issue?

diagnose test application wad 99 

Thanks,

Tri

Top Kudoed Authors