Process IPSEngine High Memory

Hi @DaVeiga19,

You can check by adjusting the engine count and socket size.

config ips global

set engine-count 2 [integer, 0-255]
set socket-size 16 or 32 [integer, 0-512]
end

Refer link:
https://community.fortinet.com/t5/FortiGate/Technical-Tip-IPS-memory-optimization-steps/ta-p/197486

If the issue persists I would suggest to open a TAC case and share these logs for further investigation.

# Issue Timestamp
# System event logs around this time.
# Run the below commands

get system performance status (run this command 5 times in interval of 1 minutes)
diag sys top 1 40(Run for 30 Sec and CTRL C to stop)
diag debug crashlog read ---check for any crash triggered

Also collect the tac report

# execute tac report

Regards,
Joshi

Hello Daveiga19,

Thank you for reaching on the Fortinet forum.

As suggested earlier by Joshi please collect all logs and would suggest opening a ticket with TAC for log analysis further if it matches with any known issues reported.

-As of now i couldn't see any known issues on release notes for ips memory 

https://docs.fortinet.com/document/fortigate/7.2.4/fortios-release-notes/236526/known-issues

-Crashlogs and memory logs would help further identify issues.

get system performance status (run this command 5 times in intervals of 1 minute)
diag sys top 1 10(Run for 30 Sec and CTRL C to stop)
diag debug crashlog read | grep 2023---check for any crash triggered

-Can you please for the time being disable logging on firewall policy and use the below article steps to optimize memory 

Technical Tip: IPS memory optimization steps - Fortinet Community

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Reduce-memory-usage-by-reducing-the-number-of/ta-p/194322

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Free-up-memory-to-avoid-conserve-mode/ta-p/241415

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Steps-on-how-to-optimize-Memory-consumption/ta-p/192323

Best regards,

Manasa.

Hi Joshi,

Hi Manasa,

Unfortunately I am not able to extract logs since when fortigate enters conserve mode nothing works, neither access via the mgmt interface nor via the console and after the manual reboot we lose all logs and fortigate returns to its normal state.
Ipsengine processes in the normal state of fortigate consume few resources, however, in just a few minutes, a process triggers RAM consumption and fortigate crashes.
In this sense, how can I act to collect the logs?

Best Regards,

Da Veiga

Hello Manasa,

By configuring this automation stiches, if fortigate enters conserve mode will these event logs be available? Or will the event logs be sent by email as well?

Because my main problem is when it is in conserve mode, I lose all records after the manual reboot.
However, I will follow the suggested recommendations.

Best Regards

Da Veiga

Did you ever get this solved? I have the same issue run since upgrading to 7.2.5.

I have an email stitch setup to send sys top information and sometimes if the system can send the email out before it crashes I see it is the ipsengine eating up all available memory.

This happens really fast, as quick as a couple minutes from using its normal amount of memory to causing enough to go into conserve mode then eventually rebooting.

We ended up moving all of the SSL VPNs off of this system and setup another stitch to reboot the system so the standby can take over before it gets to conserve mode.

Right now we are at a crossroads because he have had nothing but bugs since 7.2.1 where every upgrade until 7.2.5 is making our system noticeably worse. At this point we are too apprehensive to update to 7.2.6 as it might cause even more problems.

Opening a TAC case doesn't do much because they ask for logs that are not logged when the system is crashing and you cannot force the error to happen as it is random.