Technical Tip: IPS memory optimization steps

Description

This article describes how to optimize the system when a high memory issue occurs with the IPS process.

Scope

FortiGate.

Solution

Adjust the following settings, where:

• 'set socket-size' is the IPS socket buffer size. The max and default values depend on available memory. A lower value reduces memory usage. However, if set too low, it may cause the IPS engine to go into a fail-open state. (Default = <256>.)

• 'set engine-count' is the number of IPS engines running. With the default value of 0, FortiOS sets the number to optimize performance depending on the number of CPU cores. Reduce it in small increments, and monitor the CPU usage per core; the fewer IPS engines spawned, the more load will be focused on fewer cores. Find the balance between Memory and CPU usage.

• 'set database': 'regular' protects against the latest common and in-the-wild attacks, whereas 'extended' includes protection from legacy attacks.

config ips global
    set socket-size [integer, 0-512] 
    set engine-count [integer, 0-255] 
    set database [regular|extended] 
end 

After changing the engine, database, and socket size, restart the IPSEngine using the following command:

diagnose test application ipsmonitor 99  

Example: In the following example, engine count is set to 2:

config ips global
    set socket-size 256
    set engine-count 2
    set database extended
end

This will cause FortiOS to reduce the number of processes for scan engines and limit the number of processes spawned by IPS. These commands need to be run on the primary member only; this change will be replicated to the secondary member. Performing the activity of optimizing the IPS engine and restarting it will terminate IPS-related sessions, so care should be taken to run them during off-peak hours.

To verify if the changes are applied or not, run the following command:

diagnose sys top

Press CTRL + C to exit diagnostics.

 
Verify if the IPS engine process is running more than 2:

diagnose sys top-mem

Verify overall memory usage on the FortiGate:

get system performance status 

Note:

Multiple CLI tools can be used to investigate IPS engine memory and CPU usage, which are useful when troubleshooting such a scenario:

diagnose sys session stat
diagnose ips session status
diagnose ips session performance
diagnose ips dissector status
diagnose ips packet status
diagnose ips session list
diagnose ips ssl status
diagnose ips memory status

One way to troubleshoot memory leaks by the IPS engine or as a step to improve IPS engine memory usage is to disable hardware acceleration for the IPS engine:

config ips global
    set cp-accel-mode none
    set np-accel-mode none
end

Note:

If FortiOS v7.4.x is in use and high memory usage spikes are observed immediately after IPS signature updates, review the installed IPS engine version. Certain signature or engine updates may increase memory consumption depending on the model and available RAM.

diagnose autoupdate versions
diagnose ips memory status

Related articles:

• Technical Tip: Changing the IPS database

• Technical Tip: How to manually upgrade the IPS Engine

• Technical Tip: Reduce memory usage by reducing the number of spawned daemons