Technical Tip: How to optimize memory consumption for smaller FortiGates

Description

This article describes the typical steps involved to optimize memory consumption on desktop FortiGate models, and some 1RU models of FortiGate (smaller than 200F).
Every enabled feature on the FortiGate will consume memory. Models with lower amounts of available system memory and that have many features enabled may not have enough working memory for typical firewall tasks.

Scope

FortiGate models with 4GB system memory or less.

Solution

High baseline memory use on a device might be expected, particularly depending on the hardware platform. Total system memory is visible using the 'get hardware memory' command.
The commands below should be used under the 'config system global' in the multi-VDOM environment. 

get hardware memory | grep MemTotal
MemTotal: 1963668 kB

get system performance status | grep Memory -A 1
Memory: 1963668k total, 1207888k used (61.5%), 562756k free (28.7%), 193024k freeable (9.8%)
Average network usage: 39 / 37 kbps in 1 minute, 39 / 36 kbps in 10 minutes, 41 / 165 kbps in 30 minutes

If memory used is 60% or more during nominal traffic use, or the system experiences frequent conserve mode, consider the following optimization steps:

1. Disable features that are not required (for example, DHCP, Reporting, Logging, etc).
2. Review the site's policies and enable only UTM features required by the security posture (example features that may not be required in all environments: Antivirus, Web Filter, IPS, Application Control, DNS Filter, and SSL Deep Packet Inspection).
3. Do not use UTM scanning for trusted traffic (like Server <-> Storage).
4. Fine-tune IPS signatures applied (for example, disable Linux/Mac signatures if only Windows is used).
5. Tweak the IPS engine and profiles when necessary:
   
   

config ips settings
config ips sensor

6. Adjust the antivirus engine and profile settings when necessary:
   

config antivirus profile

config antivirus settings

Adjust IPS process count:

config ips global
    set engine-count 2      <----- Set it to the number of cores the unit has available, or less.

    set cp-accel-mode none
    unset exclude-signatures <----- If the OT database is not needed.

       set database regular
    set av-mem-limit 20

end

Note:

This configuration should be implemented during a maintenance window and carefully monitored during production hours for any performance impact.

Each daemon is bound to a single CPU core. Since the number of daemons/processes is reduced, depending on the volume of traffic or number of requests handled by a single daemon, this can lead to high CPU usage on that core, potentially resulting in dropped connections. It is recommended to monitor network usage, CPU load, and memory consumption and adjust the configuration accordingly.

For example, if the device is handling around 100-200 Mbps of traffic (as shown in the output of 'get system performance status' under average network usage), allocating more than one WAD worker or IPS engine may be necessary. This should be done while ensuring efficient memory utilization so that the device does not enter conserve mode.

7. Schedule FortiGuard updates outside of business operating hours:
   
   

config system autoupdate schedule
    set frequency daily
    set time 03:00
end

8. Disable dashboard widgets with dynamic content for ALL local users. The reason for doing this is that when the user logs in on the GUI, they will see the historical data in the widget (activity in the last 24 hours, etc). This means the widgets are always working in the background for all local users and consuming CPU/MEM resources.
   
   
9. Disable on-device logging/reporting. Instead, log externally to FortiManager, FortiAnalyzer, FortiCloud, and Syslog.
   FortiGates can store logs in different locations (memory, disk, external logging servers like FortiAnalyzer, or Syslog). When logging to memory is enabled, logs are temporarily stored in the device's RAM (volatile memory), which is useful for short-term logging. This could help in scenarios where conserving memory is important.

config log memory setting

    set status disable

end

10. Disable forward traffic logging: By disabling this setting, the FortiGate will no longer store logs related to forward traffic on its disk. This could be done to conserve disk space, reduce logging overhead, or if another logging method (for example, logging to FortiAnalyzer or Syslog) is being used for this traffic.

config log disk filter
    set forward-traffic disable

end

11. Reduce the maximum number of records in the DNS cache: Configuring the DNS cache limit to 300 restricts the maximum number of DNS records that can be stored in the cache to 300 entries.

    Reducing the cache size is beneficial in environments where memory or performance optimization is a priority, as it ensures the DNS cache remains efficiently managed, preventing excessive resource consumption.

    
    

config system dns
    set dns-cache-limit 300 <----- The default value is 5000.
end

12. Disable the submission of Security Rating results to FortiGuard and scheduled runs of Security Rating:

      
config system global
    set security-rating-result-submission disable
    set security-rating-run-on-schedule disable
end

Note: Starting from FortiOS v7.4.4, the CLI setting 'set security-rating-result-submission' was removed under 'config system global'. Refer to FortiOS v7.4.4 Release Notes: Changes in CLI.

13. Reduce internet-service-database: By configuring the setting to 'on-demand', the FortiGate will load the Internet Service Database (ISDB) only when required, instead of keeping it continuously loaded in memory. This approach optimizes system resource usage, particularly in environments where the ISDB is not frequently utilized.
    

     

config sys global
    set internet-service-database on-demand
end

     execute update-ffdb-on-demand

See the following article for more information on the above setting: Technical Tip: Internet-service-database: On-demand.

Advanced steps to optimize memory use:

If device memory use is low during periods of low traffic but increases significantly with use, it may be desirable to further tune memory consumption.

Note: Care should be taken with the following changes as they affect the overall behavior of the system and are not appropriate for all environments.

1. Reduce the maximum file size for inspection:
   
   

config firewall profile-protocol-options

    edit <profile>

        config <service>

            set oversize-limit <maximum size>

        end

    next
end

2. Reduce the FortiGuard TTL cache:
   
   

config system fortiguard
    set webfilter-cache-ttl 600     <----- The default value is 3600.
    set antispam-cache-ttl 600      <----- The default value is 1800.

end

3. Reduce DNS cache size:
   
   

config system dns

    set dns-cache-ttl 600           <----- The default value is 1800.

end

4. Reduce TCP session timeouts:
   
   

config system global
    set tcp-halfopen-timer    <----- Enter an integer value from <1> to <86400> (default = <120>).

end

5. Reduce global session TTL:
   
   

config system session-ttl

    set default 900

end

For more granular session-ttl settings based on matched firewall policy or service, see Technical Tip: Session timeout settings.

6. Reduce firewall policy TTL & Service/port TTL to improve session recycling efficiency: (config firewall policy, config firewall address).
   
   

config system session-ttl
    set default 900
        config port
            edit 1
                set protocol 17
                set timeout 120
            next
        end
end

7. Reduce logging to only important events (config log setting).

    

8. Reduce worker count:
   

config system global
    set miglogd-children 1
    set sslvpn-max-worker-count 1
    set wad-worker-count 2
    set scanunit-count 2   <----- scanunit-count value must be in the range of 2-4 or = 0.
end

Note: The setting 'set sslvpn-max-worker-count' is only available on FortiGate models where SSL VPN/Agentless VPN is supported. On platforms where SSL VPN tunnel mode and web mode (Agentless VPN) are not supported, this command is not applicable. For details and affected models, refer to: Technical Tip: Changes on SSL VPN modes starting from v7.6.3.

Refer to the following article for more information: Technical Tip: Reduce memory usage by reducing the number of spawned daemons.

9. Optimize dropped traffic session offload:
   
   

config system settings
    set ses-denied-traffic enable
end

10. Adjust the IPS setting that affects memory consumption.

config ips global     

    set socket-size [integer, 0-512, 0-128]  <----- IPS socket buffer size. The maximum and default values depend on available memory. A lower value reduces memory usage at the cost of CPU.

    set database     [regular|extended] <----- Regular protects against the latest common and in-the-wild attacks. Extended includes protection from legacy attacks.

end

diagnose test application ipsmonitor 99 <----- After changing the engine, database, and socket size, restart the IPSEngine.

Note 1:

If a device is undersized for the required feature-set or traffic volume, memory optimization will have only a very limited benefit. If the steps above do not produce satisfactory results, consider using a higher-capacity FortiGate.

Note 2:

If memory optimization changes are made via CLI, they will need to be made individually on both primary and secondary units in a High-Availability cluster configuration.

Note 3:

In v7.6.3, further optimizations were made that remove or rework certain features on devices with 2GB of memory or less.

For further details, see Optimizations for physical FortiGate devices with 2 GB RAM 7.6.3.

Note 4:

After applying the above commands, the reduction in memory usage is not very significant-about 7%. However, it continues to decrease gradually. The main purpose of this change is to ensure that FortiGate manages its processes differently, preventing future memory spikes.

Note 5:

Note that every organization has different requirements for security posture. Consult the relevant internal IT security team as necessary before implementing any of the memory optimization measures.

Note 6:

Beginning with FortiOS 7.6.5, additional mechanisms have been implemented to improve memory utilization on FortiGate models equipped with 2 GB or 4 GB of RAM. These optimizations target startup configurations, Network Processors (NPs), and NTurbo acceleration. For further information, refer to Memory optimizations for startup configurations, NPs, and NTurbo.

Related articles:

Technical Tip: Free up memory to avoid conserve mode

Technical Tip: Priority of session-ttl settings in FortiGate

Technical Tip: How to extend the TCP Half-Close timer for specific TCP services