Technical Tip: Automation stitch for conserve mode

FortiGate by default turns on conserve mode when memory consumption reaches 88%.

Read the following articles to understand better how the conserve mode is triggered:

• Technical Tip: How conserve mode is triggered.
• Technical Tip: Conserve mode changes.
  
  

It is possible to generate an automation script to collect the relevant logs when the conserve mode is triggered. This is essential for support engineers to understand which process consumes more memory.

To create an automation stitch, check the following steps:

For FortiGate v7.0, v7.2, v7.4 and v7.6.

Using the GUI:

• Navigate to Security Fabric -> Automation -> Under Stitch tab, Create New.
• Name the Stitch.
• Status: Enable
• Action execution: Sequential.
• Add Trigger -> Create -> FortiOS Event Log -> Name the event trigger and in Event select 'Memory conserve mode entered' -> OK -> select the created Trigger and Apply.
• Add Action -> Create -> CLI script -> Name the CLI script action, and in the Script field type the following
  
• For a non-VDOM-based environment:

get system status
get system performance status
diagnose sys top 5 10 5
diagnose sys top-mem 10
diagnose sys mpstat 1 2
diagnose debug crashlog read
diagnose hardware sysinfo memory
diagnose hardware sysinfo slab
diagnose hardware sysinfo shm
diagnose hardware sysinfo conserve
diagnose hardware deviceinfo disk
get system session status
diagnose sys session full-stat

• For a VDOM-based environment:

config global

get system status
get system performance status
diagnose sys top 5 10 5
diagnose sys top-mem 10
diagnose sys mpstat 1 2
diagnose debug crashlog read
diagnose hardware sysinfo memory
diagnose hardware sysinfo slab
diagnose hardware sysinfo shm
diagnose hardware sysinfo conserve
diagnose hardware deviceinfo disk
get system session status
diagnose sys session full-stat

• Select admin profile as super_admin -> OK -> Select the created Action and Apply
• Select the + sign to add another action -> Create -> Email -> Name the Email action.
• Add the email address in the 'To' field, and the preferred Subject message in the field, next change the email Body to: %%results%% -> Select 'OK'.
• Select the created email Action and Apply
• Save the automation script by selecting 'OK'.
  
  

Example:

Once saved to verify if the email alerts are working, 'right-click' over the created stitch and then select '►Test Automation Stitch'.

The CLI can also be used to test an automation stitch using the following command:

diagnose automation test <stitch-name> <log> <-- <log> will depend on the event configured to trigger automation.

It may take a couple of minutes to receive an email. If the email is not received after selecting Test Automation Stitch, refer to this article to see why the email alerts are not working.

Technical Tip: How to check why automation stitch is not working as expected

Using CLI.

config sys automation-action

    edit "Script"

        set action-type cli-script

        set script "get system status
get system performance status
diagnose sys top 5 10 5
diagnose sys top-mem 10
diagnose sys mpstat 1 2
diagnose debug crashlog read
diagnose hardware sysinfo memory
diagnose hardware sysinfo slab
diagnose hardware sysinfo shm
diagnose hardware sysinfo conserve
diagnose hardware deviceinfo disk
get system session status
diagnose sys session full-stat"

            set accprofile "super_admin"

    next

    edit "Conserve mode_email"

        set action-type email

        set email-to "<email id>"

        set email-subject "Conserve mode"

        set message "%%results%%"

    next

end

config system automation-trigger

    edit "Conserve mode trigger"

        set event-type low-memory

    next

end

config system automation-stitch

    edit "Conserve mode"

        set trigger "Conserve mode trigger"

            config actions

                edit 1

                    set action "Script"

                    set required enable

                next

                edit 2

                    set action "Conserve mode_email"

                    set required enable

                next

            end

    next

    end

Note: By design, the maximum buffer size is 16KB, so the output will not exceed this value.

FortiOS v7.2.6 and v7.4.0 and above will support a 192KB buffer limit.

Version 7.2 does not have a pre-defined 'memory conserve mode entered' event.

To create an automation stitch for these versions, select a pre-defined trigger named 'Conserve Mode'.

See the image below for this trigger:

To configure automation stitch before the unit goes into conserve mode at a specific memory usage percentage (i.e., 80%), refer to the following article: Technical Tip: Getting automation stitch before the unit goes into Conserve Mode.

Related articles:

• Technical Tip: Creating automation stitches
• Technical Tip: Execute a CLI script based on HIGH CPU
• Troubleshooting Tip: Test automation stitch is grayed out in GUI when schedule trigger is configured
• Technical Tip: 'Test Automation Stitch' button is greyed out when trying to test a newly created automation stitch
  
• Technical Tip: Execute a CLI Script based on High Memory using Automation Stitch
• Technical Tip: How to check why automation stitch is not working as expected