FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Nivedha
Staff
Staff
Article Id 269685
Description This article describes how to collect logs when FortiGate is in conserve mode due to IPS Engine or WAD
Scope FortiGate v6.4 and above.
Solution

Conserve mode is triggered when memory consumption reaches the red level and traffic starts dropping when memory consumption reaches an extreme level.

Check the following references to understand how the conserve mode is triggered:

 

Technical Tip: How conserve mode is triggered
Technical Tip: Conserve mode changes in FortiGate 5.6 and above

 

  • Run diag sys top 1 99 to check if IPSEngine  or WAD is consuming a lot of memory.

IF IPS Engine consumes a lot of memory :

  • The second column lists the process id of the IPS Engine. Make a note of the process ID.
  • Further, collect the following logs and open a TAC case for further troubleshooting.

 fnsysctl df -h

 

Find the process ID of the IPS engine daemon, then run these commands:

 

fnsysctl cat /proc/[process id]/status

fnsysctl cat /proc/[process id]/maps

fnsysctl cat /proc/[process id]/smaps

 

Along with this also collect the following debugs:

get sys performance status

diagnose hardware sysinfo memory

diagnose sys session full-stat

diagnose ips session status

diagnose ips packet status

diagnose ips memory status

diagnose ips memory track-glib

diag sys top-mem 50

diag sys top 1 99 <----- Let  this run for 10 seconds and then press q to exit.
diag sys top-fd 50

diagnose test application ipsmonitor 3

diagnose test application ipsmonitor 14

diagnose test application ipsmonitor 15

diagnose test application ipsmonitor 24

diag sys top-sockmem

fnsysctl df

fnsysctl ls -al /tmp

fnsysctl ls -al /dev/shm

 

IF WAD consumes a lot of memory:

Collect the output of these commands during the issue:


get system status
diag hardware sysinfo memory
diag hardware sysinfo slab
get sys perf status
diag sys session stat
diag sys top-mem 50
diag sys vd list | grep fib
diag sys top-fd 30
diagnose sys mpstat 1 5 <----- Wait for 5 seconds and then press q to exit
diag sys top-all 2 30 <----- Wait for 5 seconds and then press q to exit.
fnsysctl df -k
fnsysctl ls -l /tmp
fnsysctl du -i /tmp
fnsysctl du -a /tmp
fnsysctl du -a / -d 1
fnsysctl du -i /dev/shm
fnsysctl du -a /dev/shm
fnsysctl du -i /node-scripts
fnsysctl du -a /node-scripts
diag debug reset
diag debug enable
diag test application wad 1000
diag wad stats worker
diag wad stats worker.sysmem
diag test application wad 803
diag test application wad 2
diag test application wad 13
diag test application wad 21
diag test application wad 70
diag test application wad 103
diag test application wad 104
diag test application wad 105
diag test application wad 112
diag test application wad 113
diag test application wad 114
diag test application wad 117
diag test application wad 120
diag test application wad 123
diag test application wad 130
diag test application wad 132
diag test application wad 156
diag test application wad 157
diag test application wad 158
diag deb disable

Contributors