Troubleshooting Tip: Collecting IPS engine related debugs

[Renante_Era]

Description	This article describes how to collect IPS engine debugs.
Scope	FortiGate v7.0 and above.
Solution	Show FortiGate stats and memory usage: get sys status get system performance status diagnose hardware sysinfo memory diagnose system session stat diagnose ips session list by-mem 15 diagnose ips session status diagnose autoupdate version | grep "IPS Attack" -A 6 diagnose ips memory status diagnose test application ipsmonitor 1 diagnose system top 2 30 4 diagnose system top-mem 10 diagnose system top-mem 10 diagnose system top-mem 10 If the IPS engine's memory usage appears to be higher than normal, run 'diag sys process stack $pid' (four times) to get the stack for the process. diagnose system process pstack <PID with high usage> fnsysctl cat /proc/$PID/smaps Restart the process. diagnose system kill 11 <PID> Get the crash log as well. diagnose debug crashlog read Note: Additional debugging commands for TAC. diagnose debug reset diagnose ips debug enable all <- Instead of 'all', use one from a list of categories displayed by pressing '?' after enable. diagnose ips filter set "host x.x.x.x" <- x.x.x.x = testing host IP. diagnose debug enable Disable afterwards with the following command: diagnose debug disable Note: Selecting all for the IPS debug will cause high memory usage and can lead to kernel conserve mode as this debug is copying sessions inspected by the IPS engine, which will lead to doubling the sessions. It is always recommended to specify the type of session on the debug also, it is strongly recommended to use the filtering command 'diagnose ips filter...' unless the goal of the debug is to collect events for all hosts. In cases where an IPS profile is assigned to a firewall policy in proxy inspection mode, the recommended live debug commands are: On the Filter, add several hosts, for example: diagnose ips filter set 'host x.x.x.x and host z.z.z.z' Filter parameters need be used as it was used for IPS debugging. diagnose debug reset diagnose wad debug enable category [all, http, session ,...] <----- Use question mark '?' to view all available categories. diagnose wad debug enable level verbose <- Verbosity level. When troubleshooting, it is recommended to use verbose. diagnose wad filter dst 10.10.10.10  <----- Filtering the logs based on the destination address. diagnose wad filter port 443         <----- Filtering the logs based on the destination port. diagnose debug console timestamp enable diagnose debug enable To confirm which filters are in use: diagnose wad filter list  drop unknown sessions: disabled  dest ip: 10.10.10.10-10.10.10.10 dest port: 443-443 To disable the debug: diagnose debug disable diagnose debug reset