FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
agrakov
Staff
Staff
Article Id 226804
Description This article describes how to deal with a kernel panic.
Scope All versions of FortiOS.
Solution

A kernel panic is an issue that occurs when the kernel cannot handle operations and the system shuts down or reboots.

 

To diagnose the cause of the kernel panic, collect the information outlined by this article when it occurs and send the debug output to the support team for investigation. See the instructions below.

 

  1. Connect a Laptop or PC to the FortiGate with the console using PuTTY. See How to create a log file of a session using PuTTY for more information.
     
agrakov_0-1665945072633.png

 

  1. Log all output to a file. After, change the 'Lines of scrollback' to 999999999 (Category -> Window -> Lines of scrollback) to view the log file in its entirety.

agrakov_1-1665945072633.png

 

  1. Run the following debug commands. (Depending on feature availability, some commands may not be accepted. Ignore them.)


Important note:
On lower-end units (such as FortiWiFi 6xF/4xF models), do not enable any debug commands — connect to the console and wait for the kernel panic log to print directly on the console.

diagnose debug console timestamp enable

diagnose debug kernel level 8

diagnose debug duration 0

diagnose sys nmi-watchdog enable

diagnose debug comlog enable

diagnose debug app ipmc -1

diagnose debug enable

diagnose sys top 280

 

  1. Leave the debug commands running and periodically check the monitoring device to ensure the connection is still alive. Press the 'c' key to refresh output.

 

diag sys top should keep the session active automatically.

 

Use monitoring tools for system outages.

 

  1. Upon finding the failure, mark the date and time.
  • Check the console to see if it is responsive.
  • Press any key to refresh the diag sys top list.
  • If it fails to refresh, try pressing Ctrl + C on the keyboard to stop the command. If the system is still responsive, it will return to the CLI prompt.
  • Try holding down the NMI (applicable only for specific models). Hold the button down for 1 minute. It should output information to the console and reboot the unit automatically. If the system is completely frozen, it may not work. This step is important to try regardless.

    The NMI button is typically on the front left of the system:

 

agrakov_2-1665945072633.png

 

  • Check the LCD lights and indicate whether the LCD panel is frozen or responsive.

 

  1. After the reboot finishes and the service is restored, complete the following:

    Gather the crashlog and the debug log:

diagnose debug crashlog read

diagnose debug comlog read

 

Gather system event logs. If the system is a cluster, gather the logs from all nodes.

Attach all diagnostics to the TAC support case.|

After obtaining all necessary information, disable debug logging (optional):

 

diagnose debug comlog disable

diagnose debug comlog clear