Technical Tip: NMI Button for Troubleshooting Kernel Issues

[osoleimani]

Description

This article describes how to collect specific information regarding the kernel and CPU which help to identify the cause of system hanging or a kernel panic in FortiGate.

Scope

FortiGate.

The following models support an NMI feature:

• Low-end Models: 90E, 91E.
• Medium-end Models: 200E, 200F, 201E, 201F, 300D, 300E, 301E, 400D, 400E, 401E, 500D, 500E, 501E, 600D, 600E/601E, 600F/601F, 800D.
• High-end Models: 1000F, 1100E, 1101E, 1200D, 1500D, 1500DT, 1800F, 1801F, 2000E, 2200E, 2201E, 2500E, 2600F, 2601F, 3000D, 3000F, 3100D, 3200D, 3300E, 3301E, 3400E, 3401E, 3500F, 3501F, 3600E, 3601E, 3700D, 3800D, 3810D, 3815D, 3960E, 3980E, 4200F, 4201F, 4400F, 4401F, 5001D, 5001E.

Solution

Note: The command 'diagnose hardware test button' can be run to confirm if the button is NMI or Reset (for example: in 101E, the button does not have a 'RESET' label, but it has a reset function, not NMI).

For example:

diagnose hardware test button
Test Begin at UTC Time Tue Dec 26 13:07:05 2023

13:07:05 ( 0s) ==> Reset Button

Please prepare for reset button test.

Do you want to continue this test? (y/n) (default is n) n

Test skipped by user.

Some hardware models of FortiGate have an NMI pinhole which is used for interrupting the controller. When the NMI button is pressed, particular information about the kernel and CPU is displayed on the console session.

The generated information is very useful for investigating a kernel panic that occurs resulting in a halting state or troubleshooting system freezing when the system hangs and cannot be interrupted.

Once the procedure is completed, the FortiGate unit will reboot itself automatically.

To collect NMI outputs on the console session please do the following steps:

1. Connect a computer with a terminal emulator such as PUTTY or Tera Term to the FortiGate console port using a serial cable.
2. Configure the terminal to save CLI output to a text file. (For example, configure session logging to save the printable output to a text file in PUTTY).
3. On the FortiGate, use a pin or needle to press the NMI button and hold it down for one minute. FortiGate will output a kernel dump to the serial CLI connection and then will reboot itself automatically.
4. Save the text file and close the CLI session.

The NMI button is located typically on the front left side of the FortiGate. The position of the NMI button on some hardware models of FortiGate has been shown in the following pictures as a reference:

Related Article:

Technical Tip: How To Reset To Factory Default Configuration using external button