Technical Tip: Kernel Panic in NP7 drivers on FortiGate causes continuous device reboots

The kernel crash log can be collected through a console session to the FortiGate while kernel debugging is enabled.

Alternatively, the COMLog feature can be enabled on some devices.

For instructions on enabling COMLog on a supported device, refer to Technical Tip: How to use the COMLog feature.

Kernel debugging or the COMLog feature needs to be enabled before the kernel panic is observed.

On FortiGates with an NP7 chip and configured with the default-qos-type set to 'shaping' may experience continuous reboots, and the following log may be seen in the COMLog or with kernel debugging enabled in the console session.

The following setting may trigger the Kernel panic:

config system npu
    set default-qos-type shaping
end

A Kernel Panic, as listed below, may occur with this setting enabled:

(Press 'a' to accept):kernel BUG at drivers/net/np7/./qos/np7_sys_qtm.c:303!
invalid opcode: 0000 [1] SMP
CPU: 2 PID: 3515 Comm: authd4 Tainted: P 4.19.13 1
RIP: 0010:np7_sys_qtm_qqid_sch0_alloc+0x28e/0x300 [filter4]
Code: 66 25 fc 1f 48 89 f2 09 c8 66 89 06 48 89 de e8 a8 ca ff ff 31 c0 45 84 ff 74 1f 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 5d c3 <0f> 0b 41 0f b7 57 10 66 89 50 2e e9 ff fe ff ff 41 0f b7 cd 4f 8d

For additional information about the 'default-qos-type' configuration on FortiGate, refer to the following documents:

Configuring NP7 traffic shaping 

Configuring default-qos-type {policing | shaping} 

The issue is resolved in v7.4.8 and v7.6.1.

Workaround:

As a workaround, configure the NP7 traffic shaping to use 'Policing' instead of 'Shaping'. The FortiGate restarts after changing this setting.

config system npu

    set default-qos-type policing

end

If the device is rebooting frequently and does not allow for the configuration change, follow these steps:

1. Interrupt the boot sequence to format the unit. Refer to Technical Tip: Formatting and loading FortiGate firmware image using TFTP.
2. Load the same firmware version as before using TFTP.
3. Modify the most recent known good configuration file to set the default QoS type to 'Policing.'
4. Load the modified configuration file onto the unit using the Restore System Configuration option on the GUI. Refer to Backing up and restoring configurations from the GUI.
5. If Steps 1-4 are not possible, unplug all of the cables except the MGMT interface to change the configuration. This will prevent traffic from being processed to the QTM module.

If further information is needed, open a ticket with TAC.