Skip to main content
yzayani
Staff
yzayani
Staff
June 29, 2026

Troubleshooting Tip: High SoftIRQ on a single CPU core causes HA split-brain

  • June 29, 2026
  • 0 replies
  • 330 views

Description

This article describes an issue where a single CPU core reaches 100% SoftIRQ utilization due to high-rate non-offloaded GRE traffic combined with LLDP reception enabled on aggregate interfaces, causing HA heartbeat packet loss and a split-brain scenario.

Scope

FortiGate NP6 platforms (tested on FG-3960E). FortiOS v7.4.x. Active-Passive HA clusters carrying GRE tunnel traffic.

Solution

The following symptoms may be observed:

  • A single CPU core (reported as Core 45) spikes to 100% SoftIRQ while all other cores remain at normal utilization.

  • HA heartbeat packet loss is reported on both heartbeat interfaces simultaneously.

  • The cluster triggers a split-brain failover. After failover, the same core on the new primary saturates again and remains at 100% for 20 - 30 minutes.

  • SNMP connectivity to the primary unit is lost during the event.

  • IPS engine restart does not resolve the CPU spike.

  • Two compounding factors drive this issue.


  1. Non-offloaded GRE traffic (primary trigger).


diagnose npu np6 npu-feature
np_0 np_1 np_2 np_3 np_4 np_5 np_6 np_7 np_8 np_9 np_10 np_11 np_12 np_13 np_14 np_15
------------------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- --------- ---------
GRE tunnel No No No No No No No No No No No No No No No No


  1. LLDP reception on aggregate interfaces:

  • When 'lldp-reception enable' is configured on an aggregate interface, the LLDP daemon registers an AF_PACKET + ETH_P_ALL listener on every physical member port.

  • This causes every packet arriving on those ports to be evaluated by a BPF filter, adding overhead proportional to total packet rate not just LLDP traffic. The 'fnsysctl cat /proc/net/ptype' output will show the following output:


ALL  port1    packet_rcv+0x0/0x390
ALL  port2    packet_rcv+0x0/0x390


  • This issue is resolved in FortiOS v8.0.1.

Workarounds:

  • Move GRE traffic off this device.

  • Remove LLDP from AGG interfaces.

  • Use mgmt2 as an HA heartbeat port to decouple HA from NPU-to-CPU path congestion.

  • Enable 'dedicated-management-cpu'.

  • Consider upgrading to an NP7 platform which supports GRE offloading.

      Terms & ConditionsAccessibility statement