Technical Tip: PDQ_OSW_EHP and PDQ_OSW_IPSEC drop in NP6 units

The following drop can be observed in the NP6 series processor while running the diagnostics command of the drop counter engine(DCE) with NPU ID.

FGT# diag npu np6 dce 0

IHP0_PKTCHK :0000000000000001 [5a] PDQ_ISW_SSE1 :0000000000000014 [98]
PDQ_OSW_EHP0 :0000000000016254 [a1] PDQ_OSW_EHP1 :0000000000040633 [a2]
PDQ_OSW_EHP2 :0000000000039225 [a3] PDQ_OSW_EHP3 :0000000000052612 [a4]
PDQ_OSW_IPSEC0I :0000000000000001 [a5] PDQ_OSW_IPSEC0O :0000000000150383 [a6]
PDQ_OSW_IPSEC1O :0000000000217502 [a8]

FGT# diag npu np6 dce 0

PDQ_OSW_EHP0 :0000000000000013 [a1] PDQ_OSW_EHP1 :0000000000000015 [a2]
PDQ_OSW_EHP3 :0000000000000016 [a4] PDQ_OSW_IPSEC0O :0000000000000077 [a6]
PDQ_OSW_IPSEC1O :0000000000000325 [a8]

FGT# diag npu np6 dce 0

PDQ_OSW_EHP0 :0000000000000026 [a1] PDQ_OSW_EHP1 :0000000000000017 [a2]
PDQ_OSW_EHP2 :0000000000000024 [a3] PDQ_OSW_EHP3 :0000000000000055 [a4]
PDQ_OSW_IPSEC0O :0000000000000047 [a6] PDQ_OSW_IPSEC1O :0000000000000384 [a8] 

In general, the drop counter is observed to get increased after running the CLI command 'diag npu npX dce <npu_id>' multiple times. The above report is printing the counter for the counters 'PDQ_OSW_EHP' and 'PDQ_OSW_IPSEC'.

• The PDQ_OSW_IPSEC drop counter on a FortiGate's NP6 processor indicates the number of IPsec packets dropped due to issues in the packet descriptor queue during outbound switching. This typically occurs when the NP6 processor encounters problems while processing or transmitting IPsec traffic, possibly due to resource constraints or configuration issues.
  
• The PDQ_OSW_EHP drop counter in FortiGate’s NP6 processor represents the number of packets discarded during outbound switching due to issues in the packet descriptor queue related to encapsulated host protocol traffic. This typically happens when NP6 struggles to process or forward EHP packets, often due to resource limitations or configuration mismatches.
  
  

Here the packet descriptor queue refers to the mechanism to manage the flow of packets through the hardware acceleration pipeline.

These drops 'PDQ_OSW_IPSEC' and 'PDQ_OSW_EHP' show that the NPU was busier than expected while handling the load and dropped the packets due to the reason. Therefore, the scenario indicates that the NPU limitation while handling a huge volume of load and the following workarounds can heal if there is no large number of NPU drops:

• Offloading traffic from the NPU.
  
• Disabling the hardware acceleration.
  
  

Additionally, the SOFTIRQ counters may appear which is also expected with these NPU counters in the CPU performance report without triggering any packet drop or errors in the interface level. In any situation the drops cause a huge network performance interruption or the softirq rises high in the CPUs, it is requested to contact and report the incident to the Fortinet Technical Assistance Center

Related articles:
Disabling NP offloading for individual IPsec VPN phase 1s
Technical Tip: FortiGate Disable Hardware Acceleration
Troubleshooting Tip: Check SoftIrq increments (recommended when experiencing high CPU usage)