FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
This article describes a scenario where IPsec throughput on a FortiGate VM in AWS is suboptimal because both encryption and decryption of a single IPsec tunnel are processed on the same CPU core. This causes high CPU usage on SoftIrq.
Scope
FortiGate.
Solution
Symptoms:
High CPU utilization on a SoftIrq.
Low IPsec throughput even on instances with multiple vCPUs.
Latency increase and inconsistent performance across tunnels.
Debug output shows traffic for a single tunnel mapped to the same CPU:
Enabling ipsec-soft-dec-async sometimes improves, sometimes worsens performance, depending on the network environment and traffic pattern.
Root Causes:
On FortiGate-VM in AWS, IPsec traffic is distributed based on hash-based CPU affinity, which binds both encryption and decryption of the same tunnel to a single core by default.
This behavior is expected for VM instances with a single vNIC or limited queues, and results in uneven CPU utilization.
Soft-dec-async and affinity-packet-redistribution can partially mitigate, but are version-dependent and not stable in all traffic scenarios.
Debug:
Check CPU allocation per tunnel: Observe enc/dec cores for each tunnel. If enc and dec cores are the same, the tunnel is single-core bound.
Check total CPU usage: Confirm one CPU core is pegged while others remain low.
The distribution is handled by RPS/round-robin.
Workaround:
Enable software switch/VXLAN underlay: Deploy tunnels over VXLAN/bridge interfaces to distribute traffic across multiple vNIC queues and CPU cores.
Use multiple tunnels for ECMP: Split traffic across tunnels to achieve load distribution.
Affinity/packet redistribution: Use the config system affinity-packet-redistribution to adjust core assignment if necessary.
Scale VM instance: Increase the number of vCPUs or NICs to allow better interrupt distribution.
Long-term Solution: Use VXLAN underlay with a software switch to allow FortiGate VM in AWS to distribute IPsec encryption and decryption across multiple cores.
