Description
This article describes how to configure and troubleshoot packet distribution between FortiGate VM cores when using single or multiple IPSec tunnels.
Scope
FortiGate-VM.
Solution
For a scenario like this, with three IPsec tunnels, IPsec decrypted traffic is being distributed among three CPU, one for each IPsec tunnel. If more tunnels are added, they will be using other CPU.
If the infrastructure contains only one tunnel up (VPN to 10.120.26.71), all the decrypted traffic will be handled by only one CPU and this can cause performance issues.
For distribution the decrypted traffic evenly through all the CPU with single or multiple IPSec tunnels affinity-packet-distribution needs to be used.
First, it is important to understand that for each CPU there is an affinity CPU mask (hexadecimal value) representing the CPU core ID.
| Affinity-cpumask | CPU |
| "0x0000000000000001" | CPU0 |
| "0x0000000000000002" |
CPU1 |
| "0x0000000000000004" | CPU2 |
| "0x0000000000000008" | CPU3 |
| "0x0000000000000010" | CPU4 |
| "0x0000000000000020" | CPU5 |
| "0x0000000000000040" | CPU6 |
| "0x0000000000000080" | CPU7 |
If the goal is to distribute the traffic amount the first four CPU (CPU0 to CPU3) the affinity-cpumask should be set to 0xf (sum of first four CPU masks: 0x1 + 0x2 + 0x4 + 0x8 = 0xf), for distribution among CPU4 to CPU7 the value should be 0xf0 and so on.
Putting this on a table:
| CPU | Affinity-cpumask | Affinity-cpumask (non-zeros) |
| CPU0 to CPU3 | "0x000000000000000f" | "0xf" |
| CPU3 to CPU7 | "0x00000000000000f0" | "0xf0" |
| CPU8 to CPU11 | "0x0000000000000f00" | "0xf00" |
| ... | ... | ... |
| CPU0 to CPU7 (8x CPU) | "0x00000000000000ff" | "0xff" |
| CPU0 to CPU11 (12x CPU) | "0x0000000000000fff" | "0xfff" |
With this information, the configuration should look like this for distributing traffic among the 8x CPU.
config system affinity-packet-redistribution
edit 1
set interface "port2"
set round-robin enable
set affinity-cpumask "0xff"
next
end
The interface selected will be the interface where the tunnel is being created, in this case, port2.
Note: round-robin option should be enabled if only one IPsec tunnel is being used, because as seen before, with only one IPsec tunnel, traffic will use only the same CPU for decrypting.
With round-robin enabled, traffic will be round-robin between each CPU even with a single IPsec tunnel as seen in the following example.
Related articles:
FortiGate-VM affinity packet redistribution
Technical Tip: Configuring CPU affinity for URL filters Technical Tip: How to assign value as affinity CPU mask to load balance the traffic between CPU Core... Technical Tip: How to increase the receive queue/buffer on the NIC to optimize throughput Technical Note: Optimize FortiGate-VM performance by configuring CPU interrupt affinity
