Technical Note: Optimize FortiGate-VM performance by configuring CPU interrupt affinity

[Dinesh_FTNT]

Description

Optimizing FortiGate-VM performance by configuring interrupt-affinity attributes to improve efficiency and resource utilization. 

Scope
FortiOS v5.6, v6.0
Solution

Interrupt affinity (also called CPU affinity) maps FortiGate-VM interrupts to the CPUs assigned to the FortiGate-VM. The CPU affinity mask is used to define the CPUs to the corresponding interrupts.

A common use of this feature would be to improve the FortiGate-VM's networking performance by:

• On the VM host:
  
• Add multiple host CPUs to the FortiGate-VM.
• Configure CPU affinity to specify the CPUs the FortiGate-VM can use.
• Configure other VM clients on the VM host to use other CPUs.
• On the FortiGate-VM:
  
• Assign network interface interrupts to a CPU affinity mask includes the CPUs used by FortiGate-VM.
  

In this way, all of the available CPU interrupts for the configured host CPUs are used to process traffic on the FortiGate interfaces. This configuration can lead to improved FortiGate-VM network performance as there is are dedicated VM host CPU cycles processing the FortiGate-VM's network traffic.

The following CLI command is used to configure interrupt affinity for FortiGate-VM:

config system affinity-interrupt
edit <index>
set interrupt <interrupt-name>
set affinity-cpumask <cpu-affinity-mask>
next
end

Where <interrupt-name> is the name of the interrupt to associated with a CPU affinity mask. View the FortiGate-VM interrupts by using the diagnose hardware sysinfo interrupts command. Normally all of the interrupts for a given interface are associated with the same CPU affinity mask.

<cpu-affinity-mask> defines the CPU affinity mask for the CPUs associated with the interrupt.

For example, consider the following configuration:

• port2 and port3 interfaces of a FortiGate-VM send and receive most of the traffic.
• On the VM host the CPU affinity is configured between and four CPUs (CPU 0, 1 , 2, and 3) for the FortiGate-VM
• SR-IOV is enabled and SR-IOV interfaces use the i40evf interface driver.

The output from the diagnose hardware sysinfo interrupts command shows port2 has the following transmit and receive interrupts:

i40evf-port2-TxRx-0
i40evf-port2-TxRx-1
i40evf-port2-TxRx-2
i40evf-port2-TxRx-3

The output from the diagnose hardware sysinfo interrupts command shows port3 has the following transmit and receive interrupts:

i40evf-port3-TxRx-0
i40evf-port3-TxRx-1
i40evf-port3-TxRx-2
i40evf-port3-TxRx-3

Use the following command to associate the port2 and port3 interrupts with CPU 0, 1 , 2, and 3.

config system affinity-interrupt
edit 1
set interrupt "i40evf-port2-TxRx-0"
set affinity-cpumask "0x0000000000000001"
next
edit 2
set interrupt "i40evf-port2-TxRx-1"
set affinity-cpumask "0x0000000000000002"
next
edit 3
set interrupt "i40evf-port2-TxRx-2"
set affinity-cpumask "0x0000000000000004"
next
edit 4
set interrupt "i40evf-port2-TxRx-3"
set affinity-cpumask "0x0000000000000008"
next
edit 5
set interrupt "i40evf-port3-TxRx-0"
set affinity-cpumask "0x0000000000000001"
next
edit 6
set interrupt "i40evf-port3-TxRx-1"
set affinity-cpumask "0x0000000000000002"
next
edit 7
set interrupt "i40evf-port3-TxRx-2"
set affinity-cpumask "0x0000000000000004"
next
edit 8
set interrupt "i40evf-port3-TxRx-3"
set affinity-cpumask "0x0000000000000008"
next
end

Below are CPU mask values with reference to CPU.

Affinity-cpumask	CPU
"0x0000000000000001"	CPU0
"0x0000000000000002"	CPU1
"0x0000000000000004"	CPU2
"0x0000000000000008"	CPU3
"0x0000000000000010"	CPU4
"0x0000000000000020"	CPU5
"0x0000000000000040"	CPU6
"0x0000000000000080"	CPU7
"0x0000000000000100"	CPU8
"0x0000000000000200"	CPU9
"0x0000000000000400"	CPU10
"0x0000000000000800"	CPU11
"0x0000000000001000"	CPU12
"0x0000000000002000"	CPU13
"0x0000000000004000"	CPU14
"0x0000000000008000"	CPU15