DPDK acceleration for ipsec tunnels on virtual fortigates

Hello,

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible.

Thanks,

Hello,

We are still looking for an answer to your question.

We will come back to you ASAP.

Thanks,

Hi @depereo ,

​You're observing limited performance gains from DPDK acceleration on your FortiGate VMs during IPsec throughput testing. Despite enabling DPDK and confirming that IPsec traffic is being offloaded, the throughput increase from approximately 1.2 Gbps to 1.5 Gbps suggests that additional optimizations may be necessary.​

• CPU Allocation and Affinity: Your current configuration assigns DPDK processes to CPUs 0 and 1. This limited allocation may not provide sufficient resources for optimal performance. Consider increasing the number of vCPUs and distributing DPDK processes across additional cores to enhance parallel processing capabilities.

         https://docs.fortinet.com/document/fortigate-private-cloud/7.6.0/kvm-administration-guide/79696/dpdk-cpu-settings

         https://docs.fortinet.com/document/fortigate-private-cloud/7.6.0/kvm-administration-guide/96687/isolate-cpus-used-by-dpdk-engine

• IPsec Soft Decryption Asynchronous Processing: In FortiOS versions prior to 7.4.2, enabling the ipsec-soft-dec-async setting can distribute IPsec session decryption across multiple CPU cores, potentially improving throughput. However, note that this setting has been removed in FortiOS versions 7.4.2 and later.

         https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-Improving-IPsec-performance-throughput-for/ta-p/250960

• Phase 1 and Phase 2 Proposal Optimization: Ensure that your IPsec Phase 1 and Phase 2 proposals utilize hardware-accelerated algorithms. For instance, AES256 is generally more efficient than 3DES and is supported by FortiGate's hardware acceleration features.​
• Host and VM Configuration: Some basic configurations might also be required on the virtualization side. These can be summarized as follows:

• Hugepages: Verify that the hugepage allocation is sufficient for your workload.​

         https://docs.fortinet.com/document/fortigate-private-cloud/7.6.0/vmware-esxi-administration-guide/73591/hugepages

• NUMA Awareness: Ensure that vCPUs and memory are allocated within the same NUMA node to reduce latency.

         https://docs.fortinet.com/document/fortigate-private-cloud/7.6.0/vmware-esxi-administration-guide/495998/numa-identification

• SR-IOV and VT-d: If supported by your hardware and hypervisor, enabling SR-IOV and VT-d can provide direct I/O access, reducing overhead.​

         https://www.linkedin.com/pulse/boost-virtual-fortigate-firewall-performance-using-dpdk-joe-brunner-2pdnc

• Virtio and RPS Optimization: FortiOS 7.6.1 introduces enhancements for IPsec performance through virtio and Receive Packet Steering (RPS), which can distribute packet processing across multiple CPUs.​

         https://fortinetweb.s3.amazonaws.com/docs.fortinet.com/v2/attachments/c8258039-3f9c-11ef-bfe5-fa163e15d75b/FortiOS-7.6-KVM_Administration_Guide.pdf

         https://docs.fortinet.com/document/fortigate-private-cloud/7.6.0/kvm-administration-guide/758496/dpdk-diagnostic-commands

If you’re still experiencing issues after following the steps provided in the reference links above, could you please run the following commands and share their output with us? This will help us better analyze the situation.

diagnose dpdk log show

diagnose dpdk statistics clear all

diagnose dpdk statistics show

diagnose dpdk performance show

diagnose sys mpstat 2 3

BR.

If my answer provided a solution for you, please mark the reply as solved it so that others can get it easily while searching for similar scenarios.

CCIE #68781

Hello,

I would recommend to disable "sleep-on-idle" setting for performance reasons.

Moreover, I would recommend to check whether SR-IOV is enabled (all benchmarks are run when SR-IOV is enabled):

https://www.fortinet.com/content/dam/fortinet/assets/data-sheets/FortiGate_VM_KVM.pdf (page 7)

Also, I would recommend to check IPsec performance while AES256GCM cypher is set (as in the benchmark).

https://docs.fortinet.com/document/fortification/7.0.0/new-features/954778/add-ipsec-fast-path-in-vpn-and-dpdk-for-fortigate-vm-7-0-6 (list of supported cyphers)

Based on your description, it seems the offload is not occurring. Check the npu_flag in the output of the diagnose vpn tunnel list command. For more details, refer to this knowledge base article:  Technical Tip: Ensuring IPSec traffic is offloaded for improved throughput