Technical Tip: The Importance of planning to benefit from hardware offloading (NPU Accelerated traffic).

[npaiva]

Description

This article describes ideal ways to carefully plan deployments to benefit from hardware offloading.

Scope

FortiGate Physical Appliances.

Solution

This article will use an example with a FortiGate 200E.

This unit is connected to a FortiTester to perform a TCP throughput test, and a policy has been created to authorize traffic between two VLANs. Offloading has been disabled at the policy level, for demonstration purposes.

Testing initiated:

get sys performance status

CPU states: 0% user 1% system 0% nice 6% idle 0% iowait 0% irq 93% softirq

CPU0 states: 0% user 1% system 0% nice 6% idle 0% iowait 0% irq 93% softirq

CPU1 states: 0% user 1% system 0% nice 6% idle 0% iowait 0% irq 93% softirq

Memory: 4057316k total, 1476048k used (36.4%), 2348676k free (57.9%), 232592k freeable (5.7%)

Average network usage: 858670 / 858699 kbps in 1 minute, 214292 / 214320 kbps in 10 minutes, 265037 / 265068 kbps in 30 minutes

Maximal network usage: 939917 / 939836 kbps in 1 minute, 974512 / 969045 kbps in 10 minutes, 974512 / 969045 kbps in 30 minutes

Average sessions: 50075 sessions in 1 minute, 49986 sessions in 10 minutes, 27938 sessions in 30 minutes

Maximal sessions: 50107 sessions in 1 minute, 50110 sessions in 10 minutes, 50110 sessions in 30 minutes

Average session setup rate: 0 sessions per second in last 1 minute, 0 sessions per second in last 10 minutes, 0 sessions per second in last 30 minutes

Maximal session setup rate: 5 sessions per second in last 1 minute, 40 sessions per second in last 10 minutes, 41 sessions per second in last 30 minutes

Average NPU sessions: 0 sessions in last 1 minute, 2245 sessions in last 10 minutes, 11070 sessions in last 30 minutes

Maximal NPU sessions: 0 sessions in last 1 minute, 49900 sessions in last 10 minutes, 50000 sessions in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 0 days,  18 hours,  25 minutes

As shown, with an average load of 858Mbps in the last minute, the CPU is close to maxing out. Also, note that this unit has minimal configuration, and it is just running this test: it is not a production appliance. A production appliance would be even more impacted since it would possibly be dealing with encryption/decryption/UTM/IPsec, etc.

Now the same test with offloading enabled:

get system performance status

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

CPU0 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq

CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq

Memory: 4057316k total, 1390288k used (34.3%), 2434436k free (60.0%), 232592k freeable (5.7%)

Average network usage: 871224 / 871237 kbps in 1 minute, 309879 / 309908 kbps in 10 minutes, 132795 / 132807 kbps in 30 minutes

Maximal network usage: 901753 / 901748 kbps in 1 minute, 901753 / 901748 kbps in 10 minutes, 901753 / 901748 kbps in 30 minutes

Average sessions: 50056 sessions in 1 minute, 18060 sessions in 10 minutes, 7726 sessions in 30 minutes

Maximal sessions: 50088 sessions in 1 minute, 50108 sessions in 10 minutes, 50108 sessions in 30 minutes

Average session setup rate: 2 sessions per second in last 1 minute, 250 sessions per second in last 10 minutes, 83 sessions per second in last 30 minutes

Maximal session setup rate: 38 sessions per second in last 1 minute, 25010 sessions per second in last 10 minutes, 25010 sessions per second in last 30 minutes

Average NPU sessions: 49987 sessions in last 1 minute, 17996 sessions in last 10 minutes, 7664 sessions in last 30 minutes

Maximal NPU sessions: 49988 sessions in last 1 minute, 50000 sessions in last 10 minutes, 50000 sessions in last 30 minutes

Virus caught: 0 total in 1 minute

IPS attacks blocked: 0 total in 1 minute

Uptime: 0 days,  20 hours,  45 minutes

Now with an average load of 871 Mbps in the last minute, the CPU is almost in a total idle state. Notice that the average NPU sessions value almost totals the Average total sessions, meaning nearly 100% of the traffic is being handled by the NPU leaving the CPU free for other tasks.

NPU accelerated traffic is one of the major strong points about having a FortiGate Physical Appliance, and careful planning should be made when configuring the device to be able to use it.

As a rule of thumb every interface that is software based will not be able to have it’s traffic offloaded to the NPU, so instead if using software switches, use hardware switches, instead of using VDOM-Links, use NPU-VDOM Links, etc.

The specifics of which traffic can be offloaded can be found in the following documents:

• NP6 session fast path requirements
• NP7 session fast path requirements

Some units also have special requirements for offloading to be possible between a set of interfaces. Consider the 200E unit:

get hardware npu np6lite port-list

Chip   XAUI Ports            Max   Cross-chip

                             Speed offloading

------ ---- -------          ----- ----------

np6lite_0

       2    port9            1000M          NO

       1    port10           1000M          NO

       4    port11           1000M          NO

       3    port12           1000M          NO

       6    port13           1000M          NO

       5    port14           1000M          NO

       9    port15           1000M          NO

       10   port16           1000M          NO

       8    port17           1000M          NO

       7    port18           1000M          NO

np6lite_1

       2    wan1             1000M          NO

       1    wan2             1000M          NO

       4    port1            1000M          NO

       3    port2            1000M          NO

       6    port3            1000M          NO

       5    port4            1000M          NO

       8    port5            1000M          NO

       7    port6            1000M          NO

       10   port7            1000M          NO

       9    port8            1000M          NO

As shown above, this unit has not one but two NPUs: np6lite_0 and np6lite_1. Each of them is bound to different interfaces. This unit also does not have an Internal Switch Fabric (ISF), which means that traffic incoming from Port1 and destined to Port9 will have to go through the CPU. As a result, traffic will not be offloadable.

Careful planning needs to be undertaken to ensure that interface pairs with heavy traffic are configured on interfaces under the same NPU. This is not a concern if the FortiGate Appliance has ISF.

More info about 200E architecture is available in this section of the documentation.

For other models and appliances:

• NP6 architectures
• NP6XLite architectures
• NP6Lite architectures

Confirm if traffic is being offloaded by using the command 'get system performance status' as shown in the tests at the start of this document, and by exporting the session table.

For more info on how to check the session table and if traffic is being offloaded, see this document.