|
Suppose traffic shaping is configured on NP7 processor-based FortiGates. In that case, the corresponding traffic shaping policies are applied to the offloaded traffic with one of two mechanisms in the NP7 driver - QTM engine-based or TPE engine-based. This is configurable using the npu system setting 'set default-qos-type'. It is important to understand the difference between the two options, identify which is the default setting in a FortiOS version, and when the setting needs to be changed after an upgrade. There may be intermittent packet drops under certain conditions after upgrades, depending on which option was configured: how to identify this and remediate it proactively is explained in this article as well. Traffic shaping profiles and policy configurations are explained in the Traffic Shaping Policies section of the FortiGate Administration Guide, while this article focuses on the implications of NPQoS type setting for NP7 platforms.
FortiGate-2601F-NP7# config system npu
set default-qos-type {policing | shaping}
end
policing QoS type policing <----- Configure TPE engine to be the traffic shaper
shaping QoS type shaping. <----- Configure QTM engine to be the traffic shaper
If the default-qos-type configuration is changed, the FortiGate immediately restarts. So ensure to implement this command with caution and preferably during a maintenance window. If FortiGate is in HA mode, both units will be restarted at the same time.
Under certain conditions, the NP7 default-qos-type might be different than the option configured under config system npu -> 'default-qos-type'. Verify that the current npu qos-type reflects the option configured using the following diagnose command.
FortiGate-2601F-NP7 # diagnose npu np7 system-config
default_qos_type : policing (0) <----- TPE engine in use
vep_mode : 100G*2 (0x0)
combo_shaper_enable : disabled
max_sse_tmo : 40 (seconds)
per_sess_accounting : enabled-by-log (0)
sess_acct_intvl : 5 (seconds)
mcast_sess_accounting : tpe-based (0)
ip_assembly : disabled
ip_assembly_min_tmo : 64 (us)
ip_assembly_max_tmo : 200000 (us)
hyper_scale : Disabled
hhtbl-spctrl : Enabled
vlan-lookup-cache : Enabled
htab_msg_que : data-queue
nr_ded_que_mbr : 2
sse-tpe-accounting : Enabled
background-sse-scan : Disabled
dvlan-tag : 0x0
htx-icmp-csum-chk : Drop
ip-fragment-offload : Enable
ull_port_mode : 10G
isf_hash_algo : unknown (255)
ipv6_hash_sel : 255
rlt : Disable
Below is an example when the NP7 default_qos_type is set to 'shaping'.
FortiGate-2601F-NP7 # diagnose npu np7 system-config
default_qos_type : shaping (1) <----- QTM engine in use
vep_mode : 100G*2 (0x0)
combo_shaper_enable : disabled
max_sse_tmo : 40 (seconds)
per_sess_accounting : enabled-by-log (0)
sess_acct_intvl : 5 (seconds)
. . .
The 'default-qos-type' configuration is used to set the shaping engine option (either QTM or TPE engine) for the NP7 driver.
-
Traffic shaping with 'shaping' as default QOS type: If 'default-qos-type' is shaping, the NP7 driver will use the QTM engine (Queuing based Traffic Management engine) as the shaping engine. With the QTM engine, a round robin algorithm is used to schedule traffic in available queues for shaping. Under certain conditions, the QTM engine may cause the NP7 driver to intermittently drop packets even when configured shaping limits are not exceeded (more on this in a later section covering upgrades). As a result, in the newer FortiOS versions, the default qos-type is set to policing.
- Traffic shaping with 'policing' as default QOS type: If 'default-qos-type' is set to policing, the NP7 driver will use the TPE engine (Traffic Policing Engine) as the shaping engine. Traffic shaping is done with policing by using the NP7 accounting stats. This is the recommended mode for v7.2+, v7.4+, and v7.6+.
The default value of 'default-qos-type' is different depending on the FortiOS version. Here is a summary:
Note:
The default qos-mode setting is force-changed to 'policing' after upgrading to v7.2.11+, v7.4.8+, and v7.6.3+. If an upgrade is made to earlier versions, the default might remain as it was on the version before the upgrade (i.e, shaping). Run the CLI command 'diagnose npu np7 system-config' to confirm the setting that is currently active on the FortiGate.
Verify the traffic shaper setting after an upgrade:
If a FortiOS upgrade is made to a version that has a different default setting for 'default-qos-type' than the version it is upgraded from, it is important to verify what is the active setting is after upgrade and confirm that is the intended setting. So run the commands 'show system npu' and 'diagnose npu np7 system-config' on the old version before the upgrade, and on the new version after the upgrade - to verify the default-qos-type setting that's active after upgrade, and change the configuration if needed.
Random packet drops after an upgrade:
If there are random packet drops observed after an upgrade (even when traffic rate is within the shaping/policing limits), verify if shaping is the default-qos-type as mentioned in the previous section, and if yes, consider changing it to policing to use the TPE engine as the traffic shaper. If it can't be immediately changed (since a restart happens immediately after entering the cli command to change the QOS-type), workaround below can be used in the meantime to stop the NP7 QTM from blocking the traffic:
config system npu
set qtm-buf-mode 4ch
end
qtm-buf-mode <----- QTM channel configuration for packet buffer.
4ch <----- 4 DRAM channels for packet buffer.
Note: Run the CLI diagnostic command 'diagnose npu np7 getreg 0 qtm.qtm_dbg' about 5 times with 10-second intervals, and observe if this specific counter 'sch0_enq_drop_cnt' is increasing. This indicates possible packet drops due to the QTM buffer being full or being in an incorrect state.
If packets drop and traffic disruption continues even after applying the above workaround, contact Fortinet support with this diagnostic output and any other relevant details.
Note:
Starting from v7.4.8 GA (What's new for FortiOS 7.4.8), v7.6.3 GA, the option 'default-qos-type' under 'npu config' no longer supports shaping, and the only selection for this option is policing.
This affects NP7 models:
config system npu
set default-qos-type policing
end
The reason for this change is a known issue with fragmentation when the Queuing based Traffic Management QTM is utilized as result of selecting shaping for this option where packets above 6000 MTU, causing the QTM to hang and fragmentation to stop working. Detailed information can be found on the following release notes link: Changes to NP7 traffic shaping
That default-qos-type setting cannot be changed if the hyperscale firewall license is enabled.
Related documents:
Traffic shaping policies flowchart and configuration guide
Troubleshooting Tip: Traffic shaping troubleshooting commands
|
