Technical Tip: DoS policy on WAN interface can cause slowness in IPsec throughput or block IPsec traffic

[vprabhu_FTNT]

Description	This article describes how a DoS policy can cause slowness in traffic and IPsec issues due to the setup of rate-limiting on the external interface.
Scope	FortiGate.
Solution	Note: DoS policy applies to both tunnel establishment traffic (IKE) and also the encrypted traffic in the tunnel itself. Even though the traffic is encrypted in the tunnel, the DoS policy can still match the source and destination of the traffic flowing in the tunnel. DoS policy will apply to this traffic even though the interface in DoS policy is selected as the WAN interface and not the tunnel interface.  Generally, DoS policy is configured for rate-limiting traffic on external interfaces from public networks to internal to mitigate attacks coming in from the Internet. However, this can cause issues with IPsec tunnel operations and data transfer and is hard to detect. If the threshold is defined under 'udp_flood' in the DoS policy, it can limit IPsec throughput when NAT-T is set to enabled/forced under IPsec phase1-settings. If NAT-T is set to disabled under IPsec phase1-settings, ESP is sent instead of ESP encapsulation with UDP port 4500 which will not trigger the 'udp_flood' threshold under DoS policy. In another scenario, DoS-policy can cause traffic slowness on VIP listening on the external interface or cause slowness of traffic on FortiGate. This is due to the DoS policy rate limiting traffic on UDP ports flood/scan/src-session and TCP src-session. To allow traffic expected from sources like IPSec where UDP packets are always traversing and are legitimate traffic, the new DoS policy is to be placed on top allowing traffic from legitimate public hosts with no limit on expected traffic. This allows legitimate traffic to flow uninterrupted. For the same, go to Policy and Objects -> IPv4 DoS-Policy -> and create a new DoS-policy. Select remote-ipsec gateway IPs and service as IKE to allow for IPsec traffic and IPsec connection setup. If the IPsec traffic is being blocked by the DoS policy then a similar policy can be created with service all and Source Address as the remote private IP sending traffic in the tunnel  Ensure that the UDP-scan and UDP-src-session are disabled. Save changes. Move the new DoS policy to the top so it is first to be checked and allow that traffic without rate-limiting setup on the generic policy. Create any other policy specific to legitimate traffic to be allowed unrestricted and move it to the top. Check the anomaly logs under 'Log and Report' if any blocks are done by the DoS policy.