Technical Tip: SYN mitigation

[cbenejean]

Description

This article explains why FortiDDoS may continue to forward SYN packets under a SYN flood attack even though the threshold for SYN packets has been set.

Scope

All FortiDDoS B/E/F Models.

Solution

The SYN mitigation happens only when the following criteria/settings are set in the SPP setting (B/E-Series) or TCP Profile for the specific SPP (F-Series):

• SYN validation needs to be enabled in the TCP session feature control.
• The SYN mitigation Inbound direction checkbox is checked. Do NOT use Outbound.
• SYN Validation type is selected. Use SYN Cookie option ONLY. This article assumes this feature is used.
• SYN flood has been detected, which means that the SYN or SYN per Destination Thresholds have been crossed. 
• SPP is in prevention mode.

First, remember that these parameters are 'Scalars' with Adaptive/Estimated Thresholds. SYNs will be allowed to pass without validation until the higher of the System Recommended Threshold or the Adaptive Estimated Threshold is crosses. FortiDDoS F-Series shows both these Thresholds on the Scalar graphs.

When the over-threshold SYN flood mitigation starts, all SYN packets are dropped unless they are from sources that are in the LIP* table. Every SYN packet that does not come from a source in the LIP table is challenged with a SYN-ACK sent by FortiDDoS. The SYN-ACK contains a 'cookie' which allows FortiDDoS to validate good source ACK responses without the use of a state table that can fill under flood. As soon as a legitimate client passes the SYN validation challenge the source IP is added to the LIP table and allowed to proceed to the server.

Thus, if there is a 'flood' from legitimate source IPs, the system will allow over-threshold SYNs. This helps to prevent false-positive drops from too-low Thresholds. If this is a connection-based flood. other mitigations at Layer 4 and Layer 7 will see the flood and mitigate further.

Note 1: There are 3 SYN validation options but ONLY the SYN Cookie option should be used. This option has worked very well for more than 10 years. Others create more traffic or are unreliable with some traffic.

Note 2: When the SYN-per-Source Threshold is crossed, the Source IP is blocked for the blocking period with no SYN Validation and all traffic including SYNs from that Source IP is dropped. SYN-per-Source floods are normally used to create SYN-ACK Reflection floods from the protected servers. If FortiDDoS sent SYN-ACK validations it would be contributing to the attack.

* LIP table = Legitimate IP table which contains any external Source IP that successfully completed a 3-way TCP handshake for an established session during normal traffic or from the SYN Validation above. The LIP table is aged quickly to prevent attackers from making one connection to get a LIP entry and then sending a flood of SYNs.