FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Article Id 192590

This article describes how to configure DoS protection’s quarantine/elapse/reset time for thresholds based on the number of concurrent sessions.

For example.

Default TCP source concurrent session is 5000, no timer will be set by default, and sessions will be blocked as long as they exceed 5000 concurrent sessions.
To specify a period of blocking the attacker, it can be configured as commands below.

For example, block the IP for 1 minute once it exceeds the 5000 concurrent sessions:

# config firewall DoS-policy
    edit 1
        set interface "port15"
        set srcaddr "all"
        set dstaddr "all"
        set service "ALL"
# config anomaly
    edit "tcp_dst_session"
        set status enable
        set log enable
        set action block
        set quarantine attacker
        set quarantine-expiry 0d0h1m    <----- m=minutes, h=hours, d=day.
        set quarantine-log enable
        set threshold 5000

Please note that if the anomaly is on FortiGate itself, the traffic is considered local-in traffic in the firewall. FortiOS only do such quarantine for forwarding traffic and not for local-in traffic. Hence, the anomaly on FortiGate itself will only take effect with the IPS meter (when the threshold is met, there will be a drop based on PPS) and does not quarantine itself (endpoint check).

Hence, it is expected behaviour that Fortigate will respond to the traffic until the threshold value is reached although the IP address is mentioned in the quarantine list.