Technical Tip: Use custom IPS signature to block HTTPS traffic on large traffic volume

Description

This article describes how to use a custom IPS signature to block HTTPS from a high traffic volume.

Scope

FortiGate v7.0 and earlier.

Solution

Some ISPs need to block HTTPS traffic from high volumes of end-user traffic.

The ISPs need a solution where the traffic inspection is minimal but able effectively block specific internet domains, especially those domains which are hosted on Content Delivery Networks (CDN) (e.g. Akamai).

The UTM webfilter is an option, but it can perform extra validations (certificate CN, SAN, etc) at application layer which is not the objective to high traffic volume.

Using an IPS custom signature to search for a pattern in the TLS SNI (Server Name Indication) is an option, reducing the processing to only a pattern match in the clear text traveling in the packets.

Firewall address type FQDN is advisable to reduce the amount of traffic sent to the IPS, in order that the firewall policy only matches that IP destination address obtained from the DNS resolution of the end-user.

Requirements:

1. DNS resolution from end clients passing through FortiGate.

See Using wildcard FQDN addresses in firewall policies to create the wildcard FQDN addresses to have FortiGate populate it with the IP address contained in the DNS answer.

These FQDN addresses will be declared as 'dstaddr' in the firewall policy.

config firewall address
    edit "site1.com"
        set type fqdn
        set fqdn "*.site1.com"
    next
end

config firewall address
    edit "site2.com"
        set type fqdn
        set fqdn "*.site2.com"
    next
end

diagnose test application dnsproxy 6nvfid=6 name=*.site2.com ver=IPv4 wait_list=0 timer=0 min_ttl=600 cache_ttl=0 slot=-1 num=1 wildcard=1
         203.0.113.3 (ttl=600:559:559)nvfid=6 name=*.site1.com ver=IPv4 wait_list=0 timer=0 min_ttl=600 cache_ttl=0 slot=-1 num=1 wildcard=1
         203.0.113.3 (ttl=600:556:556)n

2. During the TLS handshake, the TLS SNI field must be included within the TLS hello packet.

With a packet capture of the HTTPS traffic, confirm the TLS Handshake contains the TLS SNI extension with the name of the site in clear text.

Create the custom IPS Signatures to match the FQDN to sites hosted on the same CDN network.

config ips custom
    edit "IPS_AllowSNI_SITE1"
        set signature "F-SBID( --name \"IPS_AllowSNI_SITE1\"; --attack_id 2559; --service SSL; --flow from_client;--pattern \"site1\";--no_case;--context packet;--severity low;)"
        set comment ''
    next
end
config ips custom
    edit "IPS_BlockSNI_SITE2"
        set signature "F-SBID( --name \"IPS_BlockSNI_SITE2\"; --attack_id 2178; --service SSL; --flow from_client;--pattern \"site2\";--no_case;--context packet;)"
        set comment ''
    next
end

Create the IPS sensor, which will allow the site1 and deny the site2 with TCP.RST action. 

config ips sensor
    edit "IPSProf_BlockByTLSSNI_CDN"
        config entries
            edit 1
                set rule 2559
                set action pass
            next
            edit 2
                set rule 2178
                set action reset
            next
        end
    next
end

Create a new firewall policy on top of firewall policy matching all HTTPS traffic.

The new policy 'dstaddr' must be the FQDN address objects.

Set the IPS sensor with the custom IPS signatures.

config firewall policy    
    edit 26
        set name "HTTPS_TLSSNI_IPS"
        set srcintf "CORE"
        set dstintf "VLAN3102"
        set action accept
        set srcaddr "MOB_Subscribers"
        set dstaddr "site1.com" "site2.com"
        set schedule "always"
        set service "HTTPS"
        set utm-status enablen        set ips-sensor "IPSProf_BlockByTLSSNI_CDN"
        set logtraffic all
    next
end

Only the traffic to www.site1.com and www.site2.com will match the firewall policy ID 26, because the FQDN address object contained the IP address obtained on the DNS resolution from end clients.

The IPS will search from a pattern in clear text into the packet and will allow www.site1.com and TCP.RST to www.site2.com.