Technical Tip: FortiGate Web Filtering and the impact of HTTP/2 multiplexing and connection coalescing

With HTTP/1.1 and earlier, HTTP web clients would traditionally make separate TCP connections for each domain being requested. Since the TLS handshake occurs separately for each TCP connection, it was relatively straightforward for the FortiGate to identify the FQDN associated with each stream for Web Filtering.

With HTTP/2 and later, multiplexing and connection coalescing were made available as new feature enhancements. When a client makes HTTP/2 requests to several FQDNs that have overlapping IP addresses, it is possible for a single shared TCP session to be created that supports simultaneous HTTP requests going to multiple different sub-domains/FQDNs. 

This HTTP/2 multiplexing and coalescing functionality has a notable effect on FortiGate-based Web Filtering, specifically when using TLS certificate-inspection and when attempting to make allow/block decisions based on specific sub-domains (i.e., allow a.example.com but block b.example.com).

Notes:

• HTTP/1.1 supported a feature called 'pipelining', which is similar to HTTP/2 multiplexing, but in general, the feature is disabled by default on most web browsers and is seldom observed in the real-world.
• HTTP/2 connection coalescing requires the destinations to have a number of factors in common from an IP address and certificate extension standpoint. For more specific info on HTTP/2 connection reuse, refer to Section 9.1.1 of RFC 7540.

The problem:

As a reminder, Web Filtering + certificate-inspection will scan a connection while the TLS handshake is occurring and uses the following information to make an allow/block decision:

• The Server Name Indication (SNI) in the TLS Client Hello (potentially hidden from view in TLS 1.3 using Encrypted SNI).
• The Common Name (CN) on the server's TLS Certificate (may require TLS active probing by the FortiGate for TLS 1.3 connections).

One issue here is that Web Filtering + certificate-inspection can only make an allow/block decision during this scan of the initial TLS handshake. After that point, the connection is encrypted, and the HTTP payload is inaccessible to Web Filtering.

Since an HTTP/2 connection allows the client to establish a single connection that can be reused for multiple sub-domains (that share the destination IP address), it becomes possible for a client to make an initial connection to an allowed domain, then make multiple follow-up requests within that now-encrypted connection for potentially disallowed sub-domains.

For example:

• Consider a situation where an administrator wants to allow web clients to access 'news.example.com' but block 'mail.example.com'.
• When a client visits 'news.example.com' using HTTPS, the FortiGate establishes this connection using Web Filtering + certificate-inspection. Since the FortiGate sees the client requesting 'news.example.com', it allows the TCP and TLS connection to establish.
• While on 'news.example.com', the client then selects a link to 'mail.example.com', which resolves to the same destination public IP as 'news.example.com'. The client's web browser determines that it can reach 'mail.example.com' over the TCP session that it had established previously, and so it makes an HTTP/2 request for that site over the existing connection.
• The FortiGate is unable to see this new request since the existing TCP connection is TLS-encrypted and the HTTP payload is not viewable, and so it cannot block this request. The client is then able to successfully connect to 'mail.example.com'.

Resolving this issue:

As noted earlier, this issue only applies when pairing Web Filtering with TLS certificate-inspection, and so the first recommendation is to consider implementing TLS deep-inspection.

When using TLS deep-inspection, the FortiGate acts as a machine-in-the-middle (MITM) and can continuously decrypt and inspect the actual HTTP payload. This allows Web Filtering to continue scanning the encrypted HTTPS connection and make allow/deny decisions for any HTTP requests made by the client over the shared/multiplexed connection.

If TLS deep-inspection is not feasible to deploy in the environment (see also: SSL/TLS deep inspection), then administrators should instead consider making allow/block decisions based on whole domains (e.g., block example.com as a whole, rather than specific sub-domains).

Additional notes:

• The DNS filter is a viable workaround as it does not rely on the Client Hello to determine what website a user visited.
  Even if 'news.example.com' and 'mail.example.com' resolve to the same IP, the browser would still make individual lookups for each subdomain and allow the chance to block it.
• Connection coalescing behavior is not easily predictable, as each web browser/client has different criteria and a level of 'aggressiveness' towards coalescing HTTP/2 connections. It is possible to see a client establishing a shared connection to a domain and reusing it for several related sub-domains, while at the same time establishing an entirely separate TCP connection for a different but still-related sub-domain.
• In practice, this means that some sub-domains for some websites may trigger the issue described above (leading to Web Filter misses), whereas other HTTP/2 websites may be consistently matched by Web Filtering + certificate-inspection without issue.
• Additionally, HTTP/2 multiplexing and connection-coalescing will generally not be used to share a TCP connection to multiple different domains, so it is unlikely that this can be used by a client to visit one website and 'sneak through' to a wholly different website.
• The FortiGate does support HTTP/2 multiplexing and connection coalescing when acting as an HTTP server (e.g., for Virtual Server and ZTNA functionality). See the related documents section below.

Related documents:

Virtual server load balance multiplexing

Virtual server stops working after upgrading to FortiOS firmware 7.2.4 build 1396 

HTTP2 connection coalescing and concurrent multiplexing for ZTNA

Connection management in HTTP/1.x

HTTP/2 connection coalescing

Connection coalescing in principle