Struggling to get my head around Web Filtering and SSL Exemptions

[x_member]

Hi,

I've been on a steep learning curve as I work towards implementing a Fortigate 60D running 5.2.2, however I have got to grips with most elements so far.

However I've hit a wall in understanding how I can create a policy allowing access to Microsoft & Windows updates for all.

Using Microsoft's lists of required URLs (some of which are wildcard) as published here: https://technet.microsoft.com/en-us/library/dd939870(WS.10).aspx I've attempted to create a urlfilter webfilter as below (based on comments in this post https://forum.fortinet.com/tm.aspx?m=122223) :

config webfilter urlfilter edit 141 set name WindowsUpdate set comment 'Windows Update URLS' config entries edit 1 [size="2"] set url [link]http://windowsupdate.microsoft.com[/link][/size] set type simple set action allow set status enable next edit 2 [size="2"] set url [link]http://*.windowsupdate.microsoft.com[/link][/size] set type wildcard set action allow set status enable next edit 3 [size="2"] set url [link]http://*.update.microsoft.com[/link] [/size] set type wildcard set action allow set status enable next edit 4 [size="2"] set url [link]http://*.windowsupdate.microsoft.com[/link][/size] set type wildcard set action allow set status enable next edit 5 [size="2"] set url [link]http://download.windowsupdate.com[/link][/size] set type simple set action allow set status enable next edit 6 [size="2"] set url [link]http://download.microsoft.com[/link][/size] set type simple set action allow set status enable next edit 7 [size="2"] set url [link]http://*.download.windowsupdate.com[/link][/size] set type wildcard set action allow set status enable next edit 8 [size="2"] set url [link]http://stats.update.microsoft.com[/link][/size] set type simple set action allow set status enable next edit 9 [size="2"] set url [link]http://ntservicepack.microsoft.com[/link][/size] set type simple set action allow set status enable next end

I then added an exemption for the SSL wildcard URLs to my existing ssl inspection policy, 

config firewall ssl-ssh-profile edit deep-inspection config ssl-exempt edit 0 set type fortiguard-category set fortiguard-category 140 end end

I can see that an additional exemption has been added to my deep-inspection profile both in the CLI (as an additional ID) and the UI as a reference to category "custom1". However if I go to view "custom1" in FortiExplorer under Security Profiles\Advanced\Web Rating Overrides (or Web Profile Overrides) it shows no evidence of my entered URLs.

With this in mind I have 2 questions:

(1) is it possible to view my urlfilter rules in the UI?

(2) How do I build an IPv4 policy to incorporate the configured urlfilters as permitted (yet excluded from SSL inspection)?

Any insights gratefully received.

[Christopher_McMullan · Accepted solution]

I'll provide a little more context, since the change between OS 5.0 and 5.2 was fairly dramatic.

In OS 4.3 and 5.0, you needed to define your inspection parameters in two places: the protocol options profile (4.3) or SSL/SSH inspection profile (5.0), as well as the webfilter profile. The combination of the settings would determine how scanning was accomplished. Historically, you would NOT select to scan HTTPS traffic in the protocol options but leave "Scan Encrypted Connections" checked in the webfilter profile in 4.3 to allow for CN inspection only (what hostname does the server respond with when the client requests a certificate in the Client Hello message of the SSL exchange). Otherwise, by the time the GET request is made in order for the urlfilter list or FortiGuard category webfilter profile to take effect, deep scanning had already taken place. In 5.0, you would do the opposite (don't ask me why): scan HTTPS traffic in the SSL/SSH inspection profile, but leave "Scan Encrypted Connections" unchecked in the webfilter profile. The CLI equivalent was "set https-url-scan enable".

This led to some avoidable confusion, so in 5.2 you explicitly define in the SSL/SSH inspection profile what kind of scanning you want to accomplish: deep inspection or certificate inspection. The former will perform MITM replacement of the server's certificate with a self-signed copy from the FortiGate. The latter will look at the CN field and SNI (if available) to determine what server the client is contacting. This obviously rules out filtering based on full URL paths like example.com/file, but still works on example.com. The profile also now allows you to protect a server instead of inspect client traffic, mimicking a load-balancing VIP.

I personally like the consolidation in 5.2.

What this means in practice is that exemptions take place all through the SSL/SSH inspection profile now. Previously, you absolutely would use wildcards or exempt entries in a URL filter list, and tailor the webfilter profile and protocol options to only inspect the Client Hello message.

[Christopher_McMullan]

Could you create address objects (FQDN and wildcard as the case may be) for the sites you want to exempt instead, and add them to the ssl-ssh-inspection profile in place of the custom category? The urlfilter list should be considered more as access control than a list enabling exemption from deep inspection.

The best place for that determination is in the SSL/SSH profile itself.

[x_member]

Christopher McMullan_FTNT wrote:

Could you create address objects (FQDN and wildcard as the case may be) for the sites you want to exempt instead, and add them to the ssl-ssh-inspection profile in place of the custom category? The urlfilter list should be considered more as access control than a list enabling exemption from deep inspection.

The best place for that determination is in the SSL/SSH profile itself.

Thanks for responding.

I agree that it's preferable to determine exemption within the SSL/SSH profile, however I was under the impression (based on some Googling rather than practical experimentation) that a urlfilter was the only way to define a wildcard.

Having done some further reading just now it appears I trod the wrong path.

I'll revert and try this approach - thanks for the signpost!

[Dave_Hall · Accepted solution]

Been using a variation of this for our remote sites on really slow sat. connections, but you really have to trust the DNS servers your fortigate(s) are connected to.

config firewall address
    edit "update.microsoft.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "update.microsoft.com"
    next
    edit "download.windowsupdate.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "download.windowsupdate.com"
    next
    edit "windowsupdate.microsoft.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "windowsupdate.microsoft.com"
    next
    edit "msftncsi"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "www.msftncsi.com"
    next
    edit "download.microsoft.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "download.microsoft.com"
    next
    edit "wustat.windows.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "wustat.windows.com"
    next
    edit "ntservicepack.microsoft.com"
        set associated-interface "wan1"
        set type fqdn
        set fqdn "ntservicepack.microsoft.com"
    next
end
config firewall addrgrp
    edit "Window-updates"
        set member "download.microsoft.com" "download.windowsupdate.com" "msftncsi" "ntservicepack.microsoft.com" "update.microsoft.com" "windowsupdate.microsoft.com" "wustat.windows.com"
    next
end
config firewall policy
    edit 0
        set srcintf "internal_net"
        set dstintf "wan1"
        set srcaddr "All_Internal"
        set dstaddr "Window-updates"
        set action accept
        set schedule "always"
        set service "ALL"
        set nat enable
    next
end

Keep in mind that MS in recent months (years?) have been using 3rd party content providers for pushing down updates, so using the above method can only go so far.   Addition to the above, we have enabled most updates via the App Sensor, including MS.

[Christopher_McMullan · Accepted solution]

I'll provide a little more context, since the change between OS 5.0 and 5.2 was fairly dramatic.

In OS 4.3 and 5.0, you needed to define your inspection parameters in two places: the protocol options profile (4.3) or SSL/SSH inspection profile (5.0), as well as the webfilter profile. The combination of the settings would determine how scanning was accomplished. Historically, you would NOT select to scan HTTPS traffic in the protocol options but leave "Scan Encrypted Connections" checked in the webfilter profile in 4.3 to allow for CN inspection only (what hostname does the server respond with when the client requests a certificate in the Client Hello message of the SSL exchange). Otherwise, by the time the GET request is made in order for the urlfilter list or FortiGuard category webfilter profile to take effect, deep scanning had already taken place. In 5.0, you would do the opposite (don't ask me why): scan HTTPS traffic in the SSL/SSH inspection profile, but leave "Scan Encrypted Connections" unchecked in the webfilter profile. The CLI equivalent was "set https-url-scan enable".

This led to some avoidable confusion, so in 5.2 you explicitly define in the SSL/SSH inspection profile what kind of scanning you want to accomplish: deep inspection or certificate inspection. The former will perform MITM replacement of the server's certificate with a self-signed copy from the FortiGate. The latter will look at the CN field and SNI (if available) to determine what server the client is contacting. This obviously rules out filtering based on full URL paths like example.com/file, but still works on example.com. The profile also now allows you to protect a server instead of inspect client traffic, mimicking a load-balancing VIP.

I personally like the consolidation in 5.2.

What this means in practice is that exemptions take place all through the SSL/SSH inspection profile now. Previously, you absolutely would use wildcards or exempt entries in a URL filter list, and tailor the webfilter profile and protocol options to only inspect the Client Hello message.

[x_member]

So I've clearly picked up on (1/2) an older approach whilst trawling the 'net and confused myself.

Many thanks for the guidance.

[x_member]

Just popped back to say thanks all for contributing with practical tips and context.

I managed to get my Windows Update rules configured successfully in the end.

On to the next  [strike]challenge[/strike] opportunity..