manage application control and web filter

[edoutreleau]

Hi

I have a problem of designe between application control and web filter

i have a machine that i want to 

- make update through apt-get

- acces to specific url  

i have application control with apt update and it works

but as soon as i apply my welb filterting to the only specific adress i want to allow the apt update doesn't work anymore. 

what are the relation vetween web filter and application control? 

[hmtay_FTNT]

Hello edoutreleau,

There are 2 modes that you can run in the profile-based firewall policy. Proxy-mode and Flow-mode. In flow-mode, packets are scan by all the security modules at the same time.

All of the applicable flow-based security modules are applied simultaneously in one single pass, and pattern matching is offloaded and accelerated by CP8 or CP9 processors. IPS, Application Control and CASI, flow-based Web Filtering and flow-based DLP filtering happen together. 

In proxy-mode, the packets are first scanned by App Control before they are scanned by the Web Filter proxy.

Decrypted SSL traffic is sent to the IPS engine (where IPS, Application Control, and CASI can be applied) before re-entering the proxy where actual proxy-based inspection is applied to the decrypted SSL traffic.

The links below should help you understand the flow chart for both modes in a better way hopefully.

http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-life-of-packet-54/lop-packet-flow-flo...

http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-life-of-packet-54/lop-packet-flow-pro...

A couple of questions about your configuration:

1. My guess is you want to block everything else and allow only Apt-Get and certain URLs. Is that correct?

2. If 1 is yes, are you blocking everything but Apt-Get on App Control?

3. If 2 is yes, and you use Web Filter to whitelist certain URLs, that would not work because while the new policy has Web Filter whitelisting the URLs, App Control does not whitelist the URLs. Packets have to be able to get through all the security modules to be allowed through. If any one module drops the traffic, then it will be dropped.

>>but as soon as i apply my welb filterting to the only specific adress i want to allow the apt update doesn't work anymore. 

This sounds weird.

Can you send me your configuration file, a packet capture, App Control logs and Forward Traffic logs in a PM? I will check them and let you know what could have been wrong. Thanks.

HoMing

[ted_barker]

Dear HoMing

Interesting topic (currently finished NSE4, tomorrow NSE7 training) and I still do not understand exactly the difference and how to apply which feature to the current problem. Maybe you can shed some light.

From what I understand is that the goal is

-- to allow apt-get update only to a single host, i.e. http://ch.archive.ubuntu.com/ubuntu/

So the application control signature apt-get sound like a good thing to do. I.e. allow apt-get.

Now in order to limit to the above HOST or URL a static URL webfilter with a match on it could be used to further limit apt-get to only those URL.

1) application control with signature apt-get update

2) webfilter static URL with a match on ch.archive.ubuntu.com (no idea how you filter on the host part only, without some regex, no support from FortiOS to match protocol, domain, host, path?)

Now if proxy inspection or flow inspection should not make a difference for that, right? It looks like the order of applying is not different, Application Control before Web Filter?

Question A) in your reference to flow based it lists "L7 decoder" web filtering before application control, but maybe Static URL webfilter is considered a "Local URL Filtering"

Question B) Is there a way to tie 1 and 2 together in a policy, i.e. only allow if application is apt-get and Static WebFilter matches? (Not sure if you put application control and webfilter into a single policy if it is AND or OR)

Question C) Is there a better way to achieve the goal?

Question Flow/Proxy: We have all our default VDOM in proxy mode, but some inspections such as IPS are always flow based. When should a customer use only flow based VDOM/inspection?

[hmtay_FTNT]

Hi Ted,

I will answer your questions in order.

A) The Static URL webfilter is considered "Local URL Filtering" in the flow diagram, not the L7 decoder. The L7 decoder only tries to identify and decode the packets coming in with known decoders. The rule matching is done at "Single-Pass Rule Matching".

B) Yes. In this case, where you want to block everything else and allow only certain sites/applications, you would need both Application Control and Web Filtering. That said, you can look at Web Filtering like a subset of Application Control that covers HTTP and HTTPS sessions. 

Case 1: Let's say we want to block Facebook. We can do it with either the "Facebook" signature under Application Control, or we could add Facebook's domains manually in a Static URL Filter. 

Case 2: Let's say we want to block Skype. In this case, we have to use the "Skype" signature under Application Control. Web Filtering will not work because a large portion of Skype's sessions uses a proprietary protocol - non HTTP/HTTPS.

In short, whatever the Web Filter can do, Application Control can do the same. What the Application Control can do, Web Filter might not be able to do the same.

Back to the original question. Yes, you can tie them both in the same policy. What you can do is set the HTTP.BROWSER_*, SSL_*, HTTPS.BROWSER signatures in Application Control to Allow - and also Apt-Get specifically for this case. This way, you let Web Filter do the controls for HTTP and HTTPS sessions. When the Application Control module scans a HTTP/HTTPS packet, it lets the packet pass to the Web Filter module to scan it. Then, you apply the filters accordingly in your Web Filter profile - allow "xxx.com" and block "*" if that's what you want.  

C) In 5.6.0, we introduced the "Policy-based" NGFW mode which works differently than the regular "Profile-based" mode of the past versions. Configuring the policy in this mode is slightly different than the regular "Profile-based" mode. I would think many people would find it easier to do the configuration under "Policy-based". Note that since this is a new OS release, there might still be issues to work on on some new features as we get more feedbacks. If you have a production environment that cannot tolerate downtime well, please be cautious.

https://forum.fortinet.com/tm.aspx?tree=true&m=147758&mpage=1

Flow/Proxy) It's up to your environment and the tradeoffs you are willing to accept. Proxy-mode provides more features for Web Filter and AV and also more security modules. Flow mode, while providing less, provides better performance.

[ted_barker]

Dear HoMing Thanks a lot. I wonder though if one could not do a -- custom signature to match the host name in application control together with the apt-get? instead of -- apt-get application signature -- WebFilter regex for host PS: Why does Fortigate not support host, domain, path filters but only regex? It is very easy to do it wrong and cause performance issues. With the application control custom signature you have a --host match. PS2: Is there somewhere a description what the application control apt-get does? Is it using USER AGENT match?

[hmtay_FTNT]

>>I wonder though if one could not do a  -- custom signature to match the host name in application control together with the apt-get? instead of -- apt-get application signature -- WebFilter regex for host PS: Why does Fortigate not support host, domain, path filters but only regex? It is very easy to do it wrong and cause performance issues. With the application control custom signature you have a --host match.

I forgot to mention this. Yes, it's easily do-able with the Application Control module. In that case, the HTTP and HTTPS browsers signatures do not have to be set to Allow/Monitor. All the work will be done in Application Control. 

We do support host, domain and path filters under the syntax combination of --service HTTP, --context host/body/url.

>>PS2: Is there somewhere a description what the application control apt-get does? Is it using USER AGENT match?

Yes, on the FortiGuard page, we put the description of what the signature does. And yes again, it is a User-Agent match for this signature.

HoMing

[ted_barker]

Dear HoMing, Thanks again. So you could define Application Control Security Profile with: 1) Block everything 2) add signature apt-get, set to allow 3) add custom signature to match host ubuntu.com and also set to allow WebFilter Questions: For the WebFilter not regex help is available to match part of the URL? Only in custom signature? Does Fortinet have best recommended regex for use with WebFilter to not cause issues? PS: As the user agent can be easily cheated what could be done to secure the traffic more?

[hmtay_FTNT]

Hi Ted,

>>So you could define Application Control Security Profile with: 1) Block everything 2) add signature apt-get, set to allow 3) add custom signature to match host ubuntu.com and also set to allow

Yes. Correct.

>>For the WebFilter not regex help is available to match part of the URL? Only in custom signature? Does Fortinet have best recommended regex for use with WebFilter to not cause issues?

I am not sure if I understand your question correctly. You can match part of the URL. For example, www.match.com/testing.php. You can match "testing.php". If HTTPS is used, you would need deep-inspection.

With regex, the standard practice is to avoid using '.*'. That's a match any bytes for as long as possible. In Application Control especially, it is not kind to the performance since the entire packet can be scanned compared to the Web Filter.

>>PS: As the user agent can be easily cheated what could be done to secure the traffic more?

Enable the other security modules like AV and IPS, Web Filter FortiGuard category. If a malicious client tries to download a file using the Apt-Get User-Agent, Application Control would allow it but AV, Web Filter and IPS will scan the content if they are enabled. 

You are welcome!

HoMing

[ted_barker]

Dear HoMing, This is what I meant with WebFilter URL matching. I'm used to have some of the options below in Zscaler available to me: WebFilter matching. http://2abc.abc.com:888/a..om/abc?ed.harris=5#tag2 scheme:[//[user:password@]host[:port]][/]path[?query][#fragment] What are the recommendations from Fortinet on how to define the regex that it does not is to heavy. So: 1) match any part: abc 2) match host (2abc.abc.com) contains: 3) match host begins: 4) match host ends: 5) match domain (abc.com) contains|begins|ends 6) path contains|begins|ends 7) query contains|begins|ends 8) fragment contains|begins|ends 9) path,query,fragment in total contains|begins|ends Also can FGT match against SNI if no SSL decryption done? A) in WebFilter B) in Application Control

[hasangaw]

Thanks