Internet dies when any layer 7 is applied to policies

[MikePruett]

Regardless of the protocol, the ability to pass traffic stops the moment we apply layer 7 on a rule.

For instance, if we apply any layer 7 to the HTTP rule, web dies. This is a fully registered and active 90D HA Cluster.

Have you guys ever seen this happen before?

[Christopher_McMullan]

Yes, but in a wide variety of contexts, for a decent number of reasons.

Do any of the UTM profiles or the policy itself have logging enabled? What logs get generated after applying the Layer-7 inspection?

Could you provide the CLI configuration of the proxy options, and a relevant UTM profile, and the policy itself?

[MikePruett]

Christopher McMullan_FTNT wrote:

Yes, but in a wide variety of contexts, for a decent number of reasons.

Do any of the UTM profiles or the policy itself have logging enabled? What logs get generated after applying the Layer-7 inspection?

Could you provide the CLI configuration of the proxy options, and a relevant UTM profile, and the policy itself?

All policies have logging. They show the traffic as allowed but it goes from the normal allow deny action to timeout action.

Standard UTM profiles that we use on all of our Fortigates that we deploy. Works on this version of code all the time. Literally no changes.

Will see about grabbing a CLI print out

[MikePruett]

Christopher McMullan_FTNT wrote:

Yes, but in a wide variety of contexts, for a decent number of reasons.

Do any of the UTM profiles or the policy itself have logging enabled? What logs get generated after applying the Layer-7 inspection?

Could you provide the CLI configuration of the proxy options, and a relevant UTM profile, and the policy itself?

Firewall Policy Affected:

BATTLECREEK2 (policy) # edit 10

BATTLECREEK2 (10) # get policyid : 10 srcintf: == [ Inside ] name: Inside dstintf: == [ Outside ] name: Outside srcaddr: == [ all ] name: all dstaddr: == [ all ] name: all rtp-nat : disable action : accept status : enable schedule : always schedule-timeout : disable service: == [ SSH ] name: SSH utm-status : disable logtraffic : all logtraffic-start : disable capture-packet : disable auto-asic-offload : enable wanopt : disable webcache : disable session-ttl : 0 wccp : disable fsso : disable disclaimer : disable natip : 0.0.0.0 0.0.0.0 match-vip : disable diffserv-forward : disable diffserv-reverse : disable tcp-mss-sender : 0 tcp-mss-receiver : 0 comments : block-notification : disable custom-log-fields: tags: replacemsg-override-group: srcaddr-negate : disable dstaddr-negate : disable service-negate : disable timeout-send-rst : disable identity-based : disable traffic-shaper : traffic-shaper-reverse: per-ip-shaper : nat : enable permit-any-host : disable permit-stun-host : disable fixedport : disable ippool : disable central-nat : disable

Example of Layer 7 being Applied (webfilter monitor only policy)

BATTLECREEK2 (MonitorOnly) # get name : MonitorOnly comment : replacemsg-group : inspection-mode : proxy options : block-invalid-url https-url-scan https-replacemsg : enable ovrd-perm : bannedword-override urlfilter-override fortiguard-wf-override contenttype-check-override post-action : comfort override: ovrd-scope : user profile-type : list ovrd-dur-mode : constant ovrd-dur : 15m ovrd-user-group: == [ ] name: profile: web: bword-threshold : 10 bword-table : 0 urlfilter-table : 1 --More-- content-header-list : 0 --More-- safe-search : --More-- log-search : enable keyword-match: ftgd-wf: options : error-allow http-err-detail rate-image-urls rate-server-ip category-override : g22 Local Categories 140 custom1 141 custom2 exempt-ssl : exempt-quota : ovrd : filters: == [ 142 ] id: 142 == [ 141 ] id: 141 == [ 83 ] --More-- id: 83 == [ 5 ] id: 5 == [ 1 ] id: 1 == [ 6 ] id: 6 == [ 12 ] id: 12 == [ 3 ] id: 3 == [ 4 ] id: 4 == [ 62 ] id: 62 == [ 59 ] id: 59 == [ 7 ] id: 7 == [ 9 ] id: 9 == [ 64 ] --More-- id: 64 == [ 2 ] id: 2 == [ 15 ] id: 15 == [ 11 ] id: 11 == [ 66 ] id: 66 == [ 57 ] id: 57 == [ 13 ] id: 13 == [ 8 ] id: 8 == [ 14 ] id: 14 == [ 63 ] id: 63 == [ 67 ] id: 67 == [ 65 ] --More-- id: 65 == [ 16 ] id: 16 == [ 24 ] id: 24 == [ 19 ] id: 19 == [ 75 ] id: 75 == [ 76 ] id: 76 == [ 72 ] id: 72 == [ 25 ] id: 25 == [ 26 ] id: 26 == [ 61 ] id: 61 == [ 86 ] id: 86 == [ 17 ] --More-- id: 17 == [ 29 ] id: 29 == [ 18 ] id: 18 == [ 77 ] id: 77 == [ 82 ] id: 82 == [ 71 ] id: 71 == [ 85 ] id: 85 == [ 54 ] id: 54 == [ 30 ] id: 30 == [ 28 ] id: 28 == [ 58 ] id: 58 == [ 20 ] --More-- id: 20 == [ 40 ] id: 40 == [ 33 ] id: 33 == [ 69 ] id: 69 == [ 34 ] id: 34 == [ 55 ] id: 55 == [ 35 ] id: 35 == [ 36 ] id: 36 == [ 70 ] id: 70 == [ 87 ] id: 87 == [ 48 ] id: 48 == [ 80 ] --More-- id: 80 == [ 38 ] id: 38 == [ 78 ] id: 78 == [ 39 ] id: 39 == [ 79 ] id: 79 == [ 42 ] id: 42 == [ 37 ] id: 37 == [ 44 ] id: 44 == [ 46 ] id: 46 == [ 47 ] id: 47 == [ 68 ] id: 68 == [ 23 ] --More-- id: 23 == [ 53 ] id: 53 == [ 49 ] id: 49 == [ 31 ] id: 31 == [ 43 ] id: 43 == [ 51 ] id: 51 == [ 52 ] id: 52 == [ 50 ] id: 50 == [ 41 ] id: 41 == [ 81 ] id: 81 == [ 56 ] id: 56 == [ 84 ] --More-- id: 84 == [ 88 ] id: 88 quota: max-quota-timeout : 300 extended-utm-log : enable log-all-url : disable web-content-log : disable web-filter-activex-log: disable web-filter-command-block-log: enable web-filter-cookie-log: disable web-filter-applet-log: disable web-filter-jscript-log: disable web-filter-js-log : disable web-filter-vbs-log : disable web-filter-unknown-log: disable web-filter-referer-log: disable web-filter-cookie-removal-log: disable web-filter-sdns-action: redirect web-filter-sdns-portal: 0.0.0.0 web-url-log : disable web-invalid-domain-log: disable --More-- web-ftgd-err-log : enable web-ftgd-quota-usage: disable

[Christopher_McMullan]

I can explain away 'set utm-status disable', since it's probably disabled in production to avoid another outage, but the service of the policy is SSH only. Are you showing the right rule that blocks traffic when a webfilter profile is applied?

[MikePruett]

Christopher McMullan_FTNT wrote:

I can explain away 'set utm-status disable', since it's probably disabled in production to avoid another outage, but the service of the policy is SSH only. Are you showing the right rule that blocks traffic when a webfilter profile is applied?

Sorry, I was looking at the SEQ # and not the ID# when pulling the policy. This is the policy is does it to. I even created a wide open dummy webfilter profile and applied it for shits and giggles to see if it was my security profile but it wasn't. (that is the one that you see applied to this rule currently

BATTLECREEK2 (22) # get policyid : 22 srcintf: == [ Inside ] name: Inside dstintf: == [ Outside ] name: Outside srcaddr: == [ all ] name: all dstaddr: == [ all ] name: all rtp-nat : disable action : accept status : enable schedule : always schedule-timeout : disable service: == [ HTTP ] name: HTTP utm-status : enable logtraffic : all logtraffic-start : disable capture-packet : disable auto-asic-offload : enable wanopt : disable webcache : disable session-ttl : 0 wccp : disable fsso : disable disclaimer : disable natip : 0.0.0.0 0.0.0.0 match-vip : disable diffserv-forward : disable diffserv-reverse : disable tcp-mss-sender : 0 tcp-mss-receiver : 0 comments : block-notification : disable custom-log-fields: tags: replacemsg-override-group: srcaddr-negate : disable dstaddr-negate : disable service-negate : disable timeout-send-rst : disable identity-based : disable profile-type : single av-profile : webfilter-profile : test_policy spamfilter-profile : dlp-sensor : ips-sensor : application-list : voip-profile : icap-profile : profile-protocol-options: default deep-inspection-options: traffic-shaper : traffic-shaper-reverse: --More-- per-ip-shaper : nat : enable permit-any-host : disable permit-stun-host : disable fixedport : disable ippool : disable central-nat : disable

BATTLECREEK2 (22) #

[MikePruett]

ran a "get webfilter status" and it shows as not running even though the fortigate is licensed and registed. assume this has something to do with it. Possible Fortigaurd DB issues on the device? Status page shows it as licensed and registered so it should be good.

[Christopher_McMullan]

That may be a false positive when the webfilter profile isn't applied to a single policy. I can also get the same result when I remove UTM from my testing appliance:

McFortiGate # get webfilter stat Locale : english

The service is not enabled.

McFortiGate # di de rating Locale : english

The service is not enabled.

McFortiGate #

Especially the second command, 'diag debug rating', should return a list of resolved IPs for FortiGuard Web Filtering servers.

If you can stomach the outage, or could craft a policy with low impact on production traffic, could you apply a WF profile with FortiGuard web filtering enabled to at least one firewall policy and run the two diagnostics again?

get webfilter status

diag debug rating