Latency During Fortiguard Updates

[kphed]

FGT-1500D (HA) (A-P)

FortiOS v5.4.8

IPS Extended DB Enabled

During scheduled Fortiguard updates, a constantly running UDP data stream is temporarily interrupted causing significant service disruption.  The Fortiguard updates were moved from 12 AM to 4 AM and the problem followed the time change.  Recently changed the updates from daily to weekly (Sunday @ 4 AM) and the problem now only occurs on Sunday @ 4 AM.  This issue was occurring when the device was running v5.0.10 as well.  We have worked with Fortinet Support at length with no avail. I have not been able to find any previous discussions of this type of problem but I figured a shot in the dark in the forum may yield something to go on/investigate.

Fortinet did mention the following:   "During an update, the FortiGate unit will continue to detect to scan network traffic. Sessions occurring right before an update will be scanned using the current signatures.  Sessions that occur during the update, when the signature database is reloading, will be on hold until the signatures load, at which point the new signatures are used to scan these sessions.  Sessions occurring right after the update will also use the new signatures."

I'm wondering if there is a way to modify this "holding" behavior to allow the traffic to continue passing until the IPS engine has been reloaded with the new signatures?

Any Suggestions Are Appreciated!

[Alexis_G]

Configuring fail-open

IPS is likely more important to your network than uninterrupted flow of network traffic, so the fail-open behaviour of the IPS engine is disabled by default. If you would like to enable the fail-open option, use the following syntax. When enabled, if the IPS engine fails for any reason, it will fail open. This applies for inspection of all the protocols inspected by FortiOS IPS protocol decoders, including but not limited to HTTP, HTTPS, FTP, SMTP, POP3, IMAP, etc. This means that traffic continues to flow without IPS scanning. To enable:

config ips globalset fail-open {enable | disable}end 

The default setting is disable.

See:

http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-security-profiles-54/IPS/Configure%20...

[kphed]

Thanks!  I actually started researching the same thing after posting this yesterday...at the very least, would confirm if this "holding behavior" is the root cause of the temp. latency.  We'll have to request approval but will update with results when possible.

[darwin_FTNT]

Depends on the type of traffic to improve the latency.  If the traffic don't require security utm, can create a custom application control signature to pass the session.  If the session is bypassed by IPS, kernel no longer forward packets from the session to ipsengine.  See 'diagnose sys session list' and check the state bits:

session info: proto=6 proto_state=01 duration=516100 expire=3562 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=5  origin-shaper=  reply-shaper=  per_ip_shaper=  ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255  state=log may_dirty ndr app_valid <---------- ndr means forwarded to ipsengine queue.  If session is ips by-passed, this is cleared.

Another option to do an early IPS bypass of session is to enable intelligent-mode (default is enabled):

config ips global     set database extended     set traffic-submit enable     set intelligent-mode disable <-------------- set to enable, end

See link for more info on IPS intelligent mode config:

http://kb.fortinet.com/kb/documentLink.do?externalID=FD39369

In the later FOS version, v5.4 or higher I think.  The ipshelper process helps pre-process configuration and database updates and push this binary data directly to all ipsengine process.  This would help minimize traffic latency.  The ipsmonitor process controls and spawns both these ipshelper and ipsengine processes.

Need more specific info on environment setup to measure latency and perform improvements/optimizations.