IPS Engine CPU load - ssl deep inspection

[mcdaniels]

Hi,

I wonder if none of you is having issues with the IPS-Engine (flow mode) on Forti-OS 6.2.x (6.2.3) and CPU-load? We have a huge problem (on a FGT 60F and a FGT 100D), after installing Forti-OS 6.2.x:

When activating SSL-Deep-Inspection for our outgoing policies, the first thing is that some sites (HTTPS) do not open on the first attempt, but when reloading the site. (On the first attempt there is a ssl error page, the second attempt then works). (We have imported the SSL certificate from the FGT to the clients browsers).

The second BIG issue is, that 3 cores are sometimes freaking out (99.9% CPU load caused by ips engine). If this happens, nothing works. (Internet is dead). After a few seconds /minutes, the ips engine goes back to normal load.

I am in contact with FGT support and we try to hunt down the problem, which is definitly caused by IPS-Engine & flow-mode & ssl-deep-inspection. After analyzing the logs (the ips engine seems to crash  - after not reacting for 30sec - and got restarted) During these 30 seconds nothing (I mean the Internet) is working. We are still examining debug-logs with FGT support.

When the 99% CPU is happening, there is no high amount of open sessions (900-1500) and no massive throughput. (Last time we got a throughput of 2Mbit (all policies summed up).

The only workaround for us is only activationg ssl-certificate-inspection, which is a massive security problem, because SSL-connections will not be scanned for e.g. viruses.

Hope to get it sorted.

[Toshi_Esumi]

Hove you tried downgrading them to 6.0.x? I'm assuming at least the 100D was upgraded from 6.0.x before 6.2.3. So that would be the first option. For 60F, although 6.0.x still seems to support it, you probably don't have a backup config with the older version, and it's newly released hardware, so it might be risky causing more problems if you doengrade it.

[mcdaniels]

Hi,

thanks for answering. We got a second FGT 60F. On this one I tried to downgrade from 6.2.3 to 6.0.9 (after importing the config). There are a lot of issues after that. So, you are right: It is risky ;)

Unfortunatly in  6.0.x there is no possibility to choose flow/proxymode per policy. This only works on 6.2.x (as I have examined).

If we use proxymode on all policies the memory consumption will go over the limit (conserve mode).

So we are in some kind of "no forward, no back - situation".

[tanr]

Possibly related to a couple known bugs with MSS/MTU in 6.2.3? 

Per https://forum.fortinet.com/tm.aspx?m=181463&mpage=3 make sure you're setting tcp-mss-sender and tcp-mss-receiver in your deep inspection security policy.

[Toshi_Esumi]

Also FTNT SE warned me possible performance/cpu usage issue with "per-policy" inspection mode usage saying use it carefully. It doesn't help in your case though. Hate to say this but you might need to wait at least until 6.2.4 comes out unless TAC can find some config issues.