strange behavior of FortiGate on SSL Websites

[sw2090]

I keep encountering this behavior:

all of a sudden on some clients https websites stop working.

every time this starts the only thing noticable on the FGT is that the memory usage is >=60%. Mostly around 63-65%. It however does not reach the threshold for conserve mode (at 70%). CPU remains between 0% and 1%.

To find more detail I had to trace this down to bare packet capturing. Flow debug or analyzer logs did not show any clue.

The packet capture on Client and also on the FGT showed that client gets stuck amidst the ssl handshake and then runs into timeout because it doesn't get any more answer from remote side.

Usually SSL Handshake starts like this

Client sends a Client hello to remote side

remote side confirms this with an ACK or SYN ACK

remote side sends a Server hello to client

Client confirms with ACK or SYN ACK.

In my case the packet capture on client show it sops after the second one. Client hello is sent and it gets confirmed by remote side with an ACK/SYN ACK.

But there is no Server hello coming in after that so client finally runs into a tmeout and resets the connection.

Packet caputure on the FortiGate shows that the Server hello does come in from remote side but for some reason the FGT does not hand it over to the client anymore.

FGT is on FortiOS 7.0.12 currently. It is a FGT 100F.

The only way to fix this is to reboot the FortiGate. After reboot memory usage is at 50-55% and everything works fine again.

To me that looks as if this is an issue with sessions, NAT and Memory usage.

Has anyone else encountered this on their FortiGates yet?

[abarushka]

Hello,

I would recommend to run in CLI "diagnose debug crashlog read" and check for wad / ipsengine process crash around the time of incident.