timeout issue with ssh

Hi, this is a one of the strangest issue I've ever had.  I have two offices that are connected over S2S vpn using two Fortigate.  The VPN tunnels is up and stayed up.  I can GUI into Fortigate on both sites.  However, when I SSH into a switch that is sitting behind the FW, it disconnects me if I don't type anything for 5-7s.  At first, I though it was a settings on the Cisco switch.  However, this also happens when I SSH into Fortigate FW.  It seems like there is a setting on the Fortigate.  Any suggestions?   

Thanks.