I am running ver. 4 MR1 Patch 2. I can confirm that the web interface can be woefully slow. I only allow access via https. I cannot say that I have found any performance issue with ssh.
I have a call ticket on this issue with Fortinet but as of yet we have not come up with a definitive answer to this problem. I will say from experimentation that in my case, an Active-Active HA, that when I turn off one of the units gui admin access improves dramatically even when the CPU is pegged over 80%. Given the amount of traffic we have, I cannot maintain this solo operation for very long.
I had assumed, due to certificate issues with banking sites and IE7 & IE8 in the version 4 code (these versions are using a rare 128-bit certificate), that my problems were caused by some strange anomaly with https & HA, but from your comments, it would seem that some of you are simply using http.
Can you confirm as it will be helpful. I will certainly alert my support engineer to this thread. I would also encourage you, if you have not done so, to open a call ticket with Fortinet.
Victor