Deploying FortiWeb VM in HA - Our findings (might be useful, would appreciate feedback)

[kelv1n]

Hi Guys

I'm in the process of rolling out 2 FortiWeb VM's in HA to replace some aging Barracuda Load Balancers. Unfortunately documentation is a little vague/dispersed and there is no clear cut "How To" or "Cookbook" when compared to other Fortinet products for deploying the VM's in HA. So I thought I'd share some details that other new customers might find useful.

we're still in the process of configuring, so apologies if some of the terminology or facts are incorrect, I'd really appreciate any feedback on correction, improvements, problems or thing that might cause a security concern -

How HA works on FortiWeb - FortiWeb HA is only Active-Passive, each unit must be configured separately, each with its own IP. Once you configure HA, the Secondary unit will become a slave to the Main unit, it will lose the config you gave it including IP,  so it can only be managed directly from the Main unit. This differs to some other manufacturers such as Barracuda where each devices retain the IP you gave them. 

Configure HA first or with a Clean config - We initially deployed a single VM for PoC, which we configured up, we then decided to move this to production and simply add a second VM and place it into HA. When we setup HA, the 2 VM's had a conflict and the interfaces and VIP's started flip flopping. After getting no joy from Fortinet support, we factory reset the 2 VM's, with a clean config HA worked immediately. This probably could have been resolved through tweaking the config, but you could save yourself some time by just setting up the HA at the very beginning.

Configure your Virtual Network - We tested with various settings, but it turns out the default VMWare configure should be sufficient. Though we found we had to set Mac Address Changes to "Accept" on the "VM Network" and ports which handles the VIPs.  We tried disabling this as an experiment, but the IPs and FortiWebs became unreachable. (Any feedback on this please, does anybody know if this is required? Or if we're doing something wrong?)

Flapping on member rejoining - We found that when a member was rebooted or just rejoining the group, the IP's appeared to start flapping, which caused long period (3-5 minutes) of interruption and intermittent ping losses. This appears to be caused by a combination of settings, which I'm still trying to fathom and find the best. But what we found was

[ol]

Monitor ALL ports that are handling VIPs and traffic (Not just the uplink port), we found unmonitored ports took longer to stabilize.

Increase the "ARP Packet Numbers", default is 3, we increased this to 10 and then 16 (Ensure you understand the consequences of this though).

Stop the returning member from attempting to take control so quickly through changing the time it waits to 60 seconds, this needs to be done through the CLI and running[/ol]

config system ha
set boot-time 60
end

Through doing this we've managed to get HA stable, flip flopping reduced from the mention 3-5 minutes intermittent disruption to availability, down to a couple of ping long loses.

UPDATE -

We discovered a configuration issue on our Firewalls, they were not Source-NATing outbound traffic from the FortiWebs correctly, I can't see why this would cause flapping on internal subnets. But after fixing this we undid the settings above, and failover has been painless since. Though I'll leave the notes in, as they may be useful to somebody else.

[Courtney_Schwartz]

Excellent. Let us know if there's anything else. :)

[Pavel_Rovnov]

Hi All,

I found the answer to my questions in FortiWeb-VM_5_5_Patch_1_Install_Guide_Revision2.pdf

FortiWeb-VM_5_5_Patch_1_Install_Guide_Revision2.pdf wrote:

FortiWeb-VM requires an Internet connection to periodically re-validate its license. It cannot be evaluated in offline, closed network environments such as isolated labs. If FortiWeb-VM cannot contact Fortinet’s FortiGuard Distribution Network (FDN) for 24 hours, access to the web UI and CLI will be locked. The web UI may display a message such as: License has been uploaded. Please wait for authentication with registration servers.

Can anyone confirm that traffic processing is not affected by UI and CLI lock?

Thanks!