Clustering the fortigates

Hi ​@Sridharan_Lenovo 

So all 3 FGT will be on different DC or on same DC?

You can refer the best practice for HA:

Am i good to leverage HSRP across these 3 Fortigates without using FGCP and use only FGSP for session synchronization .

The bottom line with FGCP HA is any new session needs to be handled by the primary unit at least initially. So other secondary units can’t take any new incoming session packets independently, even active-active setup.
This condition would make FGCP not an option for most of multi-location set up.

Toshi

The premise is broken on two levels, not just "watch the latency"

1. The hardware mix kills FGCP outright. You listed a 101F and a 400F. FGCP requires identical models and identical firmware in a cluster. A 101F will not cluster with a 400F. No amount of tuning hb-interval or the dead timer changes this, because the cluster never forms in the first place. The generic HA best practices posted earlier do not apply, because the underlying assumption does not hold.

2. FGCP is the wrong tool for one unit per DC anyway. The FGCP heartbeat is L2 (a non-routable ethertype), so you would need stretched L2 between the DCs (VXLAN/EVPN/VPLS/dark fibre). And even if you build that, the point Toshi raised is the decisive one: in FGCP, including active-active, only the primary accepts new ingress sessions. The secondaries do not do independent local breakout at their own site. So a true "each DC terminates its own traffic" design is architecturally impossible with FGCP. A stretched FGCP cluster across three DCs is an anti-pattern.

On the HSRP question, there is a wrong assumption baked in

FortiGate does not support HSRP. That is a Cisco protocol. On FortiGate the equivalent is VRRP, which is where you were correctly redirected. But VRRP does not solve the multi-DC problem either: VRRP routers share a virtual IP and virtual MAC inside the same broadcast domain, so VRRP over WAN without an L2 stretch does not work. VRRP is useful inside a single DC, not across three.

The correct design for 3 DCs, one unit per DC, mixed models

Run standalone FortiGates per site, with two building blocks:

• Dynamic routing (BGP preferred) for path selection and failover. Routing decides which DC attracts which ingress and how traffic reroutes when a site fails. No shared virtual-IP construct is needed.
• FGSP for session synchronisation. This is exactly what FGSP is built for: independently configured FortiGates, asymmetric routing across sites, with session state synced so a failover does not drop established sessions.

Why FGSP fits this case and resolves the model issue:

• FGSP runs over L3 (peer-ip) by default, so it is routable between subnets and DCs. No L2 stretch required.
• FGSP tolerates different models, so a 101F alongside a 400F is fine. This is precisely what FGCP cannot do.
• With L3 unicast standalone config sync, peers can even sit in different subnets, which also covers cloud scenarios without L2.

Caveats that belong in any serious answer

• Firmware consistency: since 7.2.1 there is a group-id in the protocol header, so 7.2.1+ units will not sync with older ones. Keep all units on the same major firmware.
• Asymmetry: FGSP is designed for asymmetric routing, but UTM/IPS inspection on asymmetric paths needs deliberate planning (which node inspects, offload behaviour).
• NP offload: check session offload behaviour in FGSP setups, or more lands in CPU than expected.

Bottom line: Toshi is correct, the rest of the thread is turning the wrong screw. The answer the OP actually needs is: drop FGCP (model mix and L2 heartbeat), HSRP does not exist on FortiGate, VRRP does not solve multi-DC, and the clean solution is standalone plus BGP plus FGSP.

Want me to tighten this into a single posted reply with the FGSP doc link, or keep it as this fuller breakdown?

Thanks ​@Toshi_Esumi ​@christian_89_  for the briefed explanations. I would like to mention that i am not gona have a crisscross HA setup(in terms of firewall models) , 3 * 101 Fs and 3 * 401F which will be placed in 3 different DCs . 101F will be forming a seperate cluster and 401Fs will be a seperate cluster.i am going to have a L2 stretched network between these DC’s as we are catering same services/setup across the 3 DCs. I am following this Doc in here : https://docs.fortinet.com/document/fortigate/7.6.4/administration-guide/151840/distributed-ha-clusters. Feel to free to comment out any caveats on the above to use FGCP across DC’s.

If the question yes/no, than - yes, distributed cluster works. I can say only from my experience, which is - I am ISP, and due to small country size latency/packet loss for heartbits is not an issue. So, our clients have distributed cluster members between 2 DCs, distance about 65-70 km, and L2 stretched by us between them. And of course the mode is A/P, haven’t seen A/A in any configuration yet. The latency in such set up is circa 10-13 msec. And it just works, no tricks were needed. A good monitoring of connectivity is a must - one client had faulty switch connecting sites which disrupted heartbits and cluster did enter split brain mode.