Technical Tip: Configure virtual cluster on FortiGate HA cluster

FortiGate virtual clustering is an additional extension of FortiGate HA FGCP, which allows multiple virtual clusters to be formed between HA members.

The diagram below shows how VDOM 'root' and 'vdom1' are active on unit1, while vdom2 is active on unit2.

Before configuring FGCP, it is first recommended to configure a normal HA cluster where vdom1/vdom2/root are active on unit1 and then form the FGCP.

Example configuration for FGCP:

Unit1 HA config:

config system ha

    set group-id 111

    set group-name "lab_name"

    set mode a-p

    set password ENC xxxx

    set hbdev "port2" 0

    set vcluster-status enable

        config vcluster

            edit 1

                set override enable

                set priority 150

                set monitor "port1" "port3" "port4"

                set vdom "root" "vdom1"

            next

            edit 2

                set override enable

                set priority 110

                set monitor "port1" "port3" "port4"

                set vdom "vdom2"

            next

        end

    set unicast-hb enable

    set unicast-hb-peerip 10.183.21.85

end

Unit2 HA config:

config system ha

    set group-id 111

    set group-name "lab_name"

    set mode a-p

    set password ENC xxxx

    set hbdev "port2" 0

    set vcluster-status enable

        config vcluster

            edit 1

                set override enable

                set priority 120

                set monitor "port1" "port3" "port4"

                set vdom "root" "vdom1"

            next

            edit 2

                set override enable

                set priority 100

                set monitor "port1" "port3" "port4"

                set vdom "vdom2"

            next

        end

    set unicast-hb enable

    set unicast-hb-peerip 10.183.22.195

end

For non VM clusters, the 'set unicast-hb enable' and 'set unicast-hb-peerip x.x.x.x' can be skipped, as this is recommended only for VM HA clusters.

Both virtual clusters have 'set override enable', this helps to control which vdom on which device to be active.

Because unit1 has higher priority for vcluster1 and vcluster2, all vdoms (root, vdom1 and vdom2) are active on that device.

To trigger manual failover and move for example vdom2 to unit1, change the priority for vcluster2 on unit2 from 100 to 130.

HA status before changing the priority of unit2 for vcluster2 from 100 to 130:

Unit1:

get system ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: HA A-P
Group Name: lab_name
Group ID: 111
Debug: 0
........
number of member: 2
unit1 , FGVM010000137212, HA cluster index = 1
unit2 , FGVM010000137259, HA cluster index = 0
number of vcluster: 2
vcluster 1: work 10.183.22.195 <-----
Primary: FGVM010000137212, HA operating index = 0 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----
vcluster 2: work 10.183.22.195  <-----
Primary: FGVM010000137212, HA operating index = 0 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----

Unit2:

unit2 (global) # get system ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: HA A-P
Group Name: lab_name
Group ID: 111
Debug: 0
........
number of member: 2
unit2 , FGVM010000137259, HA cluster index = 0
unit1 , FGVM010000137212, HA cluster index = 1
number of vcluster: 2
vcluster 1: standby 10.183.22.195 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----
Primary: FGVM010000137212, HA operating index = 0 <-----
vcluster 2: standby 10.183.22.195 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----
Primary: FGVM010000137212, HA operating index = 0 <-----

Changing the priority for vclsuter2 on unit2 from 100 to 130.

Unit1:

get system ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: HA A-P
Group Name: lab_name
Group ID: 111
Debug: 0
........
number of member: 2
unit1 , FGVM010000137212, HA cluster index = 1
unit2 , FGVM010000137259, HA cluster index = 0
number of vcluster: 2
vcluster 1: work 10.183.22.195 <-----
Primary: FGVM010000137212, HA operating index = 0 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----
vcluster 2: standby 10.183.21.85 <-----
Secondary: FGVM010000137212, HA operating index = 1 <-----
Primary: FGVM010000137259, HA operating index = 0 <-----

Unit2:

unit2 (global) # get system ha status
HA Health Status: OK
Model: FortiGate-VM64-KVM
Mode: HA A-P
Group Name: lab_name
Group ID: 111
Debug: 0
........
number of member: 2
unit2 , FGVM010000137259, HA cluster index = 0
unit1 , FGVM010000137212, HA cluster index = 1
number of vcluster: 2
vcluster 1: standby 10.183.22.195 <-----
Secondary: FGVM010000137259, HA operating index = 1 <-----
Primary: FGVM010000137212, HA operating index = 0 <-----
vcluster 2: work 10.183.21.85 <-----
Primary: FGVM010000137259, HA operating index = 0 <-----
Secondary: FGVM010000137212, HA operating index = 1 <-----

It is not recommended to trigger a manual failover using the command 'execute ha failover set 1'. For manual failover, change the priority or 'diagnose sys ha reset-uptime'.

The command ' diagnose sys ha status'  also can be used to check the HA and vcluster status on both units:

Unit1:

diagnose sys ha status
HA information
Statistics
traffic.local = s:0 p:76483 b:43141396
traffic.total = s:0 p:76484 b:43141436
activity.ha_id_changes = 7
activity.fdb = c:0 q:0

Model=80005, Mode=2 Group=111 Debug=0
nvcluster=2, ses_pickup=0, delay=0

[Debug_Zone HA information]
HA group member information: is_manage_primary=1.
FGVM010000137212: Primary, serialno_prio=1, usr_priority=150, hostname=unit1
FGVM010000137259: Secondary, serialno_prio=0, usr_priority=120, hostname=unit2

[Kernel HA information]
vcluster 1, state=work, primary_ip=10.183.22.195, primary_id=0, silent=0
FGVM010000137212: Primary, ha_prio/o_ha_prio=0/0
FGVM010000137259: Secondary, ha_prio/o_ha_prio=1/1

vcluster 2, state=standby, primary_ip=10.183.21.85, primary_id=1, silent=0
FGVM010000137212: Secondary, ha_prio/o_ha_prio=1/1
FGVM010000137259: Primary, ha_prio/o_ha_prio=0/0
Silent vcluster bitmap=00000000000000000000000000000000

Note: Run the following commands on the FortiGate devices for verification checks.

get system ha status: To confirm cluster formation and primary/secondary roles per virtual cluster.

diagnose sys ha checksum show: To verify configuration synchronization.

diagnose sys vd list: To confirm VDOMs are assigned to the correct virtual cluster.

Limitations:

1. FortiGate HA vcluster does not support VDOM links between active VDOMs on different devices. VDOM link can be configured only between root and vdom1, but not between vdom1 and vdom2 or root and vdom2.
2. In an HA virtual cluster, every VLAN must be configured in the same virtual cluster as its parent interface or LAG. For example, if VDOM vdom1 has a VLAN20, its parent interface must also be part of VDOM vdom1. The same applies to LAG interfaces: if vdom1 has LAG (port3-port7), building a VLAN on top of that LAG is only possible for a vdom1.