Description
This article describes a few steps and information on how to collect information to match the virtual mac address in use between virtual cluster units vs vdoms and virtual interfaces.
FGCP (FortiGate Clustering Protocol) is designed to provide hardware failure protection between two instances of one or more vdoms operating on two FortiGates in a virtual cluster.
When a cluster is operating in NAT mode, different virtual MAC addresses are assigned to each primary unit's physical and logical interfaces.
VLAN subinterfaces receive the same virtual MAC as the parent physical interface it's a child of.
Scope
The example below shows 2 FortiGates configured in HA Active-passive mode, set with 2 vdoms in NAT mode and having Virtual clustering enabled.
Different vdoms are set to be primary on different virtual cluster partitions so both hardware resources are used.
The root vdom is also in NAT mode.
To verify a list of vdoms and the command 'diag sys vd stats' can be used in the Global context.
The example below lists the VC1VD004 before VC1VD003 and that's because of the index number.
The command 'diag sys vd list' should provide detailed information where verification of the index of each vdom and that the index number sorts and controls a few other aspects as seen below.
Using 'get system ha status' the configuration status can be verified which should always be 'in-sync' being updated frequently.
Notice the Virtual Cluster information where each hardware is primary for a determined virtual cluster containing one or more vdoms.
virtual cluster 1 = id 0 ( bit 0x0 )
virtual cluster 2 = id 1 ( bit 0x1 )
The GUI Dashboard should show the same output as above under System - > HA.
Using 'show system interface <interface name>' one can verify the interface settings and the vdoms to which the interfaces belong.
The parent and physical interface port1 is assigned to the root vdom.
To verify the physical virtual and virtual MAC address from the physical interfaces, use 'diag sys ha mac' in the global context.
Notice that interfaces can only be linked to one vdom at a time, however, one vdom might not be linked to a virtual context ( Virtual Cluster ) where the primary unit is subject to change.
Let's note down the port1 phy_index and how the cluster controls the virtual mac between Primary and Secondary units.
To verify the MAC address that's being sent in ARP replies on logical interfaces, use the command 'fnsysctl ifconfig <interface name>'.
Let's notice that the VLAN interfaces inherit the virtual mac from its parent's physical interface.
The FGSP default virtual MAC prefix is '00-09-0f-09-<group-id_hex>-(<vcluster_integer> + <idx>)'
- port1 is using a virtual mac because it's a member from the root vdom
- the root vdom is Primary on FortiGate-1100E-1 unit
- port1 is using the physical mac address on the Secondary unit
- vlan10 is using a virtual mac because it's a member from the VC1VD001 vdom
- the VC1VD001 vdom is Primary on FortiGate-1100E-1 unit
- vlan10 is using the physical mac address on the Secondary unit
- vlan30 is using a virtual mac because it's a member from the VC1VD002 vdom
- the VC1VD002 vdom is Primary on FortiGate-1100E-2 unit
- vlan30 is using the physical mac address on the Secondary unit
- vlan30 has +8 hex added to the virtual mac's last digit because it's active on the virtual cluster
- Interfaces on 'Secondary' units do use the physical mac address
- Secondary and passive units (Active-passive) do not respond to network traffic.
Solution
Calculating the virtual MAC manually to verify the proper functionality:
00-09-0f-09-<group-id_hex>-(<vcluster_integer> + <idx>)
|| || ||
|| || ||
|| || ==> diag sys ha mac (ID = 2)
|| ============> get system ha status ( 0 or 8 )
==========================> get system ha status ( ID = 9 )
So when the vdom is Primary on vcluster1, the mac address for port1 should be: 00-09-0f-09-09-02
If a vdom is Primary on vcluster2, the mac address should receive +8 (0x1000 0000) like this: 00-09-0f-09-09-82
The Switch must keep track of mac addresses per VLAN and when querying the switch's MAC table, users notice that moving VC1VD001 to the virtual cluster 2 causes the mac address to change to 00-09-0f-09-09-82.
On FortiSwitch units, this can be checked using 'diag switch mac-address list'
During a hardware failure where FortiGate-1100E-1 is down, for example, all vdoms would be moved to FortiGate-1100E-2 and the same virtual MAC will be used allowing a seamless transition to occur.
From the workstation's perspective, as vlan10 is active on virtual cluster1, it will learn the virtual mac address 00-09-0f-09-09-02.
However, from the workstation behind vlan30 which is active on virtual cluster2, it'll learn the virtual mac address 00-09-0f-09-82.
Conclusion: when manually transitioning a vdom between vcluster1 and 2, the MAC address might change and this maneuver is considered a 'configuration change' and not a fail condition.
For more information check the Do's and Don'ts for Virtual Clusters.
Related KB Articles:
