Technical Note: How to add a FortiGate worker blade module to an ELBCv3 cluster

[Lhuillery_FTNT]

Description
This article describes the process to add a FortiGate worker in a ELBCv3 cluster.



Scope
Chassis: FG-5060B , FG-5140B
FortiSwitch blades (Load-Balancer/HA): FS-5003B
FortiGate blades (workers): FG-5001B, FG-5101C
Latest 4.0 MR3 GA release supports ELBCv3
Solution
ELBCv3 platform is composed of a FortiSwitch and up to 12 FortiGate workers operating as a cluster (grouped by Service Group, see related article below for more information).

Config Sync Master blade

In ELBCv3 chassis, a worker blade is elected as "Config Master".
The election is done within all workers (no FortiSwitch involved) according to the 'uptime' of the worker blades.
Note: ELBCv3 has only one config master blade in the inter-chassis (2 chassis) topology designed mode.

To identify the config master blade from the FortiSwitch:

• From CLI
  fs2-ELBC # get service group status
  Service Group: 1
  ELBC Master Blade: slot-3
  Confsync Master Blade: slot-3
  Blades:
  Working:  2 [  2 Active  0 Standby]
  Ready:    0 [  0 Active  0 Standby]
  Dead:     0 [  0 Active  0 Standby]
  Total:     2 [  2 Active  0 Standby]
  Slot  3: Status:Working Function:Active
  Link:      Base: Up          Fabric: Up 
  Heartbeat: Managment: Good   Data: Good 
  Status Message:"Running"
  Slot  4: Status:Working Function:Active
  Link:      Base: Up          Fabric: Up 
  Heartbeat: Managment: Good   Data: Good 
  Status Message:"Running"
  
  
• From GUI: Service Group > Service Group 1 > Status
  
  

To identify the config master from  FGT workers, use the ‘get system status’ command:

FG1-ELBC-FTNET-b~ # get system status
Version: FortiCarrier-5001B v4.0,build3329,120705 (MR3)
…
Config-Sync: Master
FG1-ELBC-FTNET-b~ #

FG2-ELBC-FTNET-b~ # get system status
Version: FortiCarrier-5001B v4.0,build3329,120705 (MR3)
...
Config-Sync: Slave
FG2-ELBC-FTNET-b~ #

The role of the "Config Master" is:

• configuration synchronization;
• connection with FortiManager;
• distribution of FDN databases;
• firmware update distribution.

The "Config Master " blade uses the “elbc-base-ctrl” interface to communicate with the other worker blades.

Add a new worker blade - Step 1

Insert the new FortiGate worker blade into a ELBCv3 chassis slot.
Connect to the CLI using the FortiGate worker mgmt interface, enter the following command and valid the restart of the blade.

config system elbc

set mode service-group

end

Then, the FortiGate restarts in load balance mode.

Add a new worker blade - Step 2

Automatic discovery of the new FortiGate worker blade that joins the load balance cluster.
The configuration sync of the new worker blade starts with the following limitations:

• Front port configuration is not synchronized;
• elbc-mgmt VDOM interfaces are not synchronized;
• Allows the creation of dedicated out-of-band management interfaces that can be used for management (provisioning, monitoring ) logging or radius accounting.

Note: Worker blades’ user traffic interfaces (mapped to internal and external interfaces) have policies and objects automatically synchronized.

Add a new worker blade - Step 3

The automatic config sync could fail and the slave unit cannot re-join the cluster:

FG2-ELBC-FTNET-bi0514 login:
slave's configuration is not in sync with master's, sequence:3
slave's configuration is not in sync with master's, sequence:4
slave starts to sync with master
logout all admin users
slave's configuration is not in sync with master's, sequence:0
slave's configuration is not in sync with master's, sequence:1
slave's configuration is not in sync with master's, sequence:2
slave's configuration is not in sync with master's, sequence:3
slave's configuration is not in sync with master's, sequence:4

This can be checked from another worker blade with the following command:

FG1-ELBC-FTNET-b~ (global) # diagnose sys confsync status
ELBC: svcgrp_id=1, slot_id=3
ELBC HB devs:
elbc-ctrl/1: active=1, hb_count=3810
elbc-ctrl/2: active=1, hb_count=3810
ELBC mgmt devs:
elbc-base-ctrl: mgmtip_set=1
FG-5KB3E11700289, Master, uptime=3809.82, priority=0, slot_id=1:3, in_sync=1
FG-5KB3E10700068, Slave, uptime=3802.86, priority=1, slot_id=1:4, in_sync=0
elbc-base-ctrl: state=3(connected), ip=169.254.1.4, last_hb_time=3869.40, hb_nr=18983

In this case, configure manually the settings that have not been synchronized (the management vdom interfaces for example).

Add a new worker blade - Step 4

Check the new worker blade appears as "Working" state from the FortiSwitch and some new sessions are sending to this new worker blade.



Troubleshooting commands

Check one worker blade is elected 'config master'
From FS:  "get service group status"
From FGT: "get system status"

Verify the config sync of the worker blade

diag sys confsync status