Technical Note: Missing logs - How to migrate former standalone FortiGate devices to HA Cluster on FortiAnalyzer

[mbernatek]

Description

This article describes how to merge logs for standalone FortiGate devices when HA Cluster is created on FortiAnalyzer.

Solution

When an HA Cluster is created on FortiAnalyzer the logs are not automatically merged.  It can be spotted after some time that some logs are not displayed in FortiView.  The procedure for proper log migration is given below.

If the auto migrate option is enabled (disabled by default) then the FortiAnalyzer system automatically merges logs of former standalone devices to one virtual HA Cluster device on FortiAnalyzer.

This option can be enabled by command on the FortiAnalyzer:

config system log settings
    set ha-auto-migrate enable
end

Prerequisites

• Two FortiGate devices are registered as standalone on FortiAnalyzer with own logs
• Both devices were put to the HA cluster (FortiGate configuration)
• On the FortiAnalyzer they are still both as standalone devices

Device and log store status before merge:

# diagnose dvm device list
TYPE OID      SN               HA  IP              NAME                                 ADOM                                 FIRMWARE
faz enabled 361      FGVM01000xxxxxx3 -   10.108.16.150   FGVMDH-40                            lab                                  5.0 MR2 (727)
|- STATUS: db: unknown; conf: unknown; cond: unknown; dm: none; conn: unknown
|- vdom:[3]root flags:0 adom:lab pkg:[never-installed]

faz enabled 349      FGVM01000yyyyyy5 -   10.108.16.51    FGVMDH-42                            lab                                  5.0 MR2 (727)
|- STATUS: db: unknown; conf: unknown; cond: unknown; dm: none; conn: unknown
|- vdom:[3]root flags:0 adom:lab pkg:[never-installed]

# execute log device logstore list
Device ID           logfiles           archive files      status
==================================================================
(1) FGVM01000xxxxxx3          0MB          0MB          active.
(2) FGVM01000yyyyyy5          0MB          0MB          active.

In GUI go to Device Manager > All FortiGates, select (right click) and edit one FortiGate device.  Enable option "HA Cluster" and pick up the second FortiGate device.  This device may be renamed. With this step the system will merge logs from both devices to the cluster automatically.

The quota limit of cluster mode must be at least the sum of standalone limits of each FortiGate device.

Device and log store status after merge:

# diagnose dvm device list

TYPE OID      SN               HA  IP              NAME                                 ADOM                                 FIRMWARE
faz enabled 349      FGVM01000yyyyyy5 -   10.108.16.150   FGVMDH-42                            lab                                  5.0 MR2 (727)               
|- STATUS: db: unknown; conf: unknown; cond: unknown; dm: none; conn: unknown               
|- vdom:[3]root flags:0 adom:lab pkg:[never-installed]

faz enabled 361      FGVM01000xxxxxx3 a-p 10.108.16.150   lab-ha                               lab                                  5.0 MR2 (727)               
|- STATUS: db: unknown; conf: unknown; cond: unknown; dm: none; conn: unknown
        HA cluster member: FGVM01000xxxxxx3 (slave 0)
        HA cluster member: FGVM01000yyyyyy5 (slave 1)               
|- vdom:[3]root flags:0 adom:lab pkg:[never-installed]

-# execute log device logstore list
      Device ID           logfiles           archive files      status
==================================================================
(1) FGVM01000xxxxxx3        N/A        N/A    zombie
(2) FGVM01000yyyyyy5        N/A        N/A    zombie
(3) FGHA001238487465_CID          0MB          0MB          active.

The log store output is the initial status after the change. Wait for the final output below when all log files are merged.


# execute log device logstore list
      Device ID           logfiles           archive files      status
==================================================================
(1) FGHA001238487465_CID          0MB          0MB          active.


All log files are now moved to the HA Cluster folder. The SQL database will now be rebuilt, but this can take some time.  The former standalone FortiGate device left in ADOM after migration can now be deleted.

The final device list:

# diagnose dvm device list

TYPE OID      SN               HA  IP              NAME                                 ADOM                                 FIRMWARE
faz enabled 361      FGVM01000xxxxxx3 a-p 10.108.16.150   lab-ha                               lab                                  5.0 MR2 (727)               
|- STATUS: db: unknown; conf: unknown; cond: unknown; dm: none; conn: unknown
        HA cluster member: FGVM01000xxxxxx3 (slave 0)
        HA cluster member: FGVM01000yyyyyy5 (slave 1)               
|- vdom:[3]root flags:0 adom:lab pkg:[never-installed]

Notes:

1) Review the device list and log store list before any changes to know the initial status and status after the change.
2) In some cases the SQL database rebuild can be needed.  The ADOM SQL rebuild does not need a reboot, but the full database rebuild requires a reboot.

# execute sql-local rebuild-adom <adom_name>

# execute sql-local rebuild-db

Related Articles

Technical Note: Missing logs - Manual migration of former standalone FortiGate devices to HA Cluster...