Technical Tip: Using log backup and restore (or import) to migrate logs

Description

This article describes how to use log backup and then either log restore or log import to migrate logs between two FortiAnalyzer units.

These approaches require an external storage server but do not require both FortiAnalyzer units to be up simultaneously.

Scope

FortiAnalyzer.

Solution

Log Backup from the old FortiAnalyzer.

This example shows how to back up all FortiAnalyzer logs to an FTP server with the IP address 10.5.50.40. In this case, the username is ftpuser, and the password is 12345678.

execute backup logs all ftp 10.5.50.40 ftpuser 12345678 /

To quit the backup process, Press 'Q/q' then <Enter>.

Uploading for device FGT1KC0000000000(FGT1KC0000000000[root])...
  Backup logs: 1/12 file(s).
  Backup logs: 10/12 file(s).
  Backup logs: 12/12 file(s).
Uploading for device FGT1KC0000000000(FGT1KC0000000000[vd1])...
  Backup logs: 1/3 file(s).
  Backup logs: 3/3 file(s).
Uploading for device FortiGate-VM64-KVM(FGVMEVIGJ13JWW8D[root])...
  Backup logs: 1/1 file(s).
Successfully uploaded log file(s) to ftp server 10.5.50.40 under /.

The size of the archive log on the destination FTP server can be checked with a diagnose log device to make sure it matches the size of the old one.

Output:

Total usage: 16 ADOMs, logs=1.5GB

• Option 1: Log Restore to the new FortiAnalyzer.

Log backup restoration is the recommended method to use if the FortiAnalyzer instance is replaced while the old FortiAnalyzer instance is not reachable.

To restore log file(s), execute the following command:

execute restore logs all ftp 10.5.50.40 <user_name> <password> /

This example shows how to restore FortiAnalyzer logs from an FTP server using the address and credentials of the previous example:

execute restore logs all ftp 10.5.50.40 ftpuser 12345678 /
Note: This command restores all logs from a specified server that
      were backed up prior to changing the RAID level or formatting
      the disks. Executing it frequently is not recommended!
Do you want to continue? (y/n)y
The restore operation will overwrite any logs already on the FortiAnalyzer.
Do you want to continue? (y/n)y
Stopping processes.
Downloading files for device FGT1KC0000000000(FGT1KC0000000000[*])...
  Restore log file: FGT1KC0000000000[root].elog.log.gz
  Restore log file: FGT1KC0000000000[root].tlog.log.gz
  Restore log file: FGT1KC0000000000[root][1529872384].elog.1530700808.log.gz
….
  Restore log file: FGT1KC0000000000[vd1][1529872384].tlog.1530706428.log.gz
Update device FGT1KC0000000000 log files, disk usage...
Downloading files for device FortiGate-VM64-KVM(FGVMEVIGJ13JWW8D[*])...
  Restore log file: FGVMEVIGJ13JWW8D[root].elog.log.gz
Update device FGVMEVIGJ13JWW8D log files, disk usage...
Restoration completed successfully.
Restarting processes.
Recommend to rebuild the database by 'exec sql-local rebuild-db'.

Note:
Restoring logs will overwrite existing logs.
After restoring the logs, Technical Tip: FortiAnalyzer SQL Database Rebuild is required.

• Option 2: Log Import to the new FortiAnalyzer.

Log importing is used only to import logs for one specific log client.

See the example below:

execute log import ftp 10.5.50.40 ftpuser 12345678 /FGT1KC0000000000/
Do you want to continue? (y/n)y
Log Import Info: Connect to ftp server 10.5.50.40 ...
Log Import Info: Found 15 .log or .csv files in remote folder: /FGT1KC0000000123.
Log Import Info: 15 log files found in remote folder, MAX import file setting is 10000, so 15 files will be imported.
Log Import Info: Downloading files from 10.5.50.40 ...###############
Log Import Info: Log file FGT1KC0000000000[root].elog.log.gz was successfully imported to FGT1KC0000000000/root/elog.1530711686.log.
…..
Log Import Info: Log file FGT1KC0000000000[vd1][1529872384].tlog.1530706428.log.gz was successfully imported to FGT1KC0000000000/vd1/tlog.1530706428.log.
Log Import Info: 15 log files are imported.
Log Import Info:
15 files are processed, 0 files remain.

Note: Log import should automatically trigger indexing of the new logs.

FTP Transfer debug:
The FTP transfer has limited troubleshooting capability. However, the output of the following CLI commands will be requested, as well as the system event log and the FTP event log:

execute tac report
diagnose sniffer packet any “host <Ip of the FTP server> and port 21” 3 0 a

Sample logs: