Troubleshooting Tip: How to fix a FortiAnalyzer that does not display analytic logs when logs are being inserted to the database

[Nur]

Description	This article describes a case where the FortiAnalyzer is still receiving logs from a FortiGate, but the logs are not displayed in Analytics.  diagnose fortilogd msgrate last 5 seconds: 1548.0, last 30 seconds: 1860.1, last 60 seconds: 1848.3 diagnose fortilogd lograte-device Logs per second Totals Last Hour Day Week ------------------------------------------------------- FGHA002XXXXXXXXX_CID: 35.87 13.78 10.07 FGHA001XXXXXXXXX_CID: 108.30 94.75 88.12 FGHA0005XXXXXXXX_CID: 40083.80 37958.09 13869.70
Scope	FortiAnalyzer Virtual Machines.
Solution	To determine what causes the issue of analytic logs not appearing, check the kernel log. If the following message exists: diagnose debug klog ... <7>[48627.951510] sd 0:0:1:0: [sdb] tag#2 Failed to abort cmd 00000000XXXXXXXX <6>[48627.951516] sd 0:0:1:0: [sdb] tag#2 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=33s <6>[48627.951519] sd 0:0:1:0: [sdb] tag#2 CDB: opcode=0x88 XXXXXXXXXXXX <3>[48627.951522] blk_update_request: I/O error, dev sdb, sector 857082200 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0 The sdb process triggered a 'Failed to abort cmd' message. I/O error = Physical read failure. Sector 857082200 = ~428.5GB into the disk (sector 857082200 × 512 bytes). READ operation failed= Cannot read data from the disk. This means that multiple areas of the disk are failing; this is a critical disk failure. diagnose system print partitions major minor #blocks name fstype 1 0 4096 ram0 1 1 4096 ram1 1 2 4096 ram2 1 3 4096 ram3 7 0 10240 loop0 ext2 8 0 2097152 sda 8 1 1048576 sda1 ext3 8 16 5905580032 sdb 252 0 5905575936 dm-0 SDB = 5.9TB virtual disk (5905580032 blocks) [Calculation 1K-block / 104876]. DM-0 = LVM logical volume using almost all of sdb (5905575936 blocks). If the LVM logical volume is almost full, check the disk health in the Virtual Machine Environment disk health. This needs to be checked internally as the TAC scope only covers FortiAnalyzer OS and not the Virtual Machine platform. If diagnose debug klog does not show any abnormal error, run the diagnose fsck harddisk to repair and check the disk.