FortiManager - 4 cpus at 100% - thousands of config change events in log

[rjanjax]

We are seeing our Fortimanager busy out all 4 CPUs to 100% with thousands of these messages in the Event logs:

Configuration change event      dev=global,adom=NonProduction,type=fw_policy,key=10461,act=edit,pkgname=DB-FW3-4,_byte=44028298(493793986),_hitcount=15384(96040),_pkts=46876

Also causes policy pushes to take a long time or never complete. Sometimes have to wait 5-10 mins to retry push.

Anybody see this issue?

Running version:  v5.4.1-build1082 160629 (GA)

Memory:

 Total: 10,265,988 KB

 Used: 1,294,288 KB 12.6%

Hard Disk:

 Total: 206,420,664 KB

 Used: 159,964,504 KB 77.5%

We do have a large number of objects in the DB, approx 28000.

Thanks,

[scao_FTNT]

from log, I think you enabled hit count function and FMG logged every hit count update triggered db change

in FMG 5.4.4, we removed this hit count update logging to avoid your mentioned case

Thanks

Simon

[chall_FTNT]

It sounds like you have FMG configured to track hitcount on policies in policy packages, which is known (prior to the most recent patches) to have some performance concerns (as noted in bug id 452464).

The fix in 5.4.4 & 5.6.1 (ETA, end of November) is to disable generating event logs on the FMG  every time the hitcount changes.

Workarounds include: 1) disable hit-count from the System Settings > Advanced Settings 2) on the FortiManager CLI, filter out the objcfg logs (corresponding to the huge amount of event logs we're receiving) as follows: config system locallog disk filter set objcfg disable end

[rjanjax]

Thanks for the quick replies !!

We are trying workaround 2 until we can get Fortimanager upgraded.