Technical Tip: FortiManager VM policy push stuck at 'Start copying policy to devdb, device'

In certain situations, when FortiManager has a huge amount of policies in a policy package (~40K or more) and pushes to multiple devices sharing within the same policy package, this can result in the task getting stuck due to performance-related reasons, mainly due to a lack of memory to finish this task.

FortiManager forks multiple processes and performs a copy to multiple devices at the same time. The system memory will then be shared among the concurrently copied devices.

Users might see certain logs related to Security Console debugs as below:

"SECURITY_CONSOLE: __read_copy_result,1937: Could not open file /var/securityconsole/xxxxxxxxxxxxx.dat" <----- This log is one of the symptoms related to that matter.

It is recommended to keep the maximum number approximately to 80K policies per device as a threshold (Based on general internal testings) to allow the FortiManager to perform as per the expectation, if multiple devices (Lets say 'n' is the number of devices sharing the Policy package) are used then the max is 'n x number of policies' which should not be more than 80K else that would introduce failures in pushing the config to those devices.

It is recommended to either increase the memory or redesign the way the policies are shared within the policy package among the devices by splitting the Policy package or dividing it into multiple ones within the same FortiManager or even multiple ones.

Troubleshooting:

diagnose debug application securityconsole 255

diagnose debug enable