Description
This article describes a recurrent issue where 'forticron' crashes approximately every 6 to 10 seconds, as observed in the System Event logs or the crash log (CLI). This is attributed to an Application Bandwidth widget configured with over 50 Apps.
Example log message:
Log Description Application crashed
Action crash
Message Pid: 28488, application: forticron, Firmware: FortiGate-VM64-KVM v7.2.7,build1577b1577,240131 (GA.M) (Release), Signal 6 received, Backtrace: [0x7f566fc1be11] [0x7f566fc05557] ...
Scope
FortiGate v7.0 and v7.2.
Solution
The continuous crash is likely caused by a dashboard widget 'Application Bandwidth' (fortiview-app-bandwidth) configured to monitor more than 50 apps. This can occur under any admin account and can be identified through debugging and by examining a backup configuration. Removing or reconfiguring this widget to monitor fewer apps should resolve the issue.
Note: This limitation is resolved in version v7.4 and later.
Debugging the Forticron Process.
During the debugging of the Forticron process, if the 'top_app_history' begins processing and fails to complete without a 'done' message, leading to a crash and restart of the process, it indicates an overload due to excessive monitoring.
diagnose debug application forticron -1
2024-08-01 16:14:45 fcron_timer_func()-23: Timer sys_stats fired
2024-08-01 16:14:45 fcron_timer_func()-32: Timer sys_stats done
...
2024-08-01 16:14:45 fcron_timer_func()-32: Timer traf_his done
2024-08-01 16:14:45 fcron_timer_func()-23: Timer top_app_history fired <----
di 2024-08-01 16:14:46 fcron_sched_add()-147: caller=0xc6f467, sched=ftgd_lic_ck(0x11b0fd18), months=0xffffffff, mdays=0xffffffff, wdays
=0xffffffff, hours=0x1 at 00:00
...
2024-08-01 16:14:47 fcron_sched_add()-147: caller=0xc6a780, sched=log_roll(0x11b0e1b0), months=0xffffffff, mdays=0xffffffff, wdays=0xfff
fffff, hours=0x1 at 00:00
2024-08-01 16:14:47 fcron_arm_sched()-59: sched-log_roll(0x11b0e1b0) will be checked in 27913s
2024-08-01 16:14:47 fcron_gui_cert_update_init()-641:
2024-08-01 16:14:47 fcron_gui_cert_bootup_check()-631:
2024-08-01 16:14:47 fcron_gui_cert_schedule_update()-616: gui_mgmt_cert will update in 10 seconds.
This will repeat every 10 seconds more or less.
How to discover what admin is with an Application Bandwidth widget with more than 50 Apps being monitored.
- Perform a backup of the FortiGate.
- Open the backup file with a text or code editor (example: Visual Studio Code).
- Search for 'fortiview-app-bandwidth' to identify which admin accounts are affected.
Solutions for Affected Admin Accounts.
- Request the admin (example: 'admin2') to adjust the dashboard widget to monitor fewer than 50 apps. This is the recommended approach.
Edit FortiView app bandwodth
- Manually edit or remove the excessive monitoring through CLI commands specific to your environment. Caution: Execute CLI commands carefully to avoid unintended changes.
config system admin2
edit "admin"
config gui-dashboard
edit 21
delete 1
end
end
end
Related documents:
Dashboards and Monitors (docs)
Troubleshooting Tip: Monitoring widget for application based bandwidth showing 'No Data'
Technical Tip: How to setup a bandwidth limit for specific applications, such as Microsoft Teams
