Table of Contents:
Introduction
Common reboot reasons
- Warm reboot
- Upgrade/downgrade
- Power source disruption or power cycle
- Kernel panic
- Config changes triggered reboots
- Shutdown
Post-reboot checks
- Firmware Image verification
- File system check
- System resources check
- Config errors and crashlog checks
- Config checks
- File hash checks
Related articles
Introduction.
If a FortiGate was found to have rebooted (especially an unplanned reboot) it is important to try to find out what caused the last reboot and the events that led to it. A reboot of FortiGate could be triggered for multiple reasons; these are discussed in detail in this article, with example logs/CLI outputs to infer the possible reasons & identify any anomalies. The first thing to check is the output of the two CLI commands below, to review the value of the 'Last reboot reason:' and 'Uptime' fields, as shown in the examples.
FortiGate # get system status Version: FortiGate-81F v7.2.10,build1706,240918 (GA.M) Security Level: 1 Firmware Signature: certified Virus-DB: 92.07928(2024-10-10 03:26) . . . Serial-Number: FGT81FXYZXYZXYZ BIOS version: 05000020 System Part-Number: P25635-20 Log hard disk: Available Hostname: FortiGate . . . FIPS-CC mode: disable Current HA mode: a-p, primary Cluster uptime: 14 days, 4 hours, 1 minutes, 59 seconds <----- Indicates the HA cluster uptime, not this specific FortiGate unit uptime. Refer to the next CLI command for the unit's uptime. Cluster state change time: 2025-08-05 08:05:37 Branch point: 1706 Release Version Information: GA System time: Tue Aug 19 12:05:11 2025 Last reboot reason: power cycle <----- The last reboot reason as saved in memory.
FortiGate #
FortiGate # get system performance status CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU0 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU1 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq . . . Memory: 3806360k total, 1547696k used (40.7%), 1748312k free (45.9%), 510352k freeable (13.4%) Average network usage: 98 / 689 kbps in 1 minute, 23 / 95 kbps in 10 minutes, 15 / 48 kbps in 30 minutes Maximal network usage: 1177 / 4707 kbps in 1 minute, 1177 / 4707 kbps in 10 minutes, 1177 / 4707 kbps in 30 minutes Average sessions: 8 sessions in 1 minute, 6 sessions in 10 minutes, 5 sessions in 30 minutes . . . IPS attacks blocked: 0 total in 1 minute Uptime: 14 days, 4 hours, 3 minutes <----- Indicates the uptime of this specific FortiGate unit (not the HA cluster). For FortiGate 6000F or 7000E/F chassis series, it indicates the uptime of the correspondent slot.
FortiGate #
Review the system event logs from the FortiGate GUI -> Log & Report -> System Events -> General System Events and look for the approximate timestamp when the reboot was noticed. The log message (msg) provides more details on the possible trigger for the reboot. Also, review the logs just before this 'Device shutdown' entry to understand the sequence of events that led to the reboot.
Common reboot reasons.
Here are the most common reboot reasons, along with example logs.
- Warm reboot: If the FortiGate is rebooted with a CLI command using 'execute reboot' or from System -> Reboot, it is considered a warm reboot wherein the FortiOS will gracefully shut down all processes during reboot. This is the most common reason for FortiGate reboots, including during upgrades, and is the least likely to have any effect on the functioning of the unit once it boots back up.
FortiGate # get system status | grep reason
Last reboot reason: warm reboot
System Events log entry:
date=2025-08-20 time=09:15:47 eventtime=1755706547643674960 tz="-0700" logid="0100032138" type="event" subtype="system" level="critical" vd="root" logdesc="Device rebooted" user="admin" ui="jsconsole(172.16.10.1)" action="reboot" msg="User admin rebooted the device from jsconsole(172.16.10.1)."
In the example above, the 'admin' user initiated a reboot using the CLI console from the FortiGate GUI.
- Upgrade/downgrade: When an upgrade of FortiGate is performed, the expected reboot reason in the CLI is 'warm reboot' with a system event log indicating the reason is 'upgrade firmware', as shown in the example below.
FortiGate # get system status | grep reason
Last reboot reason: warm reboot
System Events log entry:
date=2025-08-20 time=09:34:45 eventtime=1755707685667678800 tz="-0700" logid="0100032138" type="event" subtype="system" level="critical" vd="root" logdesc="Device rebooted" user="admin" ui="GUI(172.16.10.1)" action="reboot" msg="User admin rebooted the device from GUI(172.16.10.1). The reason is 'upgrade firmware'"
- Power source disruption or Power cycle: When power is disrupted/cut off to the FortiGate unit due to any reason (faulty PSU, or the power cable was manually pulled, or the power supply source itself has an outage), it records this event with a system event log entry as well as the last reboot reason field, as shown below:
FortiGate # get system status | grep reason
Last reboot reason: power cycle
System Events log entry:
"date=2021-12-24 time=12:29:01 eventtime=1640377664961912960 tz="-0800" logid="0100032200" type="event" subtype="system" level="critical" vd="root" logdesc="Device shutdown" msg="Fortigate had experienced an unexpected power off!""
Verify if the issue is faulty PSU(s) in the FortiGate, or the external power supply source itself had an outage (at least one PSU active and in the 'OK' status is sufficient to power the FortiGate). If the PSU unit indeed became faulty and failed, an event log would be recorded as shown below:
If a PSU was previously connected and has now failed, it may show an event log similar to the following:
date=2024-08-24 time=10:58:21 eventtime=128172165219808500 tz="+0100" logid="0100022105" type="event" subtype="system" level="critical" vd="root" logdesc="Power supply failed" action="ipmc-sensor-monitor" status="failure" msg="PSU [1] not detected".
date=2024-08-24 time=10:58:21 eventtime=181720701921820220 tz="-0800" logid="0100022105" type="event" subtype="system" level="critical" vd="root" logdesc="Power supply failed" action="ipmc-sensor-monitor" status="failure" msg="PSU [2] not detected"
If the 2nd optional PSU is not connected, but the other PSU is active and status is OK, it will show the message 'Optional power supply not detected' on the newer .2+ versions.
date=2025-08-20 time=09:17:14 eventtime=1755706631799991600 tz="-0700" logid="0100022106" type="event" subtype="system" level="information" vd="root" logdesc="Optional power supply not detected" action="ipmc-sensor-monitor" status="anomaly" msg="Power Supply is not detected: PSU [1] LOST"
The following is an example where PSU1 is not connected, but PSU2 is active, and the status is OK. This status is enough to power the FortiGate, but without PSU redundancy.
FortiGate # diagnose hardware deviceinfo psu Power Supply Status PSU[1] LOST PSU[2] OK
Note:
- If only one of the two PSUs is connected to power, the 'Power supply failed' logs will be generated frequently for that one PSU not connected to power. This does not mean the unit has lost all power: one active PSU is sufficient to power the unit. To address this, connect both PSUs to power (recommended) or ignore the power supply failed logs for the PSU that is not connected. More details on troubleshooting the faulty PSU issue are available in this article: Technical Tip: On-Site Guide for Identifying a Faulty PSU in a FortiGate Firewall.
- The CLI commands used to check the status of the PSU depend on the model type of the FortiGate, but the two common CLI commands to use are 'diagnose hardware deviceinfo psu' and 'execute sensor list'. More details on these commands can be read in Technical Tip: How to check power supply details for FortiGate.
- When the FortiGate boots back up from a 'power cycle' event, it will typically request a 'File system check' or 'disk scan' check to verify file system consistency after an ungraceful shutdown/reboot. More details on this are available in Technical Tip: File System check recommended message.
- Kernel panic: It is a safety mechanism wherein the kernel of an OS determines it is necessary to reboot or halt (shutdown) under certain exceptional conditions. When FortiGate reboots in such a scenario, it will record the last reboot reason as 'kernel panic' but may not add a system event log entry before restart - there will be an entry after the system reboots, saying 'unexpected power off!'. Check the 'diagnose debug crashlog read' output for more details on the kernel panic. If there is no entry there, it will be necessary to enable capturing the backtrace for events leading to the kernel panic and troubleshoot further. Console logs (if a console connection to FortiGate is active) could print useful logs related to kernel panic. Also, certain FortiGate medium/high-end series support COMlog, which provides useful logs to troubleshoot kernel panic issues. COMlogs need to be enabled in advance ('diagnose debug comlog enable') before another kernel panic event. More details on how to enable COMlog is here: How to use the COMLog feature. The most common reasons for kernel panic are resource starvation (sustained High CPU, High memory, etc), Hardware issues (run HQIP tests), and software defects.
FortiGate # get system status | grep reason
Last reboot reason: kernel panic
date=2025-08-12 time=09:21:48 eventtime=1755706631799991600 logid="0100032200" type="event" subtype="system" level="critical" vd="root" logdesc="Device shutdown" msg="Fortigate had experienced an unexpected power off!" date=2025-08-12 time=09:21:48 eventtime=1755706631799991600 tz="+0200" logid="0100032009" type="event" subtype="system" level="information" vd="root" logdesc="FortiGate started" msg="Fortigate started"
Note: Check the release notes of the corresponding FortiOS firmware version to review any known issues regarding Kernel Panic (e.g., FortiOS version 7.2.8 has a Known issue - 1012518, which is related to a kernel panic defect).
Review the output of 'diagnose debug crashlog read' and system event logs for possible explanations for kernel panic. Look for any entries that correlate with the timestamp of the reboot event, for example, has the device been entering conserve mode frequently around the time of reboot, or frequent wad crashes, etc. While the conserve mode or wad crashes themselves will not typically cause the FortiGate unit to reboot, it could help troubleshoot the conditions that led to the kernel panic reboot.
FortiGate # diagnose debug crashlog read
<snippet>
2025-02-21 10:14:25 green="6252 MB" msg="Kernel enters memory conserve mode"
- Config changes triggered reboots: Certain configuration changes ('config system npu' changes) could trigger a forced reboot of the FortiGate (after accepting the warning). One such example is changing the default-qos-policy or hif-queue-customize on NP7 platforms, which will trigger a reboot after confirmation:
FortiGate # config system npu FortiGate (npu) # set default-qos-type policing FortiGate (npu) # end The configuration will take effect after system reboot. Do you want to continue? (y/n)y FortiGate #
FortiGate # get system status | grep reason
Last reboot reason: warm reboot
date=2025-08-21 time=15:40:01 eventtime=1755805201323242547 tz="-0400" logid="0100032138" type="event" subtype="system" level="critical" vd="root" logdesc="Device rebooted" user="admin" ui="jsconsole(172.16.10.1)" action="reboot" msg="User admin rebooted the device from jsconsole(172.16.10.1). The reason is 'np7 QOS type changed'"
- Shutdown: If a FortiGate needs to be powered off (for maintenance, relocation, or any reason), a graceful shutdown using the CLI command (or from the GUI) using 'execute shutdown' is suggested instead of pulling the power cable. With a manual shutdown this way, the last reboot reason is appropriately recorded as 'shutdown'.
FortiGate # get system status | grep reason
Last reboot reason: shutdown
date=2025-08-15 time=22:01:53 eventtime=1755320513160322840 tz="-0700" logid="0100032200" type="event" subtype="system" level="critical" vd="root" logdesc="Device shutdown" user="admin" ui="console" action="shutdown" msg="User admin shutdown the device from console."
Post-reboot checks.
If the reboot was ungraceful/unplanned, it is recommended to do the following checks to identify any anomalies and verify that the system is running optimally:
- Firmware Image verification: Verify that the firmware on the FortiGate is the expected version and has not changed after reboot. FortiGate typically has two partitions in the Flash (Technical Tip: Selecting an alternate firmware for the next reboot), primary and secondary, which can hold two different firmware versions. During the reboot, ensure the right image has been loaded by using the following CLI command. Verify the build number and the GA (General Availability image) tag, and the Firmware Signature field marked 'certified'.
FortiGate # get system status Version: FortiGate-61F v7.2.11,build1740,250210 (GA.M) <----- Verify the firmware image name & build number. Security Level: High <----- Security level preferably High. Firmware Signature: certified <----- Ensure the firmware is certified. Virus-DB: 1.00000(2018-04-09 18:07)
. . .
FortiGate # diag sys flash list Partition Image TotalSize(KB) Used(KB) Use% Active 1FG61F-7.02-FW-build1740-250210 253871 136369 54% Yes <---------- Partition 1 with 7.2.11 firmware image. 2 FG61F-7.00-FW-build0667-240925 253871 139990 55% No <---------- Partition 2 with an old 7.0. firmware image. 3 ETDB-1.00000 28327040 227952 1% No Image build at Feb 10 2025 18:19:08 for b1740
- File system check: Both the GUI and CLI will show a warning message requesting to run a disk scan after an unsafe reboot. It is recommended to either run the disk scan immediately or schedule a time soon to run this task. Review the article Technical Tip: File System check recommended message regarding the file system check.
WARNING: File System Check Recommended! An unsafe reboot may have caused an inconsistency in the disk drive.
It is strongly recommended that you check the file system consistency before proceeding. Please run 'execute disk scan 1' Note: The device will reboot and scan the disk during startup. This may take up to an hour.
FortiGate # execute disk scan 1 scan requested for: device=/dev/nvme0n1p1 1/SSD1 status=enable media-status=enable This action requires the unit to reboot. Do you want to continue? (y/n)y
- System resources check: Check if FortiGate has high CPU utilization or High memory / conserve mode issues, which might have been a reason for the last reboot. Review the previous event logs to see the periodic system performance statistics.
FortiGate # get system performance status
FortiGate # get system top
Review the FortiGate GUI dashboard -> Status for CPU / Memory / Sessions graphs for the last several days, as well as check the event logs for system performance stats:
date=2025-08-15 time=22:01:53 id=7414764306404737051 itime="2025-08-15 22:01:53" euid=3 epid=3 dsteuid=3 dstepid=3 logver=704042662 logid=0100040704 type="event" subtype="system" level="notice" cpu=6 mem=28 disk=3 setuprate=10587 totalsession=20977 bandwidth="278384/261844" action="perf-stats" msg="Performance statistics: average CPU: 6, memory: 28, concurrent sessions: 20977, setup-rate: 10587" logdesc="System performance statistics" sysuptime=6482 freediskstorage=261721 disklograte=281 fazlograte=371 trate=57218 erate=42 orate=52812. . .
Further troubleshooting for high system resource utilization are available in the following documents:
Troubleshooting high CPU usage
Troubleshooting Tip: How to do initial troubleshooting of high memory utilization issues (conserve m...
- Config errors and crashlog checks: With the following FortiGate CLI commands, any config errors after an unplanned bootup (or upgrade) can be identified. Additionally, review the crashlogs to see any entries that match the timestamp of the last reboot.
diagnose debug config-error-log read
diagnose debug crashlog read
- Config checks: After an unplanned reboot, compare the current config with a previous, for any relevant differences in configs, and analyze further if necessary. This can be done by comparing current configs with a recent backup, or using the diff option in the FortiGate (if it was previously enabled and in use), or the most efficient way using FortiManager, which would keep multiple revisions of FortiGate config, tracking changes over time.
- File hash checks: Optionally, if a system compromise is suspected, compute the file hashes of all the files on the FortiGate file system using the CLI command shown below. Compare these hashes against a known good system (taken from this same FortiGate before, or from another FortiGate of the same hardware model type and firmware version) to check if any of the files have a different hash. For more details on file hash checks, refer to the article : Technical Tip: Best practice: Periodically do FortiGate files hashcheck
diagnose sys filesystem hash <paths> -d [depth]
If any anomaly is identified, or if the reboot was due to a kernel panic, collect the following CLI outputs and contact Fortinet support for further troubleshooting. It may also be useful to collect comlog logs (the feature has to be enabled before the kernel panic event).
show
execute tac report
diagnose debug crashlog read
diagnose debug config-error-log read
diagnose debug comlog read
execute log filter category 1
execute log filter view-lines 1000
execute log display
Related articles:
Troubleshooting Tip: How to troubleshoot unexpected crashes or system hangs
Troubleshooting Tip: How to deal with a Kernel panic
Technical Tip: How to use the COMLog feature
FortiGate: CLI command to compute file hashes
Technical Tip: Understanding the purpose of rebooting a FortiGate Firewall for troubleshooting and i...
Technical Tip: How to properly shut down or reboot a FortiGate
|