FortiGate 100E Timestamp Sudden Jump?

[SIAPP]

Hi

I am using a Fortigate 100E in a certain site. The users reported a sudden "outage" during the night - which basically meant they could not reach any website and several assets in the site itself (like a camera server was unreachable, for example).

Two minutes later, everything went back to normal - users could reach all devices and the internet as well.

While this sounded a bit strange, I was looking at the Forward Traffic logs and saw a major timestamp jump at around the same time as the reported "outage":

Timestamp jump around 03:25

BTW, the logs from "Thursday January 18th" last for about two minutes (so from 20:15:07 to 20:17:07) then go back to a different time entirely - a year back.

January 24th 2023 was a year ago.

Any idea where I should start looking?

[saleha]

Hello,

Thank you for reaching out. When the outage happened did the users get timeouts error messages on the browser or applications or was the issue is that the apps kept loading without progress for 2 minutes? from what you mentioned it sounds as if the firewall or the device between the users and the firewall froze for that outage period. I would start by checking system event logs, router logs and crashlog. Crash logs can only be viewed using cli command:
#diagnose debug crashlog read

Thank you,

saleha

[SIAPP]

Thanks for replying! So looking at the crash logs, there isn't something that corresponds to the crash:

322: 2023-12-31 04:17:52 the killed daemon is /bin/dhcpd: status=0x0
323: 2024-01-03 12:15:15 the killed daemon is /bin/dhcpd: status=0x0
324: 2024-01-18 11:15:06 the killed daemon is /bin/dhcpd: status=0x0
Crash log interval is 3600 seconds

During the crash, the users got timeouts - yeah! One page that is local was showing "Error 503".

I don't see anything unusual in the "Events" section though

[AEK]

Hi

Can it be caused by NTP server?

Try check the following if other hosts that are synchronizing from the same NTP server were affected by the same time jump. Change the NTP configuration on FGT if required.

[saleha]

Hi,

you can if you suspect ntp server connectivity debug and run sniffer on port 123:

di de reset

di de app ntp -1

di de console time en

di de en

sniffer command:
di sniffer packet any "dport 123" 4 0 l

Check also performance indicators of the firewall:

get sys performance status

di sys top

di sys top-mem 120

Thank you,

saleha