Config change causes silent reload (6.2.2 and 6.2.3)

[btp]

80 x FG80E running 6.2.2 and 6.2.3. A couple of them seem to have 'management issues'. When issuing a basic config change (after a fresh reboot), the unit becomes unresponsive for approx 3-4 minutes. Seems to be any kind of config change. The traffic through the unit also stops. When looking on it through console, it seems quite unhappy:

config system ntp    set source-ip 10.10.10.10

2020-09-08 17:31:46 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=214(/bin/newcli) [size="1"]2020-09-08 17:31:46 [_svr_d_commit:2989] pid=214, object='system.ntp'[/size] 2020-09-08 17:31:46 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:31:46 cmdbsvr recv req_type=22(CMDB_REQ_SEND_CMDB_EVENT) from pid=214(/bin/newcli) [size="1"]2020-09-08 17:31:47 [check_if_need_revert_cfg:986] 42949666,4294967289,10[/size] [size="1"]2020-09-08 17:31:47 System will reload in the next 9 seconds...[/size] 2020-09-08 17:31:47 cmdbsvr signal_init 2020-09-08 17:31:47 cmdbsvr send_event NP6LITE: session timeout message missed, in=wan1 sip=34619742, dip=0a302f80, sport=443, dport=49371, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"]2020-09-08 17:31:48 [check_if_need_revert_cfg:986] 42949667,4294967289,10[/size] 2020-09-08 17:31:48 System will reload in the next 8 seconds... NP6LITE: session timeout message missed, in=wan1 sip=34619742, dip=0a302f80, sport=443, dport=49370, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"]2020-09-08 17:31:49 [check_if_need_revert_cfg:986] 42949668,4294967289,10[/size] 2020-09-08 17:31:49 System will reload in the next 8 seconds... NP6LITE: session timeout message missed, in=wan1 sip=d9920d89, dip=0a302f74, sport=5938, dport=59385, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 2020-09-08 17:31:50 NP6LITE: session timeout message missed, in=wan1 sip=2843fe24, dip=0a302f84, sport=443, dport=65068, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"][check_if_need_revert_cfg:986] 42949669,4294967289,10[/size] 2020-09-0NP6LITE: session timeout message missed, in=wan1 sip=0a301673, dip=0a302f14, sport=5002, dport=4334, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 8 17:31:50 System will reload in the next 7 seconds... [size="1"]2020-09-08 17:31:51 [check_if_need_revert_cfg:986] 42949670,4294967289,10[/size] 2020-09-08 17:31:51 System will reload in the next 6 seconds... [size="1"]2020-09-08 17:31:52 [check_if_need_revert_cfg:986] 42949671,4294967289,10[/size] 2020-09-08 17:31:52 System will reload in the next 5 seconds... NP6LITE: session timeout message missed, in=wan1 sip=2843fb84, dip=0a302f65, sport=443, dport=57263, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"]2020-09-08 17:31:53 [check_if_need_revert_cfg:986] 42949672,4294967289,10[/size] 2020-09-08 17:31:53 System will reload in the next 4 seconds... [size="1"]2020-09-08 17:31:54 [check_if_need_revert_cfg:986] 0,4294967289,10[/size] 2020-09-08 17:31NP6LITE: session timeout message missed, in=wan1 sip=34619742, dip=0a302f80, sport=443, dport=49374, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 NP6LITE: session timeout message missed, in=wan1 sip=14be8117, dip=0a302f80, sport=443, dport=49363, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 :54 System will reload in the next 3 seconds... NP6LITE: session timeout message missed, in=wan1 sip=c3e111f0, dip=0a302f8b, sport=443, dport=63683, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 NP6LITE: session timeout message missed, in=wan1 sip=34619742, dip=0a302f80, sport=443, dport=49375, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 NP6LITE: session timeout message missed, in=wan1 sip=0a30140e, dip=0a302f66, sport=5246, dport=37574, proto=17 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"]2020-09-08 17:31:55 [check_if_need_revert_cfg:986] 0,4294967289,10[/size] 2020-09-08 17:31:55 System will reload in the next 2 seconds... NP6LITE: session timeout message missed, in=wan1 sip=2843fb84, dip=0a302f6f, sport=443, dport=49908, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 NP6LITE: session timeout message missed, in=wan1 sip=2843fe24, dip=0a302f85, sport=443, dport=65206, proto=6 in_vtag=1, in_pid=249, t=0, valid=00000001 NP6LITE: session timeout message missed, in=wan1 sip=0a30140e, dip=0a302f6e, sport=5247, dport=37579, proto=17 in_vtag=1, in_pid=249, t=0, valid=00000001 [size="1"]2020-09-08 17:31:57 [check_if_need_revert_cfg:986] 1,4294967289,10[/size] 2020-09-08 17:31:57 System will reload in the next 1 seconds... [size="1"]2020-09-08 17:31:58 [check_if_need_revert_cfg:986] 2,4294967289,10[/size] 2020-09-08 17:31:58 0: config vdom 2020-09-08 17:31:58 0: edit VPN 2020-09-08 17:31:58 0: config vpn ipsec phase1-interface [size="1"]2020-09-08 17:31:58 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.certificate'[/size] [size="1"]2020-09-08 17:31:58 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv4-exclude-range'[/size] [size="1"]2020-09-08 17:31:58 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv6-exclude-range'[/size] [size="1"]2020-09-08 17:31:58 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.backup-gateway'[/size] [size="1"]2020-09-08 17:31:58 [_svr_d_delete:3032] pid=220, object='vpn.ipsec.phase1-interface'[/size] [size="1"]2020-09-08 17:31:59 [_svr_d_delete:3032] pid=98, object='system.interface'[/size] 2020-09-08 17:31:59 0: delete "4G-2" 2020-09-08 17:31:59 0: end 2020-09-08 17:31:59 0: end 2020-09-08 17:31:59 0: config vdom 2020-09-08 17:31:59 0: edit VPN 2020-09-08 17:31:59 0: config vpn ipsec phase1-interface [size="1"]2020-09-08 17:31:59 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.certificate'[/size] [size="1"]2020-09-08 17:31:59 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv4-exclude-range'[/size] [size="1"]2020-09-08 17:31:59 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv6-exclude-range'[/size] [size="1"]2020-09-08 17:31:59 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.backup-gateway'[/size] [size="1"]2020-09-08 17:31:59 [_svr_d_delete:3032] pid=220, object='vpn.ipsec.phase1-interface'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_delete:3032] pid=98, object='system.interface'[/size] 2020-09-08 17:32:00 0: delete "4G-1" 2020-09-08 17:32:00 0: end 2020-09-08 17:32:00 0: end 2020-09-08 17:32:00 0: config vdom 2020-09-08 17:32:00 0: edit VPN 2020-09-08 17:32:00 0: config vpn ipsec phase1-interface [size="1"]2020-09-08 17:32:00 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.certificate'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv4-exclude-range'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv6-exclude-range'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.backup-gateway'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_delete:3032] pid=220, object='vpn.ipsec.phase1-interface'[/size] [size="1"]2020-09-08 17:32:00 [_svr_d_delete:3032] pid=98, object='system.interface'[/size] 2020-09-08 17:32:00 0: delete "IPSEC-1" 2020-09-08 17:32:00 0: end 2020-09-08 17:32:00 0: end 2020-09-08 17:32:07 0: config vdom 2020-09-08 17:32:07 0: edit root 2020-09-08 17:32:07 0: config vpn ipsec phase1-interface [size="1"]2020-09-08 17:32:07 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.certificate'[/size] [size="1"]2020-09-08 17:32:07 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv4-exclude-range'[/size] [size="1"]2020-09-08 17:32:07 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.ipv6-exclude-range'[/size] [size="1"]2020-09-08 17:32:07 [_svr_d_clear:3071] pid=220, object='vpn.ipsec.phase1-interface.backup-gateway'[/size] [size="1"]2020-09-08 17:32:07 [_svr_d_delete:3032] pid=220, object='vpn.ipsec.phase1-interface'[/size] [size="1"]2020-09-08 17:32:07 [_svr_d_delete:3032] pid=98, object='system.interface'[/size] 2020-09-08 17:32:08 0: delete "MGMT_IPSEC" 2020-09-08 17:32:08 0: end 2020-09-08 17:32:08 0: end 2020-09-08 17:32:09 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:09 cmdbsvr signal_init 2020-09-08 17:32:09 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:09 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:09 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:09 cmdbsvr signal_init unregister_vf: waiting for root to become free. rt_num = 2 vf_count=39 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='system.interface'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=31(CMDB_REQ_BI_VD_SET_IPROPE) from pid=151(/bin/dhcpcd) 2020-09-08 17:32:20 cmdbsvr recv req_type=32(CMDB_REQ_BI_VD_SET_IPROPE6) from pid=151(/bin/dhcpcd) 2020-09-08 17:32:20 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='system.interface'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=0(CMDB_REQ_COMMIT) from pid=151(/bin/dhcpcd) [size="1"]2020-09-08 17:32:20 [_svr_d_commit:2989] pid=151, object='router.static'[/size] 2020-09-08 17:32:20 cmdbsvr recv req_type=31(CMDB_REQ_BI_VD_SET_IPROPE) from pid=151(/bin/dhcpcd) 2020-09-08 17:32:20 cmdbsvr recv req_type=32(CMDB_REQ_BI_VD_SET_IPROPE6) from pid=151(/bin/dhcpcd) 2020-09-08 17:32:20 cmdbsvr recv AF_MSG_T_CMDB_SVR_CHECK_DAEMON 2020-09-08 17:32:21 cmdbsvr set_iprope __ha_ioctl()-210 t=12280: FW_CMD_CLEAR_BRIDGE_TIMER __ha_ioctl()-65 t=12285: FW_CMD_CLEAR_HA_MODE ip_session_change_id()-1529 t=12290: calls __ip_session_change_id()

__ha_ioctl()-210 t=16896: FW_CMD_CLEAR_BRIDGE_TIMER __ha_ioctl()-210 t=16901: FW_CMD_CLEAR_BRIDGE_TIMER NP6LITE: add vlif oid=68, vid=700 NP6LITE: insert iswa_spvact pht_idx = 220, 000af044 00000000 00000000 00000000 NP6LITE: insert tce_map pht_idx = 220, 408af044 00000010 00000000 00000000 NP6LITE: add vlif oid=68, vid=701 NP6LITE: insert iswa_spvact pht_idx = 24, 000af444 00000000 00000000 00000000 NP6LITE: insert tce_map pht_idx = 24, 40caf444 00000010 00000000 00000000 NP6LITE: lif dmz closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 2 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif dmz opened ADDRCONF(NETDEV_UP): dmz: link is not ready NP6LITE: lif ha closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 3 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif ha opened ADDRCONF(NETDEV_UP): ha: link is not ready NP6LITE: lif npu0_vlink0 closed NP6LITE: Change LIF 16 MTU from 15324 to 15324 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif npu0_vlink0 opened ADDRCONF(NETDEV_UP): npu0_vlink0: link is not ready NP6LITE: lif npu0_vlink1 closed NP6LITE: npu0_vlink0 link up, 1000Mbps NP6LITE: Change LIF 17 MTU from 15324 to 15324 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif npu0_vlink1 opened ADDRCONF(NETDEV_UP): npu0_vlink1: link is not ready NP6LITE: lif port1 closed NP6LITE: np6lite_lif_ethtool_sset: autoneg=1, duplex=0, speed=10 NP6LITE: Change LIF 4 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port1 opened ADDRCONF(NETDEV_UP): port1: link is not ready NP6LITE: npu0_vlink1 link up, 1000Mbps NP6LITE: lif port1 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 4 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port1 opened ADDRCONF(NETDEV_UP): port1: link is not ready NP6LITE: lif port2 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 5 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port2 opened ADDRCONF(NETDEV_UP): port2: link is not ready NP6LITE: lif port3 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 6 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port3 opened ADDRCONF(NETDEV_UP): port3: link is not ready NP6LITE: lif port4 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 7 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port4 opened ADDRCONF(NETDEV_UP): port4: link is not ready NP6LITE: lif port5 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 8 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port5 opened ADDRCONF(NETDEV_UP): port5: link is not ready NP6LITE: lif port6 closed NP6LITE: port1 link up, 1000Mbps NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 9 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port6 opened ADDRCONF(NETDEV_UP): port6: link is not ready NP6LITE: lif port7 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 10 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port7 opened ADDRCONF(NETDEV_UP): port7: link is not ready NP6LITE: lif port8 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 11 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port8 opened ADDRCONF(NETDEV_UP): port8: link is not ready NP6LITE: lif port9 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 12 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port9 opened ADDRCONF(NETDEV_UP): port9: link is not ready NP6LITE: lif port10 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 13 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port10 opened ADDRCONF(NETDEV_UP): port10: link is not ready NP6LITE: lif port11 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 14 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port11 opened ADDRCONF(NETDEV_UP): port11: link is not ready NP6LITE: lif port12 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 15 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif port12 opened ADDRCONF(NETDEV_UP): port12: link is not ready NP6LITE: lif wan1 closed NP6LITE: np6lite_lif_ethtool_sset: autoneg=1, duplex=0, speed=10 NP6LITE: wan1 link up, 1000Mbps NP6LITE: Change LIF 0 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif wan1 opened NP6LITE: lif wan1 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 0 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif wan1 opened ADDRCONF(NETDEV_UP): wan1: link is not ready NP6LITE: lif wan2 closed NP6LITE: wan2 link up, 1000Mbps NP6LITE: np6lite_lif_ethtool_sset: autoneg=1, duplex=0, speed=10 NP6LITE: Change LIF 1 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif wan2 opened NP6LITE: lif wan2 closed NP6LITE: np6lite_hw_tbl_lif_update_mac: update mac NP6LITE: Change LIF 1 MTU from 1500 to 1500 NP6LITE: HIF update rx buf len to 16384 NP6LITE: lif wan2 opened ADDRCONF(NETDEV_UP): wan2: link is not ready EXT3-fs: barriers not enabled kjournald starting. Commit interval 5 seconds EXT3-fs (mmcblk0p1): using internal journal EXT3-fs (mmcblk0p1): mounted filesystem with writeback data mode NP6LITE: wan1 link up, 1000Mbps NP6LITE: wan2 link up, 1000Mbps A9mp-i2c fb005000.i2c: i2c_xfer status9260

[localhost]

Does '#diagnose debug config-error-log read' show some information?

If cmdb crashes it will initiate a reboot afair.

[btp]

Nothing there..

[bommi]

Go to "config sys global" and check the values for "cfg-save" and "cfg-revert-timeout".

[localhost]

bommi wrote:

Go to "config sys global" and check the values for "cfg-save" and "cfg-revert-timeout".

Ah.. this good very will be the problem :)

https://kb.fortinet.com/k....do?externalID=FD30912

[btp]

Never seen this .. However; that indeed was the problem. Thanks a bunch!

However - I never got the prompt

        # System will reboot if no input is received in the next xx seconds...

So something is fishy 

(How come I can't set your post as 'helpful' or 'answer'?)

[bommi]

btp wrote:

However - I never got the prompt

        # System will reboot if no input is received in the next xx seconds...

This only gets displayed if you are connected using an serial console.

[btp]

I was - but guess it drowned in the debug output..

[btp]

bommi wrote:
This only gets displayed if you are connected using an serial console.

I was - but guess it drowned in the debug output..