I have a Fortigate 601E v6.4.9 as a switch controller, with 2 FotiSwitch 1048E as MCLAG peers and multiple 124F on distribution.
Few switches suddenly went offline including 1x 1048E.
in LOGS it shows
NTP is fine.
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
the most common reason for this issue is a network loop caused for cabling issues and misconfiguration on the MCLAG - ICL link between peers or between Tier1 and Tier2 switches.
Please open a TAC ticket if you need still support.
Thanks for your reply.. how can I figured it out. Any possibility through CLI.
i am afraid this is too complex to check it using forum tools. Seems your networks is some kind complex.
there is some usefull comands on FGT side
execute switch-controller diagnose-connection
execute switch-controller get-conn-status
(take note of the fortilink interface/stack name)
execute switch-controller get-physical-conn standard FortiSwitch-Stack-FortiLink
Please check with this last command the name of the fortilink stack and change if it is necessary
diagnose switch-controller switch-info mclag peer-consistency-check
on FSW side you can try to check for non planed ports on STP blocked or backup status
diagnose stp instance list
also check if your intended network topology is matching LLDP outputs.
get switch lldp neighbors-summ
this command is assuming all relevant devices have LLDP enabled. Per default it is enabled on all FSW devices, but be aware some other 3rd party devices may not have it enabled, so doble check with STP diagnose command if these ports are not blocked.
i encourage you if this is an urgent matter, get in contact with TAC support.
thanks. Will check these cmds. Everything was smooth from last 3/4 months. Don’t know suddenly it went like this. Will share the output.
Below the results from Fortigate
FortiGate-601E # execute switch-controller diagnose-connection
Fortilink interface ... OK
fortilink enabled
DHCP server ... OK
fortilink enabled
NTP server ... OK
fortilink enabled
NTP server sync ... OK
synchronized: no, ntpsync: enabled, server-mode: enabled
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:98
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:486
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:587
no data
ipv6 server(.ntp.org) unresolved -- unreachable(0xff) S:0 T:1
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:926
no data
HA mode ... disabled
#######################################################
*******************************************************
execute switch-controller get-conn-status
only core switches are Authorized/UP all distribution switches are Authorized/Down
####################################################
***************************************************
FortiGate-601E # execute switch-controller get-physical-conn standard fortilink
This will display connectivity graph information for FortiLink from FortiGate's perspective
NOTE : If FortiSwitch is not authorized, no connectivity information will be shown
NOTE : If FortiSwitch is in idle state, no connectivity information will be shown
NOTE : If FortiSwitch ISL peer has inconsistent info, no connectivity information will be shown
FortiLink interface : fortilink
FortiGate(s)
FG6H1ETB2*******(x2) <<------------------>> FS1E48T****00213(port48)
Tier 1
FS1E48T****00213(port48) <<------------------>> FG6H1ETB*****(x2)
Tier 2+
FS1E48T*****242(port45/E48T4210****-0) <<------------------>> FS1E48T*****213(port45/E48T4210****-0)
############################################
********************************************
FortiGate-601E # diagnose switch-controller switch-info mclag peer-consistency-check
Vdom: root
Managed Switch : FS1E48T****213 0
Running diagnostic, it may take sometime...
** Comparing "switch.global.mclag-split-brain-detect" config ....OK
** Comparing "switch.global.mclag-split-brain-all-ports-down" config ....OK
mclag-trunk-name peer-config lacp-state stp-state local-ports remote-ports
__________________ ___________ __________ _________ _____________ _____________
AGG-Y NOT-FOUND UNEVEN MISMATCH port1 port2
port3 port4
port5 port6
port7 port8
port10 port11
port12 port16
E48T*****242-0* OK UP OK port45 port18 port45
G6H1ET****058 OK UP OK port48 port48
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
switch:S124F does not support MCLAG feature!
Managed Switch : FS1E48T*****242 0
Running diagnostic, it may take sometime...
** Comparing "switch.global.mclag-split-brain-detect" config ....OK
** Comparing "switch.global.mclag-split-brain-all-ports-down" config ....OK
mclag-trunk-name peer-config lacp-state stp-state local-ports remote-ports
__________________ ___________ __________ _________ _____________ _____________
AGG-F1 NOT-FOUND UNEVEN MISMATCH port1
AGG-X NOT-FOUND UNEVEN MISMATCH port2 port3
port4 port5
port6 port7
port8 port9
port10 port12
port13 port14
port15 port16
E48T******213-0* OK UP OK port18 port45 port45
G6H1ET******058 OK UP OK port48 port48
What OS are stable for this. Currently :
Fortigate on 7.0.0
Forti SW 6.4.7/7.2.1
Hi,
We recommend FSWs to be on latest version and all FSWs to be on same version. So, if you are having FSWs in 6.4.x, upgrade to 6.4.11. Similarly if 7.2, then 7.2.1 on all FSWs.
When any FSW goes offline and comes back online, check the FSW logs for any log.
On FSW CLI:-
exec log filter view-lines 1000
exec log display
check for crash - diag debug crashlog read
Also, there could be a possibility where due to any one FSW in network, it could cause issue on all FSWs which causes the fortilink process to go high. So as first step, i recommend upgrading FSWs.
we encourage to use any R marked combo mentioned on this KB
https://docs.fortinet.com/document/fortiswitch/7.2.1/fortilink-compatibility
about your issue is it very likely due your FGT lost NTP sync
FortiGate-601E # execute switch-controller diagnose-connection
Fortilink interface ... OK
fortilink enabled
DHCP server ... OK
fortilink enabled
NTP server ... OK
fortilink enabled
NTP server sync ... OK
synchronized: no, ntpsync: enabled, server-mode: enabled
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:98
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:486
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:587
no data
ipv6 server(.ntp.org) unresolved -- unreachable(0xff) S:0 T:1
no data
ipv4 server(.ntp.org) -- unreachable(0x0) S:7 T:926
no data
HA mode ... disabled
it is critical to build management tunnels to have properly configured on FGT
an external NTP server.
as time drift increases between FGT and FSW units, management tunnels will lost and unable to come up.
use this kb to solve NTP issues on FGT side
this will likely solve "down" issue on access FSW.
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1705 | |
1093 | |
752 | |
446 | |
230 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.