Technical Tip: Identify a LOOP layer 2 under FortiSwitch

[ehamud]

Description	This article describes how to identify a present loop with FortiSwitch with Flags.
Scope	All FortiSwitch models and versions.
Solution	In this article, it can be proven why STP is important to be enabled under each port Switch, essentially this feature allows to prevent loops, here in this lab the STP is already disabled under FortiSwitch-6 and FortiSwitch-5 on its ports, including the topology:                                    FortiSwitch-6: 108DVI9WGUAZXBA # config switch interface S108DVI9WGUAZXBA (interface) # edit port2 S108DVI9WGUAZXBA (port2) # set stp-state disabled S108DVI9WGUAZXBA (port2) # next S108DVI9WGUAZXBA (interface) # end The same commands for 8DV-QG474QU2A-0 trunk is applied. STP is now disabled, let's check with the command diagnose stp instance list, as seen in port2 and 8DV-QG474QU2A-0 trunk does not have the FLAG EN = Enabled, all ports are under Role Designated and Status Forwarding: Port               Speed   Cost       Priority   Role         State        HelloTime  Flags   ________________   ______  _________  _________  ___________  __________   _________  _______________   port2              1G      20000      128        DESIGNATED   FORWARDING   2            port3              -       200000000  128        DISABLED     DISCARDING   2          ED   port4              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port5              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port6              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port7              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port8              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   internal           1G      20000      128        DESIGNATED   FORWARDING   2          ED   8DV-QG474QU2A-0    1G      1          128        DESIGNATED   FORWARDING   2            Flags: EN(STP enable), ED(Edge), LP(Loop Protection Triggered)   RG(Root Guard Triggered), BG(BPDU Guard Triggered), IC(PVST Port Inconsistent)   MV(PVST Port Vlan Mismatch) With the FortiSwitch 5, port3, and trunk 8DVI9WGUAZXBA-0 the stp configuration will be disabled, the FLAG EN is not present anymore, and all ports are under Role Designated and Status Forwarding: Port               Speed   Cost       Priority   Role         State        HelloTime  Flags   ________________   ______  _________  _________  ___________  __________   _________  _______________   port2              -       200000000  128        DISABLED     DISCARDING   2          ED   port3              1G      20000      128        DESIGNATED   FORWARDING   2            port4              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port5              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port6              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port7              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   port8              1G      20000      128        DESIGNATED   FORWARDING   2          EN ED   internal           1G      20000      128        DESIGNATED   FORWARDING   2          ED   8DVI9WGUAZXBA-0    1G      1          128        DESIGNATED   FORWARDING   2          ED   Flags: EN(STP enable), ED(Edge), LP(Loop Protection Triggered)   RG(Root Guard Triggered), BG(BPDU Guard Triggered), IC(PVST Port Inconsistent)   MV(PVST Port Vlan Mismatch) When proceeding sending little traffic from FortiSwitch-6 to FortiSwitch-5 the CPU process increased even the latency increased: 108DVI9WGUAZXBA # exe ping 20.20.20.2 PING 20.20.20.2 (20.20.20.2): 56 data bytes 64 bytes from 20.20.20.2: icmp_seq=23 ttl=64 time=2.9 ms 64 bytes from 20.20.20.2: icmp_seq=24 ttl=64 time=3.5 ms 64 bytes from 20.20.20.2: icmp_seq=26 ttl=64 time=1000.6 ms 64 bytes from 20.20.20.2: icmp_seq=91 ttl=64 time=4.0 ms 64 bytes from 20.20.20.2: icmp_seq=92 ttl=64 time=3.3 ms 64 bytes from 20.20.20.2: icmp_seq=93 ttl=64 time=3.2 ms 64 bytes from 20.20.20.2: icmp_seq=98 ttl=64 time=1000.7 ms 64 bytes from 20.20.20.2: icmp_seq=149 ttl=64 time=4.0 ms 64 bytes from 20.20.20.2: icmp_seq=150 ttl=64 time=3.1 ms 64 bytes from 20.20.20.2: icmp_seq=154 ttl=64 time=3.3 ms 64 bytes from 20.20.20.2: icmp_seq=155 ttl=64 time=3.0 ms 64 bytes from 20.20.20.2: icmp_seq=156 ttl=64 time=1.8 ms As shown, the FortiSwitch-5 detected high CPU levels from 9% to 28%, imagine if in a real production network, there is more traffic, the network will collapse: S108DV-QG474QU2A # get system performance status CPU states: 22% user 9% system 0% nice 69% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 15% user 7% system 0% nice 78% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 15% user 7% system 0% nice 78% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 15% user 7% system 0% nice 78% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 23% user 8% system 0% nice 69% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 21% user 9% system 0% nice 70% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes S108DV-QG474QU2A # get system performance status CPU states: 21% user 9% system 0% nice 70% idle Memory states: 28% used Uptime: 5 days,  0 hours,  40 minutes Let's check the Cisco Switch all ports are under Role Designated and Status Forwarding: Spanning tree-enabled protocol IEEE:   Root ID    Priority    1              Address     0c99.2377.0000              This bridge is the root              Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec   Bridge ID  Priority    1      (priority 0 sys-id-ext 1)              Address     0c99.2377.0000              Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec              Aging Time  300 sec Interface           Role Sts Cost      Prio.Nbr Type ------------------- ---- --- --------- -------- -------------------------------- Gi0/0               Desg FWD 4         128.1    P2p Gi0/1               Desg FWD 4         128.2    P2p Gi0/2               Desg FWD 4         128.3    P2p Gi0/3               Desg FWD 4         128.4    P2p Let's fix the issue enabling the STP under the required ports, do the same with both Switches 5 and 6: FortiSwitch-6: S108DVI9WGUAZXBA (interface) # edit port2 S108DVI9WGUAZXBA (port2) # show config switch interface     edit "port2"         set stp-state disabled         set auto-discovery-fortilink enable         set snmp-index 2     next end S108DVI9WGUAZXBA (port2) # set stp-state enabled S108DVI9WGUAZXBA (port2) # next S108DVI9WGUAZXBA (interface) # edit 8DV-QG474QU2A-0 S108DVI9WGUAZXBA (8DV-QG474QU2A-0) # set stp-state enabled S108DVI9WGUAZXBA (8DV-QG474QU2A-0) # next S108DVI9WGUAZXBA (interface) # end   FortiSwitch-5: S108DV-QG474QU2A # config switch interface S108DV-QG474QU2A (interface) # edit port3 S108DV-QG474QU2A (port3) # show config switch interface     edit "port3"         set stp-state disabled         set auto-discovery-fortilink enable         set snmp-index 3     next end S108DV-QG474QU2A (port3) # set stp-state enabled S108DV-QG474QU2A (port3) # next S108DV-QG474QU2A (interface) # edit 8DVI9WGUAZXBA-0 S108DV-QG474QU2A (8DVI9WGUAZXBA-0) # show config switch interface     edit "8DVI9WGUAZXBA-0"         set allowed-vlans 1-4094         set dhcp-snooping trusted         set stp-state disabled         set edge-port disabled         set snmp-index 12     next end S108DV-QG474QU2A (8DVI9WGUAZXBA-0) # set stp-state enabled S108DV-QG474QU2A (8DVI9WGUAZXBA-0) # next S108DV-QG474QU2A (interface) # end S108DV-QG474QU2A # Let's execute the ping again from FortiSwitch 6 to FortiSwitch 5, and it is noticeable that the CPU and latency is less: S108DVI9WGUAZXBA # exe ping 20.20.20.2 PING 20.20.20.2 (20.20.20.2): 56 data bytes 64 bytes from 20.20.20.2: icmp_seq=0 ttl=64 time=3.6 ms 64 bytes from 20.20.20.2: icmp_seq=1 ttl=64 time=1.7 ms 64 bytes from 20.20.20.2: icmp_seq=2 ttl=64 time=2.7 ms 64 bytes from 20.20.20.2: icmp_seq=3 ttl=64 time=3.4 ms 64 bytes from 20.20.20.2: icmp_seq=4 ttl=64 time=3.2 ms 64 bytes from 20.20.20.2: icmp_seq=5 ttl=64 time=3.2 ms 64 bytes from 20.20.20.2: icmp_seq=6 ttl=64 time=3.7 ms 64 bytes from 20.20.20.2: icmp_seq=7 ttl=64 time=4.1 ms 64 bytes from 20.20.20.2: icmp_seq=8 ttl=64 time=2.7 ms 64 bytes from 20.20.20.2: icmp_seq=9 ttl=64 time=3.7 ms 64 bytes from 20.20.20.2: icmp_seq=10 ttl=64 time=4.0 ms 64 bytes from 20.20.20.2: icmp_seq=11 ttl=64 time=3.8 ms FortiSwith-5 with 0% of CPU use: S108DV-QG474QU2A # get system performance status CPU states: 0% user 1% system 0% nice 99% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes S108DV-QG474QU2A # get system performance status CPU states: 0% user 0% system 0% nice 100% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes S108DV-QG474QU2A # get system performance status CPU states: 0% user 0% system 0% nice 100% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes S108DV-QG474QU2A # get system performance status CPU states: 0% user 0% system 0% nice 100% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes S108DV-QG474QU2A # get system performance status CPU states: 0% user 0% system 0% nice 100% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes S108DV-QG474QU2A # get system performance status CPU states: 0% user 0% system 0% nice 100% idle Memory states: 28% used Uptime: 5 days,  1 hours,  3 minutes Also, the Cisco Switch can ping the FortiSwitch-5, here are the before and after: Without STP enabled in FortiSwitch ports: Switch#ping 20.20.20.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 20.20.20.2, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) Switch# With STP enabled in all ports: Switch#ping 20.20.20.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 20.20.20.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 ms Switch# This is because now port3 under FortiSwitch-5 has broken the LOOP with role ALTERNATIVE and state DISCARDING: S108DV-QG474QU2A # diagnose stp instance list MST Instance Information, primary-Channel: Instance ID 0 (CST) Config Priority 32768 Bridge MAC 0c2bdef10000, MD5 Digest 9999b43d77cc58bba8854f9991c4a487 Root MAC 0c9923770000, Priority 1, Path Cost 20000, Remaining Hops 125 Regional Root MAC 0c2947b00000, Priority 32768, Path Cost 1, Root Port 8DVI9WGUAZXBA-0 Active Times Forward Time 15, Max Age 20, Remaining Hops 125 TCN Events Triggered 13 (0d 0h 15m 42s ago), Received 199 (0d 0h 14m 47s ago) Port Speed Cost Priority Role State HelloTime Flags ________________ ______ _________ _________ ___________ __________ _________ _______________ port2 - 200000000 128 DISABLED DISCARDING 2 ED port3 1G 20000 128 ALTERNATIVE DISCARDING 2 EN port4 1G 20000 128 DESIGNATED FORWARDING 2 EN ED port5 1G 20000 128 DESIGNATED FORWARDING 2 EN ED port6 1G 20000 128 DESIGNATED FORWARDING 2 EN ED port7 1G 20000 128 DESIGNATED FORWARDING 2 EN ED port8 1G 20000 128 DESIGNATED FORWARDING 2 EN ED internal 1G 20000 128 DESIGNATED FORWARDING 2 ED 8DVI9WGUAZXBA-0 1G 1 128 ROOT FORWARDING 2 EN Flags: EN(STP enable), ED(Edge), LP(Loop Protection Triggered) RG(Root Guard Triggered), BG(BPDU Guard Triggered), IC(PVST Port Inconsistent) MV(PVST Port Vlan Mismatch)