- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
1-tier MCLAG network resets when access switches connected to both dist switches
My network (currently 1x GW, 2x dist switches, and multitude of access switches) is set up for mclag. Our ultimate goal is to have a full HA system. Our 2nd gateway hasn't been added till I resolve the layer 2 issues.
Both dist switches connect to the gateway over a combined 8x ports, 4 per dist switch. The dist switches are connected to each other via 2x physical links (mclag-icl).
All access switches were initially connected to one dist switch. As a second link is activated on the second dist switch, the entire network resets and all switches lose conn. It takes almost 45mins for the network to normalize. Traffic flows over both links when things work but we haven't tested removing one link for fear of generating another 45min long "hold your breath that things normalize" event. Any help on what we're doing wrong would be appreciated.
Here's a config from one dist switch:
edit "DIST-ICL"
set port-selection-criteria src-dst-ip
set description ''
set mode static
set bundle disable
set auto-isl 0
set fortilink 0
set isl-fortilink 0
set mclag-icl enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port1" "port2"
next
edit "GATEWAY"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 1
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port22" "port21" "port24" "port23"
set lacp-speed slow
set aggregator-mode bandwidth
next
edit "Access1"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 0
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port9"
set lacp-speed slow
set aggregator-mode bandwidth
next
And here's from the other dist switch:
edit "DIST-ICL"
set port-selection-criteria src-dst-ip
set description ''
set mode static
set bundle disable
set auto-isl 0
set fortilink 0
set isl-fortilink 0
set mclag-icl enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port1" "port2"
next
edit "GATEWAY"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 1
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port22" "port24" "port21" "port23"
set lacp-speed slow
set aggregator-mode bandwidth
next
edit "Access1"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 0
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port7"
set lacp-speed slow
set aggregator-mode bandwidth
next
Also running a check shows some mismatches but I'm not sure where to start tshooting that:
diag switch mclag peer-consistency-check
Running diagnostic, it may take sometime...
** Comparing "switch.global.mclag-split-brain-detect" config ....OK
** Comparing "switch.global.mclag-split-brain-all-ports-down" config ....OK
mclag-trunk-name peer-config lacp-state stp-state local-ports remote-ports
__________________ ___________ __________ _________ _____________ _____________
Access1-0 MISMATCH UP OK port7 port11
GATEWAY MISMATCH UP OK port21 port22 port21 port22
DIST-ICL* MISMATCH UP OK port1 port2 port1 port2
Solved! Go to Solution.
- Labels:
-
FortiGate
-
FortiSwitch
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So for those wondering, and for posterity, the issue stemmed from the fact I created the MCLAG ICL between my two switches, rather than letting the switch do so itself. Initially when I configured this, the switches created their own mclag however I didn't like the name and so I recreated - or thought I did.
So we deleted the ICL mclag, rebooted one of the switches and let the network create things itself. The only action needed from my part was to ensure the proper lldp-profile was applied to the ICL ports. This ensured the ports created an MCLAG ICL trunk rather than a lag or simple ICL trunk (where one link would remain down to prevent a loop.)
Once complete I was able to turn down ports and turn them up, perform firmware upgrades and more on the l2 stack without the entire network flapping. Thanks to the Fortinet phone support person that helped with this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What I've seen so far from the fortiswitch documentation, is I'm missing lacp-active on the DIST-ICL as well as auto-isl. I also shouldn't have these on the links facing the gateway. That said, I'm not sure if this is just an ommission in the document or they shouldn't be there.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Any takers on help with this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Are you running Standlone or FortiGate Integrated?
What is the trunk config on the access switch?
Graham
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Apologies for the late reply. I need to check more frequently. Switches are all fortigate integrated and not standalone. Here's the config on one switch.
config switch trunk
edit "_FlInK1_MLAG0_"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 0
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port52" "port50"
set lacp-speed slow
set aggregator-mode bandwidth
next
end
And from a different switch:
edit "_FlInK1_MLAG0_"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 0
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port47" "port52"
set lacp-speed slow
set aggregator-mode bandwidth
next
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
And from one 1024E in our environment:
edit "_FlInK1_MLAG0_"
set port-selection-criteria src-dst-ip
set description ''
set mode lacp-active
set member-withdrawal-behavior block
set bundle disable
set auto-isl 1
set fortilink 0
set isl-fortilink 0
set mclag enable
set static-isl disable
set static-isl-auto-vlan enable
set members "port23" "port24"
set lacp-speed slow
set aggregator-mode bandwidth
next
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Our fortigate is in (HA) though the second fortigate has not been introduced. It's defacto operating in standalone regardless of the "HA Primary" shown in the gui. The issue precedes us movign the fortigate into HA - it's been going on for a long while.
We upgraded the fortigate this weekend and the network never quite recovered. We ultimately powered off one of our distro switches and things stabilized.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
So for those wondering, and for posterity, the issue stemmed from the fact I created the MCLAG ICL between my two switches, rather than letting the switch do so itself. Initially when I configured this, the switches created their own mclag however I didn't like the name and so I recreated - or thought I did.
So we deleted the ICL mclag, rebooted one of the switches and let the network create things itself. The only action needed from my part was to ensure the proper lldp-profile was applied to the ICL ports. This ensured the ports created an MCLAG ICL trunk rather than a lag or simple ICL trunk (where one link would remain down to prevent a loop.)
Once complete I was able to turn down ports and turn them up, perform firmware upgrades and more on the l2 stack without the entire network flapping. Thanks to the Fortinet phone support person that helped with this.
