30 seconds of packet loss when switch is re-connected

Hi Chris,

Does the traffic from host A to host B traverse through the firewall? If so, I would start by checking if there are any route changes happening on the firewall when the switch comes online. 

Also, perform a sniffer / flow debug to check if FGT is dropping the packets. 

Regards,

You need to put those FGTs in HA in the diagram and VLAN(s)/IP subnet(s). Otherwise nobody can know the topology involving the FGTs. But my feeling is it's on the switch side involving "port-fast".

Toshi

With 40+ switches I am pretty sure you have a central log storage for all of them, I'd check what log level is needed to catch STP port/VLAN state changes (Forwarding/Blocking) and look for them in the logs. Problem that occurs every 30 seconds each time seems more like a timers issue, especially with Spanning Tree Convergence times. 

HI ChrisStankevitz,

Since you mentioned the network is big, im afraid this issue due to the network deployment itself.
Please take a look on this KB:

Troubleshooting Tip: LACP issue
https://kb.fortinet.com/kb/microsites/microsite.do?cmd=displayKC&docType=kc&externalId=FD50620

High availability basic deployment design
https://kb.fortinet.com/kb/microsites/microsite.do?cmd=displayKC&docType=kc&externalId=FD47572

That KB should provide general idea on the deployment.
Hope that helps.

Thank you all for your replies.  A couple of comments which I should have included in my original post:

1. The dropped traffic between "Host 1" and "Host 2" is on the same VLAN.  No routing.  The fortigate does not see the traffic (in the logs), which is what I would expect.  So I doubt the firewalls are playing any direct role in the problem
2. @vponmuniraj what do you mean by "route changes"?  Are you talking about Layer 3?  Regardless, how do I monitor "route changes" on the switches or firewalls?
3. @Toshi_Esumi I have two Fortigates (running 6.2.2) in HA Active-Passive.  Each fortigate is connected to both "Switch A" and "Switch B".  Similar to how the hosts are connected to these switches.
4. @Yurisk I suspect STP or some fortinet ISL equivalent also.  Particularly in light of item 5 below.  However, the problem does not happen regularly at 30 second intervals.  When "Switch D" is connected to "Switch C", the traffic between "Host 1" and "Host 2" becomes "spotty"... and this misbehavior lasts for only about 30-45 seconds.  After this time, the traffic returns to normal and there is no more trouble.  The switch logs show nothing interesting.  They do show STP messages when individual ports come up and down.
5. The problem appears in an additional way also.  Please see the imaged I pasted below.  If I leave all switches connected and powered up... but disconnect and reconnect each of the ISLs listed below, 1-4 (waiting one minute between each disconnect or connection), the problem (packet loss between "Host 1" and "Host 2") will also happen temporarily.

My sales engineer suggested that a "loop" is forming between switches C and D, which doesn't make sense to me.

Complicating this:  I'm working in an air gapped environment so getting support is difficult.  I tried to replicate with spare fortigates and switches, but so far I'm unsuccessful.

Does anyone know how I can diagnose/trace the "logic" that the switches go through when "Switch D" joins the topology or when ISL connections come up and down?

Then you should post this at Cisco Community instead of Fortinet Community.

Toshi