Fortigate 600D & 23 Managed FortiSwitches Network File Copy/Wireshark issues

[rmoat]

Hello,

Having a new Fortinet network, we are seeing some very strange issues on the network. It is very sporadic, which is hard to pinpoint the issue, as network performance works great and the next minute it fluctuates.

First we started seeing that a file copy to our file server (1 GB connection) was occasionally fluctuating fast and slow. When copying files it would burst up to 108 MB/s, then drop down to 10 MB/s or below (sometimes pause for a couple seconds), then increase up to 50 - 80 MB/s, and would form this wave.

The second file copy we tried, it would copy around 100 MB/s all the way across (2 GB file copy). So then we try it again a minute later, and the performance is horrible.

I noticed a couple things when I started a wireshark capture: Occasionally I will see TCP Spurious Retransmission, TCP ACKed unseen segment, TCP Out-Of-Order, TCP Dup ACK (even when I'm not really actively doing anything on the network).

Now here is the strange thing, if I start a wireshark capture and have someone on the same VLAN as me start a file copy to the file server (different VLAN), my Wireshark picks up hundreds of packets all TCP Dup ACK, TCP Retransmission, TCP Spurious Retransmission between the other computer and the file server. I should not be seeing these packets. It's as if the FortiSwitch is acting as a HUB rather than a switch.

Whatever is happening is completely deteriorating our network performance. And because it's so sporadic, we'll think we have it solved, and then it starts doing the same again.

-R-

[MikePruett]

Is this occurring regardless of where you are on the network? IE, does machine connected to switch 1 do this just like machine on switch 2? I always check layer 1 first.

[rmoat]

It looks like it isn't. The speeds are still very up and down on this other switch, but within Wireshark I only see a couple Name Query NB packets. I do not see the TCP Dup Acks or TCP Spurious Retransmission from the machine on a different switch (same and/or different VLAN).

[rmoat]

Just an FYI, we found the solution.

It was a problem with STP. In Fortilink mode, we've learned that you need to set Edge Port to "disabled" for the ports connected via Fortilink. After changing each switch, the performance of the network is much better. Also file copies are quick!

[MikePruett · Accepted solution]

Awesome to hear. You would think that would be an automatic action once the device is placed in FortiLink!

[rmoat]

Exactly! You'd think that standalone versus managed mode would automatically change. Perhaps it is something that will be addressed in the future. Thanks, Mike.

MikePruett wrote:

Awesome to hear. You would think that would be an automatic action once the device is placed in FortiLink!

[josh]

Wow, okay - excuse the necropost, but I just stumbled upon this thread and it's solved a nightmare of an issue I've been having lately. I was having issues with connectivity, speed and just random timeouts all over the show.

I was seeing issues similar to you, and I was tearing my hair out over it. For some stupid reason, FortiLink ports have STP and "Edge Port" status enabled. I've disabled this and it's resolved my problem. Not sure if this is related to having my single FortiSwitch-224E-PoE connected to a FortiGate-60F via LACP (FortiLink) or not, but I would suspect not. 

Thankfully this was just at home and wasn't in a customer/production environment, but a nasty gotcha nonetheless. Running FortiOS v6.4.5 and FortiSwitch v6.4.6 -- clearly this has been a long standing issue given your post was in 2017.