Problem passing traffic across an LACP interface

[albaker1]

We used FortiConverter to convert a Firepower configuration to a FortiGate 2600, and what we didn't realize until we were ready to put the FortiGate into production was that a couple interfaces that were NOT bonded were converted to an LACP-type interface for the FortiGates (HA pair). No problem I thought - we converted the interfaces on the Cisco switch to LACP, and both sides negotiated LACP OK and seemed to be happy.

However, traffic wouldn't flow correctly. From the FortiGate, we could type "exec ping 1.1.1.1" to ping the Internet router, and the  pings woudl time out. If we told the FortiGate to source from the outside interface, the pings succeeded. If we look at the ARP entries on either the Firepower or FortiGate, there were no entries for the other side. Also, if you look at in the mac address-table of the Cisco switch, there were no entries for the interfaces the FortiGates were plugged into. However, there is another path out to the Internet, which I believe the traffic is taking.

Can someone tell me if a FortiGate link is up, and you're trying to ping an IP on a locally connected subnet, should the traffic go out another way? It seems to me as if the pings should have failed if they couldn't pass across the switch. Are there any normal instances that would require the source interface be added before a ping will actually succeed for a directly connected host? I can't understand this behavior, so any information that will help out will be greatly appreciated. Our maintenance window was about to expire when we began troubleshooting this problem, so I have a lot of questions still to ask. 

**We also created a TAC case to move the FortiGate LACP interfaces to a non-LACP interface, but we were told we'd have to remap 270'ish rules which are referenced if we did this because of how the normalized interfaces are incorporated. Is this actually correct? Is there a trick to bypass this?

In passing, there are some other bonded interfaces on the FortiGate that connects to the same switch, and these are functioning correctly - these have multiple interfaces bonded together. The difference for the ones that are not working are just single-port LACP interfaces.

Thank you. I know this is a lot of info, but hopefully it makes some sense.

[Anthony_E]

Hello,

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible.

Thanks,

[srajeswaran]

There are few questions, lets go one bye one

1. When you try to ping/reach an IP that is directly connected to a FGT interface, it will try to learn the MAC address using ARP. In your case, the MAC learning is failing, which means the ICMP packet won't go out of fortigate

Is the LACP between FGT and Switch active? You see LACP keepalives exchanged between the peers correctly?

Can you make sure there is 2 separate LACP groups created on the switch, one towards the Master Node FW and the other one towards the Slave node FW? This will make sure, the switch is not transmitting packets to the Slave unit instead of Master. You may disconnect the links towards Slave unit and check.

Can you also share the output of below commands

get router info routing-table details 1.1.1.1

get router info routing-table details a.a.a.a --> IP on the LACP interface

get router info routing-table details b.b.b.b --> IP on the outside interface

[albaker1]

Yes, the LACP showed active on both ends of the cable. We had a vendor help us troubleshoot this, and I don't recall if we specifically looked at keep-alives. We started off with one LACP group on the (Cisco) switch, but the switch negotiates this as two separate LACP groups automatically (LACP and LACP-A), because the switch does see two different devices connected. We've have this configuration with other equipment, and it functions as expected. Also, we configured two two separate LACP groups shortly before the maintenance window had expired, and the behavior didn't change. We have another maintenance window this week to re-attempt the installation, so I'll be able to get more details then.

[gfleming]

Can you sketch up your topology briefly? It's a bit confusing with all the different paths. Just want to be sure we fully understand how everything is connected and what the logical routes are in the network.