New FG1000D HA Deployment Using Nexus VPC Troubleshooting

Forum|Forum|7 years ago
June 6, 2018
3 replies
25132 views

Hi all, I'm deploying a new pair of 1000D in active/active configuration. Outside interface's are connecting to a single ISP & seem to be working fine. The issue I'm having is communication between our core Nexus 9K's & the 1000D's. This is a multi-tenant environment and therefore we are leveraging VDOM's on the FG & VRF's on the 9K's.

I'm using individual /29 networks between the FG & 9K's to route. We are also using LACP on the FG & VPC on the pair of 9K's. The 9K's are not new & have been in production for quite some time. They are configured according to Cisco best practices.

The issue we ran into was that we were unable to communicate properly between the core switches & the firewalls on one of the /29 networks. After working with Fortinet Support for hours, I was told that best practices was for a L2 switch to be physically installed between the firewalls and our core switches. This apparently is due to some requirement for HA configuration. Even though this wasn't clear to me, we did try this method and it didn't resolve the issue.

The physical ports on the FG are configured like this. 802.3ad port is in the root VDOM with no L3 config. The underlying VLAN interfaces are in a unique VDOM. These VLAN interfaces are what use the /29 network to route to the Nexus 9K's. Is there something about this design that doesn't seem right? Any ideas what might be causing this problem? Running 5.6.4 code.

FortiGate to 9K Topology.JPG

Best answer by ede_pfau

Cisco Nexus internally use non-standard Ethertypes - the same which FortiOS uses on the HA links. This is documented in the [strike]KB.[/strike] HA chapter of the 'FortiOS Handbook'.

To avoid trouble you can change the Ethertype in the HA setup ('conf sys ha'). 3 different types are used.

Your routing / connectivity problems could be a side-effect of this.

Toshi_Esumi

SuperUser

To me it should work. But for our similar case, we use a /30 subnet on the vlan to connect a vdom to each customer' VRF because it's a-p setup. We don't have any Nexus SW in the mix though.

emnoc

New Member

That should be doable, the /29or /30 is not revelent. Since you mention NXOS are they acting liek l3 routers? if so per cisco this will not work ideally for dynamic routing protocols

read this and review if anything here is in your design.

http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/

claytonmeyerAuthor

New Member

The only reason we're using a /29 rather than a /30 is we need additional IP addresses to accommodate HSRP (VIP) on Nexus 9K's. The 9K's are our core switches so they are doing layer-3. However, we are not doing any dynamic routing protocols so the dynamic route peering over VPC does not apply in this situation.