Async routing with specific fg cluster access and VRRP

[scheuri]

Dear community

I likely have a very specific issue that might be completly "normal", I just want to make sure I have my bases covered...

Situation:
We have two clusters (four fortigates in total) in two different data centers (dc 1 and dc 2). About a dozen of VLANs are connected to both of these two clusters and we use VRRP spanned over these vlans to ensure usage of both clusters.

The first three IPs in each vlan is ours - the .1 is the VRRP IP that is active on the master, .2 is the cluster on dc 1 and .3 is the cluster on dc 2.

Now we run into asynchronus routing with a specific use case:

There are location specific networks (one for dc 1 and one for dc 2) that access the respective local fortigate cluster and are allowed access to the attached vlans.

If you happen to be in the location where the VRRP master is, then everything works. You can access the local fortigate cluster (which is vrrp master) and access the hosts in the vlans. No Problem.

If you happen to be in the other location, there the fortigate cluster is backup, then nothing works. The reason very likely is, that the hosts in the vlans repsond back to the vrrp gateway IP which happens to be in the other location and therefore we have async routing.

Now, I am confident there is nothing I can do on the fortigates in terms of configuration (interfaces, vrrp, etc.). 

There might be design solutions, yes - but that would require more than just "configuration on fortigate".

Am I right or am I missing something vital that might solve this issue?

Thanks a lot

[Toshi_Esumi]

I had to test it with flow debugging on both sides. And even one vlan to another doesn't get the return packet back.

Say vlan 10 has vrip 10.0.10.1 and DC1 is master and local IP is .2, DC2 is backup and local IP is .3. Then vlan 30 has vrip 10.0.30.1  and DC1 is backup and local ip is .2, DC2 is master and local IP is .3.

When you send ping from 10.0.10.100 in vlan10 sends ping toward 10.0.30.100 in vlan20, the ping packets hit DC1 first and "routed" from vlan10 interface to vlan30 interface because it has 10.0.30.2 locally. But when that packet hits DC2 from vlan30, it checks reverse path, which would be "routed" at the DC2 into vlan10 because DC2 has 10.0.10.3 locally. This doesn't go back to vlan30 where it came from. So the DC2 drops it as "reverse path check fail, drop".

At this moment, I have to say you're right and I don't see any way around unless you make both FWs just routers by "set asymroute enable". This would work fine if both are just L3 routers, like Cisco, Juniper, etc.

Or if you really want to make this design work, you have to split between FW features and router features by vdoms. But then you can't do FWing between vlans since traffic between vlans don't hit FW vdoms.

Toshi

[Toshi_Esumi]

[Filter: Competitors] Toshi_Esumi's post body matched "cisco", board "fortinet-discussion".

Post Subject: Re: Async routing with specific fg cluster access and VRRP

Post Body:

I had to test it with flow debugging on both sides. And even one vlan to another doesn't get the return packet back.

Say vlan 10 has vrip 10.0.10.1 and DC1 is master and local IP is .2, DC2 is backup and local IP is .3. Then vlan 30 has vrip 10.0.30.1  and DC1 is backup and local ip is .2, DC2 is master and local IP is .3.

When you send ping from 10.0.10.100 in vlan10 sends ping toward 10.0.30.100 in vlan20, the ping packets hit DC1 first and "routed" from vlan10 interface to vlan30 interface because it has 10.0.30.2 locally. But when that packet hits DC2 from vlan30, it checks reverse path, which would be "routed" at the DC2 into vlan10 because DC2 has 10.0.10.3 locally. This doesn't go back to vlan30 where it came from. So the DC2 drops it as "reverse path check fail, drop".

At this moment, I have to say you're right and I don't see any way around unless you make both FWs just routers by "set asymroute enable". This would work fine if both are just L3 routers, like Cisco, Juniper, etc.

Or if you really want to make this design work, you have to split between FW features and router features by vdoms. But then you can't do FWing between vlans since traffic between vlans don't hit FW vdoms.

Toshi

Body text "Cisco" matched filter pattern "cisco".

Post by User[id=14083,login=Toshi_Esumi] has message uid 200818.

Link to post: Re: Async routing with specific fg cluster access and VRRP

[scheuri]

Thank you , Toshi

Your insights and testing are very much appreciated!

[Toshi_Esumi]

Sorry. My setup was not appropriate to test between two VLANs. I was using only one laptop as vlan10 client and pinging 10.10.30.3 on DC2.
When I actually set a destination client device (another FGT) in vlan30, and ping it form the laptop to it, the ping packet directly got to the destination device only via DC1. But ping reply hits DC2 because it's the master of vlan30 and got "no session matched" in flow debug and drops, since DC2 didn't see the ping request at all.

So the bottom line is the same.

[commsrbrad1]

Has there been any fix for this problem from Fortigate?