asymmetric routing dilemma

[johnnyba]

I'm stuck with a problem that seems to be above my head because I don't have a lot of experience with complex routing. It looks like maybe the more recently added auxiliary session feature might be helpful but it's not clear to me. 

So we have two ISP's and then upper management signed a contract with Comcast for IPTV not really knowing what it was until after the fact. It requires that we get all Comcast routes within their autonomous system via BGP such that everything Comcast related goes in and out that IPTV interface. Our external IP range has to also be advertised over there to make this work. The trouble is that includes all Comcast broadband customers which creates a variety of headaches because they are so big. Most of our employees end up getting to our VPN and websites via this link that is intended for IPTV and 1/4 the size of everything else. The documentation I have from them only says that I can prevent advertisements outside of their AS with community codes but not differentiate within it. And they are less than helpful.

I can't see a way to solve the problem without turning on asymmetric  routing. IPTV is just ports 80 and 443 so the only hook is the Fortinet application signatures which, as far as I can tell, can't be used for policy routing and probably wouldn't be reliable enough if it could be. I could subnet our range so that Comcast customers see our services on different routes but then the routing table would still send Comcast traffic back out the IPTV interface. 

Is there a way on the Fortinet to force traffic back out the same interface it comes in regardless of the routing table or some other fix I am not aware of? I need to get comcast ISP costumers to use our ISP to get to us but still get other Comcast traffic to go out the IPTV circuit. Any ideas on anything else that I might be missing or should look for?

thanks

[akristof]

Hello,

Thank you for your question. I am not sure that I really understand what exactly is the problem, but your concern is reply traffic, right? By default, FortiGate will use for reply traffic same outgoing interface as it came. So if traffic came via comcast, reply traffic will go via comcast interface. In fact, in 6.4, even if you will have policy routes, they will be ignored (this behavior is fixed in 7.0):

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Return-Reply-traffic-is-not-matching-polic...

I guess the important thing would be to 2(3) valid default routes, via your original ISPs and then via comcast.

[johnnyba]

Sorry, I was in a rush and didn't describe this well. Will try to fix.  I also stupidly used the wrong term when I meant asymmetric  routing. But when I first tried to put BGP into production on this firewall we started dropping packets. I was told by Fortinet support that I had to turn on asymmetric  routing or it would be that way. What was happening was that ISP 1 and ISP 2 only send us a default route for the purpose of failover but our external range was advertised on both. The result was some stuff would come in on ISP 2 while ISP 1 was the active default route.   When it went back out it would always try to use the interface with the active default route and drop the packet as a security risk. The result was a lot of lost email. The best fix for it that worked was using conditional advertisements so that ISP 2 only advertises the router when ISP 1 is down. I thought this was odd because I assumed it tracked sessions such that replies would go out the same interface but we could see the drops. 

I have not tried to put this back into production yet because I want to make sure I get it right first. Did I misunderstand how this scenario works? Static routers take preference of default routes as well so my assumption here was that if we got a packet via ISP 1 from Comcast's IP range on the way out it would choose the static route from BGP on the Comcast interface going out. If it won't work that way then my best fix would be to subnet our external range and advertise a small part on the Comcast interface such that only traffic originating here goes out it. 

[akristof]

Hi,

Thank you for explanation. I see. Based on this yes, conditional advertisement would be the best.

Regarding second part of your question. If traffic from Comcast will come from ISP1, FortiGate will try to make reply traffic leave same interface. This is the default behavior that FortiGate has. On reply traffic there is no routing lookup made, route back to the source exists, otherwise traffic would be dropped by RFP when it first came.

[johnnyba]

I am confused about how scenario B with Comcast routes is different than scenario A with default routes. Why was it sending reply traffic back out the wrong interface if there was no route lookup when replying? 

[akristof]

I think he meant that the FortiGate was sending traffic via ISP1 but reply traffic was coming via ISP2. Is this your question?

[johnnyba]

Ok, I apologize I was not remembering all the details because the solution involved shutting off asymmetric routing and I had multiple problems involving that. (Comcast set up the circuit up wrong causing several problems and it took a year for them to finally fix it.)  The specific issue is reverse path forwarding which requires shutting off asymmetric routing to fix in the previous scenario: Technical Note : Reverse Path Forwarding (RPF) imp... - Fortinet Community

So, what happened was that Yahoo sent mail on ISP2, our Fortigate did a reverse path check on ingress, and then dropped the packet because the route was the default on ISP 1. 

But with the other Comcast issues fixed looking at this now I wonder if it would be sufficient to disable strict source checks. If I understand this correctly currently a packet coming in ISP 1 from a Comcast address would get dropped because the best route is on the comcast interface. But with feasible path it would take it and reply. Does that make sense? 

[Matt_B]

Hello,

If the Fortigate only has one active default route, this may be the source of your issue. You are correct that the loose RPF check only requires one active route in the routing table. Unfortunately, advertising a route to the Fortigate does not necessarily add it to the active routing table.

In normal operation a static route prevents a BGP route with the same prefix from becoming active. You can check the Fortigate's active routes with:
get router info routing-table all

If two active default routes are required (to pass RPF checks) but one must be preferred (for routing outbound sessions), you can create a static default route to each ISP and define a higher priority number for the default route that will NOT be preferred. This can be done by Scenario 3a in the link:
https://community.fortinet.com/t5/FortiGate/Technical-Note-Routing-behavior-depending-on-distance-an...

or via the GUI as follows

The Fortigate will then prefer to use the default route with a lower priority number for outbound traffic, but should still pass RPF checks if the default "set strict-src-check disable" is used.

Thanks,
Matt

[Muhammad_Haiqal]

Hi Johnnyba,
Normal traffic is coming from and reply on same interface.
Example: Going out to ISP1, respond coming from ISP1.

Assymetric routing means, traffic is coming from and reply coming on different interface.
Example: Going out to VPN, respond from ISP1.

Fortigate can manage how traffic go out  from Fortigate using static route, policy route, BGP, OSPF.
But doesn't have control how traffic respond back to Fortigate. As their routing table is not manage by Fortigate anymore.

For asymmetric routing, this issue is more to network design. Its the best if you have the diagram and IP information.