BGP routes will not replace higher distance routes already in the routing table

[srnoth]

Hey guys,

I've run into a couple different scenarios where the Fortigate will ignore an eBGP route (distance 20) and instead hold on to a higher distance router (e.g. static, or ospf) which it had learned about previously.  Here is an example. My Fortigate learns about routes to 10.2.137.0/24 via both BGP & OSPF. If it learns the BGP route first, and then the OSPF, it shows both in the router database, but selects the BGP route as it is lower distance:

O 10.2.137.0/24 [110/2] via 10.4.3.178, port1, 00:02:31

B *> 10.2.137.0/24 [20/0] via 10.4.3.181, Colo, 00:02:44

However, if for some reason the OSPF route is learned first, when the BGP route is learned, it never adds it to the routing database or routing table:

O *> 10.2.137.0/24 [110/2] via 10.4.3.178, port1, 00:03:36

#get router info routing-table details 10.2.137.0/24

Routing table for VRF=0 Routing entry for 10.2.137.0/24 Known via "ospf", distance 110, metric 2, best Last update 00:04:01 ago * 10.4.3.178, via port1

However it does show up in the BGP networks:

# get router info bgp network 10.2.137.0/24 BGP routing table entry for 10.2.137.0/24 Paths: (2 available, best #2, table Default-IP-Routing-Table) Advertised to non peer-group peers:

10.4.3.181 65000 65000 65000 65000 65000 65000 65000 10.4.3.181 from 10.4.3.181 Origin incomplete metric 0, localpref 100, valid, external Last update: Thu Feb 6 22:42:48 2020

Local 10.4.3.178 from 0.0.0.0 Origin incomplete metric 2, localpref 100, weight 32768, valid, sourced, best Last update: Thu Feb 6 22:42:41 2020

Is this an intended behavior, and/or is there anything I can do to get the Fortigate to replace the existing (higher distance) route with a newly discovered, lower distance eBGP route?

Thanks.

[Toshi_Esumi]

I'm assuming you're not redistributing OSPF routes into BGP domain, but redistributing static routes into BGP.

If possible, I would try stop redistributing static into BGP, then reconfigure (wipe out first then repaste the copy back in except redistribute static) BGP again to see if the "Local" route still shows up in "get router info bgp network...".

If stop redistributing static is not possible, you need to filter that particular /24 prefix out with "prefix-list" + "route-map" and attach it to config redistribute static section so that other static route will still be redistributed into BGP.

My guess is it would go away after that in the BGP database and then BGP sends its own route to RIB. I think the prefix got stuck in BGP somehow and causing the symptom. We experienced similar with real static route (default route with high cost), which got stuck in BGP.

But if you're redistributing OSPF to BGP, you need to do similar to that redistribution.

[emnoc]

YEs you redistribution is hosed up. Is that OSPF route being injected via redistribute or via BGP and the peer? 2nd have you study bgp backdoor ? In your case you typically do not learn a route via eBGP and OSPF ( igp ) at the same time. In reality, it should never happen but here you are changing the expected behavior or EGP and IGP.

Ken Felix

[srnoth]

Hey guys I think y'all are getting to the answer - this may just be expected BGP behavior?

To give you more details, our setup is as follows:

[ul]

We have a primary connection (MPLS) and a secondary connection (IPSEC) to this subnet/location

MPLS connection uses BGP for routing

IPSEC can only be static or OSPF

If the MPLS connection is up, we want that route learned via BGP to take precedence

If the remote site's MPLS connection goes down, we need the backup (IPSEC) route to take precedence, and we need to redistribute that route back out over BGP over the MPLS to other sites.[/ul]

This all works as expected if BGP comes up first. BGP installs the route in the routing table and OSPF, being a higher distance, does not get selected. If the remote site's MPLS goes down, I stop getting that route advertised over BGP, the OSPF (or higher distance static route, if I use static instead of OSPF) route gets installed in my routing table, and I start redistributing that to the other sites over BGP. All is good.  But, when the remote site's MPLS comes back online, and starts advertising over BGP again, the OSPF route over the IPSEC stays stuck in my routing table - BGP sees it as local and prefers it, even though a BGP route should be lower admin distance.

[srnoth · Accepted solution]

If anyone is interested, I figured this out. BGP's first comparison in the decision making process is Weight. 

By default, the Fortigate was assigning weight 32768 to the local (OSPF) route, but no weight (0?) to the route learned from the BGP peer:

# get router info bgp network 10.2.137.0/24

BGP routing table entry for 10.2.137.0/24 Paths: (2 available, best #2, table Default-IP-Routing-Table) Advertised to non peer-group peers: 10.4.3.181 65000 65000 65000 65000 65000 65000 65000 10.4.3.181 from 10.4.3.181 Origin incomplete metric 0, localpref 100, valid, external Last update: Thu Feb 6 22:42:48 2020   Local 10.4.3.178 from 0.0.0.0 Origin incomplete metric 2, localpref 100, weight 32768, valid, sourced, best Last update: Thu Feb 6 22:42:41 2020

I solved this by specifying a much higher weight for that neighbor, and now the remote route is "best":

# get router info bgp network 10.2.39.0/24

BGP routing table entry for 10.2.39.0/24 Paths: (2 available, best #1, table Default-IP-Routing-Table) Not advertised to any peer 65000 65000 65000 65000 65000 65000 65000 10.4.3.181 from 10.4.3.181 (208.72.133.156) Origin incomplete metric 0, localpref 100, weight 65535, valid, external, best Last update: Sat Feb 8 01:10:52 2020

Local 10.4.3.178 from 0.0.0.0 (96.94.189.105) Origin incomplete metric 2, localpref 100, weight 32768, valid, sourced Last update: Sat Feb 8 01:09:36 2020

Thus BGP injects the remote route into the routing table, even if the OSPF route was already present.

[FortiEngineer]

But how does this explain BGP just choosing not to inject the routes into the RIB?