Fortigate BGP community problem

[Mantovani]

I’m working with a BGP setup involving three FortiGate firewalls:

FW-A

FW-B

FW-C

FW-A has two BGP neighbors with FW-B: one over a point-to-point link and another over an IPsec overlay tunnel. Both neighbors advertise the same routes to FW-A. However, I need FW-B to only advertise the routes it receives via the point-to-point neighbor.

To achieve this, on FW-A I apply an outbound route-map to the point-to-point neighbor with FW-B, matching the route 192.168.10.0/24 and tagging it with the BGP community 65002:200. FW-A also sends this same route to the IPsec overlay neighbor, but without any community tag.

On FW-B, this results in receiving the 192.168.10.0/24 route from both neighbors — one copy with the community 65002:200 (from the point-to-point link), and one copy without the community (from the overlay).

FW-B also peers with FW-C. Between these two, I configure a route-map outbound on FW-B that matches routes with the community 65002:200, so that only routes received from the point-to-point neighbor (the ones tagged with 65002:200) are advertised to FW-C.

Initially, this works as expected: FW-C receives the route with the 65002:200 community, and when the point-to-point BGP neighbor between FW-A and FW-B goes down, FW-C correctly withdraws the route since it's no longer being received with that community.

The issue arises when the point-to-point neighbor comes back up: although FW-A again starts sending the route with the 65002:200 community, FW-C does not receive it anymore. It's as if BGP on FW-B does not re-evaluate or reprocess the route attributes (community) after the neighbor is re-established.

The only workaround that makes FW-C receive the route again is to perform a manual clear bgp all on both FW-B and FW-C.

After FW-C point to point neighbor come up, FW-B has best route over another neighbors to 192.168.10.0/24 and that´s why it insn´t propagating to FW-C. When I keep only point to point neighbor it works as expected. My routing table will be dynamic because of SDWAN over a lot of neighbors. Anyone has any idea how to solve this problem?

[Toshi_Esumi]

Please share us the route-maps at FW-B and FW-C first before doing some debug like "get router info bgp neighbor xxxx received-routes".

Toshi

[Toshi_Esumi]

Or, rather much simpler. Try configuring ECMP at FW-B for either eBGP or iBGP depending on your ASN settings.
https://docs.fortinet.com/document/fortigate/7.0.0/new-features/823985/ecmp-routes-for-recursive-bgp...

Toshi

[Toshi_Esumi]

And, your objective is not so clear to me. Do you want FW-C NOT to be able to connect to FW-A if the point2point link between A-B is up?

[Mantovani]

Actually my environment has the eBGP multipath enabled. I have to communicate my local networks via SDWAN overlay (with alot of neighbors) between these 2 firewalls (A and B). But FW-C is a network from Oracle. The traffic needs to go only in the point 2 point link. The other IPsecs is for traffic between local networks. Each firewall has they´re own AS. FW-A needs to go to oracle networks at FW-C. If the point 2 point is down, FW-B stills receive routes from IPSec overlay from FW-A and keep advertising to FW-C. That´s why I tried to use bgp community. If my point to point is down, FW-B will not advertise FW-A routes to FW-C.

[Toshi_Esumi]

Are you saying Oracle has a direct point2point to FW-A? Can you post a simple diagram with ASNs in it?

Toshi

[Mantovani]

Here is attached.

FW-A needs to go to oracle via P2P with FW-B. But need´s to use only P2P link. When this P2P goes down, FW-C needs to stop to receive routes from FW-A, but still receives because of SDWAN between FW-B and A. That´s why I tried to set community. To filter out only routes learned with community

[Toshi_Esumi]

Then you are adding the community to p2p advertised routes and passing ONLY those routes at FW-B when it re-advertise it to FW-C. Since you're enabling ECMP for BGP at FW-B, those two sets of routes at FW-B should be co-existing. So when the p2p came back up from outage, the FW-B should be able to re-advertise the p2p route to FW-C again.
Check with "get router info bgp network x.x.x.x/x" at FW-B by picking one of those routes from FW-A when the p2p came back up. Do you see two routes there?

Toshi

[Toshi_Esumi]

Also share us the route-map you have at FW-B for FW-C neighbor config with "set route-map-out".

Toshi

[Mantovani]

It should be, but it isn´t lol. FW-B receive routes from FW-A after p2p came back, but it didn´t advertise to FW-C. This is happening because on the routing table of FW-B, the best route to FW-A is through SDWAN overlay, that´s why it isn´t advertising the route with community to FW-C. If I disable SDWAN neighbors, everythig works fine. Very weird