Troubleshooting Tip: Prevent advertising a black hole route via BGP when IPsec tunnel is down

[syordanov]

Description	This article describes how to prevent black hole route advertisement to a remote BGP peer.
Scope	FortiOS v7.0.x, v7.2.x, v7.4.x, v7.6.x.
Solution	When a tunnel goes down, the route to the remote subnet may be withdrawn from the routing table. Without a blackhole route, traffic destined for the remote network may follow the default route, potentially causing traffic to leak to the internet or an unintended WAN interface. In a BGP deployment where 'redistribute static' is enabled, a configured blackhole static route will still be considered a valid static route and will be redistributed into BGP unless it is explicitly filtered using a route map. For this article, the following diagram is used: There is an IPsec tunnel between FortiGate_1 and FortiGate_2. On the FortiGate_2 device, a static route is configured for network 172.168.1.0/24 via the IPsec tunnel, black hole route with an AD of 250 is configured as well. FortiGate_2 has a BGP session with FortiGate_3, and all 'static' routes are advertised to that device. FortiGate_2 BGP configuration: config router bgp     set as 65581     set router-id 2.2.2.2         config neighbor             edit "192.168.1.99"                 set soft-reconfiguration enable                 set remote-as 65513                 set password ENC xxxxx             next         end         config redistribute "connected"         end         config redistribute "rip"         end         config redistribute "ospf"         end         config redistribute "static"             set status enable         end end With this configuration, FortiGate_2 will advertise all of its 'static routes'. The routing table of FortiGate_2 when the IPsec tunnel is up and running: get router info routing-table database Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area V - BGP VPNv4 > - selected route, * - FIB route, p - stale info Routing table for VRF=0 C *> 10.10.10.0/24 is directly connected, port1 C *> 10.10.20.0/24 is directly connected, port2 S 172.168.1.0/24 [250/0] is a summary, Null, [1/0] <----- Black hole route. S *> 172.168.1.0/24 [10/0]via test tunnel 192.168.1.83, [1/0] <----- Normal route when IPsec is up. C *> 192.168.1.0/24 is directly connected, port10 C *> 192.168.250.0/30 is directly connected, vdom_link0 C *> 192.168.254.1/32 is directly connected, BGP_Loopback S *> 192.168.255.1/32 [10/0] via 192.168.250.2, vdom_link0, [1/0] When the tunnel is down, the route prefix 172.168.1.0/24 with AD 10 will be removed from the routing table, and only the blackhole route with AD 250, FortiGate_2, will continue to advertise 172.168.1.0/24 with AD 250. In order to prevent the black hole route from being advertised, a configuration change on FortiGate_2 is needed. A route map with a prefix list for every static route is needed. For the IPsec route, a prefix-list with the 'set match-interface IPSec_interface'. This will check if the interface is UP and routes under 'set match-ip-address prefix_list' are learned via that interface.  Adapted BGP configuration: config redistribute "static"     set status enable     set route-map "allow_non_blackhole" end config router route-map     edit "allow_non_blackhole"         config rule             edit 1                 set match-interface "test" <----- IPsec interface.                 set match-ip-address "allow_non_blackhole"                 unset set-ip-prefsrc             next             edit 2                 set match-ip-address "prefix_list_static_route_2"                 unset set-ip-prefsrc             next         end     next end config router prefix-list     edit "allow_non_blackhole"         config rule             edit 1                 set prefix 172.168.1.0 255.255.255.0                 unset ge                 unset le             next             edit 2                 set action deny                 set prefix any                 unset ge                 unset le             next         end     next     edit "prefix_list_static_route_2"         config rule             edit 1                 set prefix 192.168.255.1 255.255.255.255                 unset ge                 unset le             next         end     next end When the IPsec is down, FortiGate_2 does not advertise the black hole route: FortiGate_2 (root) # get router info routing-table details 172.168.1.0 Routing table for VRF=0 Routing entry for 172.168.1.0/24 Known via "static", distance 250, metric 0, best * directly connected, Null, tag 111 Routing entry for 172.168.1.0/24 Known via "static", distance 10, metric 0 via test tunnel 192.168.1.83 vrf 0 inactive, tun_id FortiGate_2 (root) # get router info bgp neighbors 192.168.1.99 advertised-routes VRF 0 BGP table version is 2, local router ID is 2.2.2.2 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *> 192.168.255.1/32 192.168.1.81 32768 0 ? <-/-> Total number of prefixes 1 When the IPsec is up, the prefix 172.168.1.0/24 is advertised to FortiGate_3: FortiGate_2 (root) # get router info routing-table details 172.168.1.0 Routing table for VRF=0 Routing entry for 172.168.1.0/24 Known via "static", distance 250, metric 0 directly connected, Null, tag 111 Routing entry for 172.168.1.0/24 Known via "static", distance 10, metric 0, best * via test tunnel 192.168.1.83 vrf 0, tun_id FortiGate_2 (root) # get router info bgp neighbors 192.168.1.99 advertised-routes VRF 0 BGP table version is 3, local router ID is 2.2.2.2 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight RouteTag Path *> 172.168.1.0/24 192.168.1.81 32768 0 ? <-/-> *> 192.168.255.1/32 192.168.1.81 32768 0 ? <-/-> Total number of prefixes 2