Troubleshooting Tip: How set-ip-nexthop can prevent BGP route installation into routing table

[fwilliams]

Description	This article describes how 'set-ip-nexthop' may prevent the installation of a BGP route into the routing table.
Scope	FortiOS. FortiManager v7.0.5 or v7.2.2.
Solution	This may occur in a new BGP setup or after an 'existing and working' FortiGate is upgraded. The BGP peering is or will be UP in both cases (whether a new or existing installation), but the route from the other BGP peer/peers will either not be received, or will be received but not make it to the routing table. The BGP Next-Hop Attribute is defined in RFC 4271. It is a well-known mandatory attribute that specifies the IP address of the next router to be used as a next-hop for prefixes (destination IP addresses) listed in a BGP UPDATE message. To ensure the proper functioning of a BGP network, there are some rules or conditions that must be met regarding the 'Next-Hop Attribute' before a route is considered valid and installed in the routing table. This article will not explore all of the conditions but will touch upon relevant conditions to this issue. In v7.0.5 and v7.2.2, there is a known issue 845656 in which FortiManager will attempt to push 'set set-ip-nexthop 0.0.0.0' to any FortiGate route-maps which has no set-ip-nexthop set. If running one of the affected FortiManager versions, it is strongly recommended to reconfigure affected route maps manually on FortiGate and plan to upgrade to a later FortiManager version. The topology used for this demonstration: The flow of events in a firmware upgrade scenario: It is a common practice to use route-map to filter what can be announced or received from BGP peer/s. In this testing scenario, route-map was used in the network shown above to allow only default route from peers (though other routes can be used). After the upgrade, the route-map could be altered, especially when FortiManager was used to perform the upgrade. This article will demonstrate a case where the route-map changes from '(a)' to '(b)' during the upgrade. (a): config router route-map edit "name" config rule edit 1 set match-ip-address "name" next end next end (b): config router route-map     edit "name"         config rule             edit 1                 set match-ip-address "name"                 set set-ip-nexthop 0.0.0.0                 set set-ip6-nexthop ::                 set set-ip6-nexthop-local ::                 set set-originator-id 0.0.0.0             next         end     next end The 'set-ip-nexthop' in the route-map is to set to 0.0.0.0 during the upgrade. This means the next hop for the route received (default route in this case) has been set to 0.0.0.0 (details in 1b). Because of step 2, BGP stopped receiving the route. If the BGP debug is completed (with the commands shown below) to investigate, the debugging shows the route was denied with the message: 'denied due to non-connected next-hop'. This happens because nexthop 0.0.0.0 is not a connected IP. BGP debug: diag ip router bgp all enable diag ip router bgp level info diag debug enable It is possible to set 'allow multi-hop eBGP neighbor' under the neighbor configuration: this means the next-hop will not have to be a connected IP anymore. After this change, the route will be received, but will not be installed in the routing table. config router bgp config neighbor edit "x.x.x.x" set ebgp-enforce-multihop enable next end The following error can be seen from the BGP debug as to why the route is not installed: BGP: 200.200.200.2-Outgoing [RIB] Update: Received Prefix 0.0.0.0/0 path_id 0 BGP: [DAMP] bgp_rfd_rt_update(): Route State: NONE, ret=0 BGP: 200.200.200.2-Outgoing [RIB] Update: ...duplicate route ignored In the routing table database, the route was received and can be seen as INACTIVE: this is because the next hop '0.0.0.0' could NOT be resolved. get router info routing-table database .. .. Routing table for VRF=0 B 0.0.0.0/0 [20/0] via 0.0.0.0 inactive .... Several steps can be performed to make it work. Set the set-ip-nexthop in the route-map to 200.200.200.2/3 (in this example). Unset the 'set-ip-nexthop' in the route-map. It will be best to also unset other parameters in the route-map which were not there before the upgrade (or if it is a new configuration, understand the implication of each 'set' line). Revert to the previous version where the system was working as intended, then upgrade directly from FortiGate without using the FortiManager. Create a static route manually, and have it point to 200.200.200.2/3 (in this example). After either unsetting 'set-ip-nexthop', or setting it to appropriate next hop IP in the route-map/s, the route can be seen installed as shown below. Note: Open a ticket with the FortiManager TAC team after resolving the issue with the above solutions to look into the issue in detail.