Troubleshooting Tip: Common issue with a Hub and Spokes dual IPsec VPN with BGP

[gmonte]

Description	This article describes a common configuration mistake on a Hub and Spokes topology with dual IPsec VPN and BGP.
Scope	FortiGate.
Solution	This article assumes the following topology is in place: The following is also assumed: IPsec - green link. IPsec2 - purple link. IPsec Hub interface: 10.100.1.1/28 IPsec Spoke1 interface: 10.100.1.2/28 IPsec Spoke2 interface: 10.100.1.3/28 IPsec2 Hub interface: 10.100.2.1/28 IPsec2 Spoke1 interface: 10.100.2.2/28 IPsec2 Spoke2 interface: 10.100.2.3/28 LAN behind Spoke1: 10.1.2.0/24 The HUB assigns the IP addresses to the IPsec interfaces on the Spokes dynamically from an IP range. The following may be observed on the routing table upon running the command get router info routing-table all: B 10.1.2.0/24 [200/0] via 10.100.1.2 (recursive is directly connected, IPSEC), 5d10h01m, [1/0]                  [200/0] via 10.100.2.2 (recursive via wan1), 5d10h01 Note: Normally, the wan1 should not be shown as the exit interface, as the FortiGate should not send internal traffic (with a private IP address as destination) on the WAN interfaces. The wan1 could be present in cases where one IPsec2 goes down. That happens because when Spoke1 advertises its local routes to the Hub, the Hub will advertise them to the Spoke2 and the next hop is the interface's IPs on Spoke1 (10.100.1.2/28 and 10.100.2.2/28). Since they are on the same subnet, it is showed as directly connected for the IPsec interface. The IPsec2 tunnel is now down and so, to reach the the next-hop 10.100.2.2, the FortiGate recursively checks the routing table and matches the default route 0.0.0.0/0 with the exit interface wan1. In order to avoid this behavior, is mandatory to set 'set next-hop-self-rr enable' on the HUB BGP configuration, since the HUB will be the route reflector. This way, the HUB will change the next-hop to itself before re-advertising the routes. In this case, if the IPsec2 is down, the next hop will be only 10.100.1.1 (HUB IP) and so will be reachable through the IPsec tunnel. If neighbor groups are configured, run the following commands to correctly set up this command: config neighbor-group     edit IPSEC1         set interface "VPN_IPSEC1"         set next-hop-self-rr enable     next     edit IPSEC2         set interface "VPN_IPSEC2"         set next-hop-self-rr enable     next end With this change applied, the routing table should be similar to the following when all of the IPsec tunnels are UP: B          10.1.2.0/24 [200/0] via 10.100.1.1 (recursive is directly connected, IPSEC), 5d3h1m, [1/0]                               [200/0] via 10.100.1.1 (recursive is directly connected, IPSEC2), 5d3h1m, [1/0]                               [200/0] via 10.100.2.1 (recursive is directly connected, IPSEC), 5d3h1m, [1/0]                               [200/0] via 10.100.2.1 (recursive is directly connected, IPSEC2), 5d3h1m, [1/0] Note: Only relevant parts of the outputs and configurations are showed in this article.