| Description | This article describes the usage of ADVPN stickiness and offers solutions for common problems. The usage of ADVPN has drastically increased over the past few years, especially where multiple overlay tunnels are being used and multiple issues has arisen. |
| Scope |
FortiGate. |
| Solution |
When multiple overlays are used between spokes and a hub, there is a common tendency for shortcuts to be created between different tunnels. This approach has advantages and disadvantages. This article will use the following single hub with 3 ADVPNs and 2 spokes to discuss this.
As shown above, each spoke has 3 overlay links, where one is a MPLS link with private addressing and the other two are internet links from LTE and fiber connections. Upon setting up the ADVPN, the links can be seen as 'up', as shown below in an example hub FortiGate:
get ipsec tunnel list NAME=HUB-INET_0 REMOTE-GW=110.110.110.6:0 NAME=HUB-INET_1 REMOTE-GW=110.110.110.3:0 NAME=HUB-INET REMOTE-GW=0.0.0.0:0 NAME=HUB-LTE_0 REMOTE-GW=120.120.120.6:0 NAME=HUB-MPLS_0 REMOTE-GW=100.100.100.6:0 NAME=HUB-LTE_1 REMOTE-GW=120.120.120.3:0 NAME=HUB-MPLS_1 REMOTE-GW=100.100.100.3:0 NAME=HUB-LTE REMOTE-GW=0.0.0.0:0
When a device behind spoke 1 tries to reach a device behind spoke 2, a shortcut will be created after the initial negotiation, as shown below.
The outputs from spoke 1 and spoke 2 below show that cross shortcuts have been created between MPLS and INET. Refer the following outputs from the Sp1 and Sp2 FortiGates:
In a real network, shortcuts between MPLS links and internet links will fail because MPLS works on a private domain and internet links working on a public domain. This will cause failures in a live network.
Note: As these examples were created in Fortinet labs and there is no way to separate Internet links from MPLS, they have connectivity. This is why shortcuts are shown between MPLS and INET.
To avoid the issues above and allow traffic to pass between the same overlay links (to create shortcuts between the same overlay), apply Policy routes as below. This will allow traffic to pass only from the same links for incoming and outgoing connections on the hub's end.
As a result, on the hub's end:
Relevant configuration on the hub:
config router policy edit 1 set input-device "HUB-MPLS" next edit 2 set input-device "HUB-INET" next set input-device "HUB-LTE" next end
Expected behavior can be observed from the spoke 1 and spoke 2 as below.
Issue: Even though this works in a setup where all spokes have same number of overlay links, spokes a different number of overlay links will cause issues as the traffic from INET (in this example) from one spoke will not reach a spoke which does not have a INET link. It is therefore necessary to be cautious when setting up this solution.
As an alternative, apply the policy route only for the MPLS links in the network and allow cross-link traffic and shortcuts between internet links as they have connectivity. |
