| Description | This article describes the usage of ADVPN stickiness and offers solutions for common problems. The usage of ADVPN has drastically increased over the past few years, especially where multiple overlay tunnels are being used and multiple issues has arisen. |
| Scope |
FortiGate. |
| Solution |
When multiple overlays are used between spokes and a hub, there is a common tendency for shortcuts to be created between different tunnels. This approach has advantages and disadvantages. This article will use the following single hub with 3 ADVPNs and 2 spokes to discuss this.
As shown above, each spoke has 3 overlay links, where one is a MPLS link with private addressing and the other two are internet links from LTE and fiber connections. Upon setting up the ADVPN, the links can be seen as 'up', as shown below in an example hub FortiGate:
get ipsec tunnel list NAME=HUB-INET_0 REMOTE-GW=110.110.110.6:0 NAME=HUB-INET_1 REMOTE-GW=110.110.110.3:0 NAME=HUB-INET REMOTE-GW=0.0.0.0:0 NAME=HUB-LTE_0 REMOTE-GW=120.120.120.6:0 NAME=HUB-MPLS_0 REMOTE-GW=100.100.100.6:0 NAME=HUB-LTE_1 REMOTE-GW=120.120.120.3:0 NAME=HUB-MPLS_1 REMOTE-GW=100.100.100.3:0 NAME=HUB-LTE REMOTE-GW=0.0.0.0:0
When a device behind spoke 1 tries to reach a device behind spoke 2, a shortcut will be created after the initial negotiation, as shown below.
The outputs from spoke 1 and spoke 2 below show that cross shortcuts have been created between MPLS and INET. Refer the following outputs from the Sp1 and Sp2 FortiGates:
In a real network, shortcuts between MPLS links and internet links will fail because MPLS works on a private domain and internet links working on a public domain. This will cause failures in a live network.
Note: As these examples were created in Fortinet labs and there is no way to separate Internet links from MPLS, they have connectivity. This is why shortcuts are shown between MPLS and INET.
To avoid the issues above and allow traffic to pass between the same overlay links (to create shortcuts between the same overlay), apply Policy routes as below. This will allow traffic to pass only from the same links for incoming and outgoing connections on the hub's end.
As a result, on the hub's end:
Relevant configuration on the hub:
config router policy edit 1 set input-device "HUB-MPLS" next edit 2 set input-device "HUB-INET" next set input-device "HUB-LTE" next end
Expected behavior can be observed from the spoke 1 and spoke 2 as below.
Issue: Even though this works in a setup where all spokes have the same number of overlay links, spokes with a different number of overlay links will cause issues as the traffic from INET (in this example) from one spoke will not reach a spoke that does not have an INET link. It is therefore necessary to be cautious when setting up this solution.
As an alternative, apply the policy route only for the MPLS links in the network and allow cross-link traffic and shortcuts between internet links as they have connectivity.
Note: If the tunnel shortcut does not come up after configuration of policy route to achieve stickiness and the following debug logs on the Hub FortiGate.
2025-05-12 10:38:07.814039 ike V=sdwan:3:hub-phase1_1: no match for shortcut-reply 8987943372216158621 5b9c37cac861413a/025699a4d e8eea12 10.254.9.18 to 10.120.36.84 0 psk 64 ppk 0, drop
Note: Starting from v7.2.5 and up, a new feature was introduced in ADVPN setup: the 'auto-discovery-crossover'. This option has been added under the 'config vpn ipsec phase1-interface' configuration to block or allow (default) the set-up of shortcut tunnels between different network IDs. This option can be enabled in the Hub or Spokes sides. For more information, check this KB article: Technical Tip: Usage of ‘auto-discovery-crossover’ option in ADVPN setup |
