AD-VPN, BGP, SDWAN and fib best match

[zoriax]

Hello,

I used AD-VPN to Spoke to spoke communication (FortiOS 7.0.7), it works perfectly excepted when I used SD-WAN.

I tried to reach 10.5.5.0/24 via ADVPN and here my bgp routing table, as you can see, I have 2 path to reach this subnet but onw with is "directly connected" -> This way tunnel is the best

10.5.5.0/24  [200/0] via 172.0.0.1 (recursive via ADVPN0 tunnel 1.2.3.4), 00:18:39
                      [200/0] via 172.0.1.0 (recursive is directly connected, ADVPN1_0), 00:18:39

Now I added SD-WAN with sla with "lowest cost SLAN" and fib best match : 

config service
   edit 1
      set name "ADVPN"
      [...]
      set tie-break fib-best-match
   next
end

With a diagnose sys sdwan service I have this result

Members(3):
1: Seq_num(2 ADVPN0), alive, sla(0x1), gid(0), cfg_order(0), cost(0), selected
2: Seq_num(3 ADVPN1_0), alive, sla(0x1), gid(0), cfg_order(1), cost(0), selected
3: Seq_num(3 ADVPN1), alive, sla(0x1), gid(0), cfg_order(1), cost(0), selected

It drives me crazy because I don't understand why my entry n° 2 is not at the first position... Someone could help me to solve this ?

Many many thanks for your help !

[zoriax · Accepted solution]

You're right, no correlation >:-)

But in my case, I have 2 hubs and 2 ways to initiate spoke-to-spoke tunnel with ADVPN.

Without network-overlay, only one tunnel can be set. When for example hub1 goes down, shortcut starts on hub2. When hub1 is available again, SD-WAN "move" traffic to it and shortcut established on hub2 is not used (here, impossible to create a new shortcut on hub1). So all the traffic goes trough hub1 and it's not the expected behavior. 

It's why I'm looking for a way to "force" SD-WAN to use hub2 even if hub1 is up again.

[pminarik]

Here's a relevant doc for fib-best-match - https://community.fortinet.com/t5/FortiGate/Technical-Tip-FortiOS-SD-WAN-SLA-Tie-Break-Feature-Overv...

My understanding is that fib-best-match is utilized for cases where you want to pick the best route out of e.g. a /8, /16, and /24 (/24 is the best match -> pick that).

In your case, if you're choosing between two /24s (ECMP situation), fib-best-match won't have an effect, and the cfg-order will be the deciding criterion.

[zoriax]

Hello,

Thanks for your feedback. I understand this behavior. 

In my case, I configured in vpn ipsec phase1-interface 

set network-overlay enable
set network-id 12345

Which allow multiple shortcut VPN on one WAN interface.

[akristof]

Hello,

I am not sure what is the correlation between network-id and sdwan rule member order.

[zoriax · Accepted solution]

You're right, no correlation >:-)

But in my case, I have 2 hubs and 2 ways to initiate spoke-to-spoke tunnel with ADVPN.

Without network-overlay, only one tunnel can be set. When for example hub1 goes down, shortcut starts on hub2. When hub1 is available again, SD-WAN "move" traffic to it and shortcut established on hub2 is not used (here, impossible to create a new shortcut on hub1). So all the traffic goes trough hub1 and it's not the expected behavior. 

It's why I'm looking for a way to "force" SD-WAN to use hub2 even if hub1 is up again.

[akristof · Accepted solution]

Hi,

That's ok. So you have 2 overlays (it doesn't really matter that 1 is on HUB1, second on HUB2).

You should have your SDWAN rules configured in a way, that in single ADVPN region you consider one overlay as preferred. So for example if ADVPN0 is your preferred one, you should try to this overlay as preferred in rules on every device. But even this config as you, it can work. You will just have 2 shortcuts (most likely). Traffic coming from ADVPN1_0 shortcut should use this interface as a reply. But if you will have traffic initiated from this device (or lan behind), it will select ADVPN0 and potentially form second shortcut.

[zoriax]

Thanks akristof. This issue is solved for me

[TT_DU]

Hi

I have a similar kind of setup and have some queries, can you check the post in this link and advise it?

https://community.fortinet.com/t5/Support-Forum/DUAL-HUB-SETUP-FOR-ADVPN-and-SDWAN-FOR-BRANCH-OFFICE... 

[TT_DU]

@zoriax please check and let me know if you have any thoughts.