BGP on Loopback traffic shaping question

[Whiteoaks]

I've setup ADVPN using BGP on loopback design with SDWAN. Traffic flows and failover to another link when when it disconnected on all devices. 

Right now I have 1 hub and 2 spoke. I can steer traffic from spoke 1 out the correct interface - but the ADVPN shortcut is sending traffic into a different interface on spoke 1. 

A practical example - I would like to send all backup traffic out our second ISP - WAN2 on spoke1. This works no issues. The traffic however ends up coming in on WAN1 on spoke2. This would normally be fine - except when it comes to traffic like an off-site backup I would prefer it not to be on our primary link.

Is it possible to steer traffic to take a certain path if using BGP on loopback - I believe this is possible with BGP per overlay but don't want to invest a significant chunk of time on that design if what I'm trying to accomplish is possible with the loopback method.

I am also waiting for a call with Fortinet support but hoping to get some more help.

We are running version 7.2 of FortiOS - I don't see 7.4 version yet for the 70G .

Thank you, 

[Stephen_G]

Hello,

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible.

Thanks,

[Whiteoaks]

I appreciate it. Yesterday it worked with the two shortcuts properly created on both tunnels and it was sending traffic to the correct locations. Upon a link failure it restored and went back to not having the correct shortcut tunnel... 

One-time Spoke1 WAN2 formed a shortcut with Spoke2 WAN1. Other times it has the 10.0.0.0/8 address that I've read is assigned when duplicates exist by design but in this case both tunnels are on different network-overlays.

The fact I did see both tunnels with shortcuts on the correct links does give me some hope that maybe this is just a configuration change required to make it reliable... Otherwise I will have to look into a different topology design. 

[Stephen_G]

Sorry it's taken so long, Whiteoaks. We're still looking for an answer to your query.

Did you hear back from support?

[Stephen_G]

While we look into answers, this resource list may be helpful! https://community.fortinet.com/t5/FortiGate/Technical-Tip-FortiOS-BGP-Resource-List/ta-p/214290

[Whiteoaks · Accepted solution]

Hi Stephen,

I have been waiting on TAC since a few days after posting this but did hear back from them and we have been discussing for a couple days. At this point I believe this issue is resolved. 

If anyone comes across this in summary - we used the config

vpn ipsec phase1-interface 
auto-discovery-crossover block

commands to block shortcuts across different overlays in combination with the overlay stickiness (policy route to keep vpn traffic only on their specifics tunnels).

This led to the side effect of the 'special' traffic working until normal traffic created a shortcut over a different WAN  - now the special traffic didn't work as it had no path. 

At that point we needed to create additional tunnles on the spokes using the 'monitor' so they would only activate upon the primary tunnel going down. This creates a temporary shortcut over the other WAN. Once the link is restored the tunnel goes down bringing the shortcut down with it - allowing the proper shortcut to be created.

We will be redoing this configuration in the office as we build out our new locations that will be going into production in late April - but in my testing this seems to resolve and work as expected. 

I think all of this may be not necessary in ADVPN2.0 with FortiOS version 7.4 but we just purchased a combination of 90G and 70G firewalls and the 7.4 release is not yet available for the 70G. 

If anyone comes across this and needs additional explanation let me know..