SD-WAN Internet Traffic over IPSec VPN

[gilmourspace]

I found a heap of posts discussing how to configure, route, and allow Branch internet traffic over an IPSec VPN to HQ, to utilise the HQ internet connection. I cannot, however, find any posts that discuss my specific situation and issue.

HQ Site - FortiGate 100F
Branch Site - FortiGate 100F

In my scenario, the Branch site and HQ site both have their own independent internet connections, as well as a direct site-to-site fibre link. The HQ site has an FTTP connection that I'd like to have both sites using, but should either the FTTP connection or the site-to-site fibre link go down, I'd want the Branch internet traffic to then go out its own internet connections.

I configured a /30 subnet for each end of the site-to-site fibre link (10.10.10.1 <> 10.10.10.2), created an IPSec VPN to secure the traffic, configured the required routes, firewall policies, etc, and got the LAN traffic routing properly.

Because I want the Branch internet traffic to use the HQ FTTP connection primarily but also have the ability to use the Branch internet connections in the event of either a HQ FTTP failure or a site-to-site fibre link failure, I figured I would need to add the site-to-site fibre link to the SD-WAN I created on the Branch FG that handles the internet connections... however... I already have the site-to-site fibre link IPSec VPN interface in another SD-WAN that manages the site-to-site LAN traffic. This SD-WAN consists of multiple site-to-site IPSec VPNs between the two sites including the site-to-site fibre link IPSec VPN to allow for better traffic management, load balancing, scalability, flexibility, and redundancy.

This being the case, I added the site-to-site fibre link interface (not the IPSec VPN interface) to my Branch SD-WAN managing the internet traffic. Ideally, we'd prefer the internet traffic to also traverse the site-to-site fibre link IPSec VPN, but not essential, and can't see any way of achieving that currently, so not a big deal.

Here's my issue - I cannot get the internet traffic at the Branch site to tranverse the site-to-site fibre link and go out the HQ FTTP connection.

At the Branch here's what I've configured:

• SD-WAN Member defined using the site-to-site Fibre link interface (10.10.10.2/30) with the gateway configured as 10.10.10.1 (tried 0.0.0.0, didn't work either)
• Added site-to-site Fibre link interface SD-WAN Member to the SD-WAN that manages the Branch internet traffic
• Default Static Route directing traffic to the SD-WAN that manages the Branch internet traffic
• Firewall policy allowing traffic from a test device at Branch to the Branch SD-WAN that manages the internet traffic
• Central SNAT policy defining all traffic from a test device at Branch to the site-to-site fibre link interface to have NAT disabled

At HQ:

• Site-to-site Fibre link interface (10.10.10.1/30)
• Default Static Route directing traffic to the SD-WAN that manages the HQ internet traffic
• Firewall policy allowing traffic from a test device at Brance to the HQ SD-WAN that manages internet traffic
• Central SNAT policy defining all traffic from the Branch test device to the HQ FTTP connection to have NAT enabled

The traffic appears to be going out the Branch firewall policy, but the HQ firewall policy does not show the same traffic. I've checked the implicit Deny policy at the HQ end, but it doesn't show the traffic either.

Any insights would be greatly appreciated :)

[knagaraju]

Hello gilmourspace,

Thank you for your detailed explanation. 

I believe flow debug outputs for the above case will give us some idea of what is happening exactly at the backend. 

Hence please collect the below command outputs from both devices

diagnose debug reset
diagnose debug flow filter clear
diagnose debug flow filter addr x.x.x.x
diagnose debug flow filter proto 1
diagnose debug flow trace start 1000
diagnose debug enable
NOTE: x.x.x.x is the destination IP. 
Please make sure you initiate ping traffic only as the filter is applied for ICMP only in the above debugs.

Regards
Nagaraju.

[gilmourspace]

Hi knagaraju,

Thanks for the reply. After running those diags and checking the routing table, it seems the site-to-site fibre link interface has been excluded in the routing table for some reason... The other members of the internet SD-WAN are there, but the site-to-site fibre link is not. Will continue to investigate today. Thanks for your assistance, I'll update this thread once the issue has been resolved :)