ADVPN - Dual WAN connectivity on spokes

[Mattmans1]

Hi all,

This is my first post on these forums, so hello to everybody :)

I'm going to start by asking a question i don't expect many people to be able to answer but i hope somebody who is familiar with BGP and ADVPN can crack this one.  I have labbed up the below scenario and its working great.  Hub/spoke topology with direct spoke to spoke connectivity on demand.

http://cookbook.fortinet.com/configuring-advpn-in-fortios-5-4-dynamic-hub-and-spoke-vpns/

I have got abit more adventurous and added a secondary WAN connection to each firewall and added a second round of ADVPN config/VPN's to establish tunnels over the new WAN connection in a bid to achieve ADVPN redundancy should the primary VPN's fail.

The interesting bit is that it does work (kind of) - If i shut the VPN's down on the hub it works, both spokes will speak to the hub via the second VPN tunnel and agree new spoke to spoke connectivity over the secondary connection.  However it does not work if i shut the VPN tunnel down on the spokes themselves, i seem to get a recursive lookup error. 

This is the BGP table showing routes for spoke to spoke flow with both VPN's up.

OFFICE-FG-ADVPN-IBGP # get router info routing-table bgp B 10.1.1.0/24 [200/0] via 172.16.1.1, ADVPN, 00:00:01 B 10.2.2.0/24 [200/0] via 172.16.1.2, ADVPN_0, 00:00:01

After i shut the primary WAN VPN down on the hub, it fails over to use the secondary

OFFICE-FG-ADVPN-IBGP # get router info routing-table bgp B 10.1.1.0/24 [200/0] via 172.16.2.1, ADVPN-MPLS, 00:01:42 B 10.2.2.0/24 [200/0] via 172.16.2.2, ADVPN-MPLS_0, 00:00:01

And this is the issue when all VPN's are up o nthe hub and i shut down a VPN on one of the spokes.

OFFICE-FG-ADVPN-IBGP # get router info routing-table bgp B 10.1.1.0/24 [200/0] via 172.16.2.1, ADVPN-MPLS, 00:00:17 B 10.2.2.0/24 [200/0] via 172.16.1.2 (recursive is directly connected, unknown), 00:00:16

The issue looks to be because the route for 10.2.2.0/24 (other spoke) is been learned from the hub but with the next hop (172.16.1.2) that is routed over the same VPN tunnel i just shutdown.  So there is no way it can use 172.16.1.2 as a next hop as there is a static route saying route 172.16.1.0/24 over the VPN interface which is down.   (this route is required according to the above article)

"This is an important special step for the spokes as they need a summary route that identifies all tunnel IP used over your topology to point towards the ADVPN interface. In our example, we use 10.10.10.0/24 (if our network planning expects less than 255 sites). Be sure to adequately plan this IP range as it needs to be hardcoded in the spokes."

I have logged this with TAC support but doubt they will help me with this.  Does anyone know who to fix my issue?

[Knox_122]

Would it be possible to just set a monitor on the secondary ADVPN to monitor the primary on the Spoke? 

config vpn ipsec phase1-interface

edit Secondary_ADVPN

set monitor Primary_ADVPN

Could it be that easy for redundancy on the spokes?

[Pdxwolf]

Has anyone found a fix for this yet on FortiOS 6.0.4? I have tried the static routes, blackhole enable, and setting the secondary ADVPN interface to monitor the primary - nothing seems to work when failing WAN1 on the spoke side.

[MBR]

Hi Pdxwolf,

Well we didn't. Even with support and other experts we were not able to build a proper working solution.

We discarded the ADVPN concept and are using plain redundant VPN tunnels now which work like a charm.

[AbbyM]

I had the same issue and managed to resolve it.

in short, this was as a result of the remote-gateway IP subnet. on one of my Spoke I configured 10.10.10.1/32 as my remote-gateway instead of 10.10.10.1/24. Check on the below route line, instead of it specifying the next hope to that network it is saying unknown and then referencing it to my default-gateway (Public !B       10.12.2.0/24 [200/0] via 10.254.0.12, unknown (recursive via 196.181.154.225), 00:39:11

and one thing you need to make sure is that when you check your routing table you should be able to see your hub and spoke network as connect route pointing to your ADVPN main interface.

!C       10.254.0.14/32 is directly connected, ADVPN_P1 you need to ensure that your subnet accommodate all the IP range of your other spokes including the Hub. which makes sense if you restudy the concept of recursive route. This means that the router is receiving routes but it does not know how to get to the host/network. I hope the post will bring some clarity and fixes. Thanks.