First time 60E user, need help with SDWAN, VPN please

[atsak]

Hi there - I'm setting up the first Fortinet I'd like to use for deploying to about 30 or 40 clinics.

The clinics VPN back to a Juniper SSG firewall in the datacenter.

I am trying to setup:

Failover from WAN1 to the LTE modems (this works using SDWAN perfectly)

Fail back when service returns (this does not work if there is demand on the circuit.  If I stop pinging, it will failback as programmed)

VPN tunnel to the datacenter (works but flaps, drops 5 or 10 pings out of 100) on the WAN1 interface with failover to the LTE Modem (failover of the VPN does not work.  Phase 1 completes to the datacenter firewall but Phase 2 never does; doesn't even try).

VPN tunnel automatically connects and stays connected after disconnect / power cycle (This does not work - I usually have to click the "bring up" and it connects about 20 seconds after that).

The offices may or may not have NAT in front of the firewall so NAT-T is enabled.

I've attached the config file from the 60E; I removed the default settings to shorten what folks have to look at and sanitized a couple IP's and passphrases with X or x.x.x.x

I would think this would be a pretty common scenario, so if anyone has a working config please let me know.   I updated it to the latest firmware since I rarely touch these after deployment unless there's a serious security issue with the firmware itself.  Any help greatly appreciated; I've been trying different configurations (primarily with the routing cost and preference, but that didn't really make any difference) for about a week on and off now and just can't get it.   

For the fortinet folks - really would suggest you finish the cookbook on SDWAN VPN.   If I could tie the VPN tunnel to the SDWAN interface this would be 100x easier.

[atsak]

So I assume therefore this doesn't work.

Instead what I did was setup wan1 and wwan, with policies to permit the traffic and tunnel interfaces bound to each of the physical interfaces.    I setup default routes with different preferences / routing distances to prefer the WAN1 connection.

The failover isn't automatic, but if you unplug WAN1 this works, and immediately fails back when you plug the interface back in.

So I guess the SDWAN isn't quite ready for prime time yet (in the current version of 5.6 firmware anyway), which is conceivably why the cookbook hasn't been published yet I would guess.  

If anyone has a better solution or can tell me how to get the interface to disable when it is not working without using SDWAN that would of course be appreciated.

[atsak]

There is one other caveat.  If the USB LTE modem loses connection at any point, the failover will no longer work.   This can be worked around by rebooting the firewall.   Not ideal, but failures are rare, but that should be fixed in a future firmware update IMO.

[oheigl]

I'm really suprised that the failover is not detected at all. Have you tried enabling the following command in the virtual-wan-link configuration?

set fail-detect enable

[oheigl]

Regarding the VPN tunnel not coming up or not failing over correct: Please enabled DPD on idle on both VPNs, otherwise I noticed some weird behavior also on 5.4

There were same issues that the VPN stays up all the time although the tunnel is long gone - DPD on idle fixed the problem for me.

[ede_pfau]

I recommend to install blackhole routes for all private IP address ranges (RFC1918) to help VPN tunnel re-establishment.

Background: in case a VPN tunnel goes down, the route to the remote network is either deleted or it might have a higher distance as the default route. Then, traffic for the remote private subnet is sent out the WAN port.

When the tunnel is renegotiated, there is already a session for this in the session table. Traffic across the VPN tunnel will only start to flow after this session is expired. Having blackhole routes in the routing table prevents sessions from being established via default route in the first place.

Attached is a batch command file with these routes; you can find the same info in the forums ("blackhole").