SD WAN Routing Problems

[RolandBaumgaertner72]

Hi,

we are changing to a new FG90F cluster and we would like to use SD WAN. We have a fiber internet access we used before for the www traffic and MPLS internet access for all branch and headquarter access. The actual HA used only routing so that all internal MPLS traffic went by A and default route for internet was used by B. We want to use SD WAN also just to have an option in the future when we need more outgoing traffic for this office. 

Now we wanted to use SD WAN also as failover if something happens with B and we tried the implicit rule in SD WAN rules with 99-1 for B.

We had a small window today and we wanted check before we change the cluster but we got problems accessing our MPLS network. We had a static route for all MPLS traffic but when we wanted to connect to our LDAP server we could not establish the connection. Removing internet access B and leaving olny MPLS A connected we just connected fine with the internal LDAP in our data center.

So something with static route and SD Wan seems not to work well. in 2 days we want to change the cluster and therefore I added also Policy Based Rules for all MPLS traffic. I dont know if I should use Policy based or only SD WAN rules but I dont want to have another issue with these important basic connections (to our DNS) when we switch the cluster.

Any suggestions how to be sure or best way to router MPLS traffic via SD WAN? 

Thanks,

[syordanov]

Hello RolandBaumgaertner72 ,

As far as i understand, your current FortiGate 90F has to 'ISP' connections 'A' is your MPLS, 'B' is your secondary connection.
Did you run any sniffer or debug flow during the time window? What is the static route for your LDAP, can you please check the static route/routes and associated prioritities.
In FortiOS you can configure AD (Administrative distance) and Priority for the static routes. Two static routes could have the same AD but different priority, please check the KB bellow :

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Setting-priority-on-static-default-routes-...

Regarding the policy route and SD-WAN, always FortiOS checks the policy route/routes and then if there is no match is moving to SD-WAN rules. Please check the KB bellow :

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Routing-in-FortiGate-route-lookup-process/...

I would suggest to check first the routing table/data base :

# get router info routing-table all
# get router info routing-table database

When traffic is not working(for example you can't connect to LDAP), run a debug flow + session list to see which FW rule/SD-WAN rule steers the traffic :

#### Session list ####

diag sys session filter src XXXXX.XXXXX.XXXX.XXXX <---- source IP
diag sys session filter dst XXXXX.XXXXX.XXXX.XXXX <---- destination IP
diag sys session filter dport XXX <----
diag sys session list

#### Debug flow #####

diagnose debug reset
diagnose debug flow filter saddr XXXXXX <---- source IP
diagnose debug flow filter daddr XXXXXX <---- destination IP
diag debug flow show function-name enable
diag debug flow show iprope enable
diagnose debug console timestamp enable
diagnose debug flow trace start 99999
diagnose debug enable

Best regards,
Fortinet

[RolandBaumgaertner72]

Hi,

No, there was no time for debugging since the window was so small. In Static Routes the default was on top and below was the route over the MPLS Interface. I changed to smaller priority for the MPLS Routing.

Also we configured Policy Based Routing so there should be no problem when we connect to LDAP again. Strange thing was, that my client behind the firewall had ping to the LDAP, so now it is hard to guess what really happened.  SD WAN seems so simple and easygoing but like in 50% of the cases setting it up it gives us problems. 

Thanks!

[syordanov]

Hello RolandBaumgaertner72 ,

If both static default routes are with the same AD, means that they are installed . Here plays a role the 'priority' , lowest priority makes the route more preferable.
Regarding the SD-WAN , it depends if you have active monitoring -> performance SLA and both members (MPLS and ISP ) are added to it .
You can check the forwarding logs (if they are saved ) and filter for the LDAP destination of the time of the problem.
But the debug flow and session list would give us more info.

Best regards,
Fortinet