FG 81E - SD-wan rule with manual select member does not hit.

[TanLee]

I am using FortiGate 81E, running OS 6.2.3 with the following setup:

- Two WAN ports are connected to the ISP using PPPoE mode. WAN1 has a distance of 5, and WAN2 has a distance of 10. Both are ON with the option "Retrieve default gateway from server" and OFF with "Override internal DNS."

- WAN1 and WAN2 are members of SD-Wan at the exact 0 cost.

- A static route 0.0.0.0/0.0.0.0 -> SD-wan interface has been added.

- There are multiple VLANs configured under Aggregate Link (downstream LAG to my Cisco switch), let's say vlan-10, vlan-20, vlan-30.

- Policies to access the Internet have been added for all Vlans above (source is vlan-x interface, and destination is SD-Wan).

** When there is just a default implicit SDWan rule with its Load-Balancing Algorithm, all nodes from above VLANs can access the Internet but are unstable due to sessions being switched between WAN1 and WAN2 continuously (look at the Forti View -> All Sessions and see the destination interface switching between sessions). To deal with this, I created an SDWan rule to manually select the wan member for each VLAN source, e.g., the source from vlan-10 will manually select WAN2 to forward the traffic.

After adding the SDWan rules for manually selecting members, I think the traffic will be routed as the definition, but it is not. It seems likely that the defined rule has not been hit, and the traffic still deals with the implicit SDWan rule. Monitor the route (with diag firewall proute list command); I see the hit_count=0 as always.

I don't know where the problem is or if there is a workaround to make the connection more stable.

It's weird to me, as I have a similar configuration on FG-100E that is running smoothly in another office.

I would appreciate it if someone could help. Please let me know if you need more information, and I will provide it.

[akileshc]

Hello Tanlee,

By default, SD-WAN rules will select a member only if there’s a valid route to the destination through that member. Since your WAN2 interface has an administrative distance (AD) of 10, and WAN1 with administrative distance (AD) of 5; the default routes associated with WAN2 are not being added to the routing table. Consequently, even if you have an SD-WAN rule that includes WAN2, the traffic won't match this rule, and it will always match the next rule or an SDWAN implicit rule.

To resolve this issue, you need to ensure to have an valid default routes via WAN2. This can be done by adjusting the AD of WAN2 to 5.

https://docs.fortinet.com/document/fortigate/7.2.0/sd-wan-deployment-for-mssps/511005/sd-wan-routing... 

[TanLee]

Thanks for your reply Akilesh, 

It's weird to me when the routing table still having both static route for 2 WAN PPPoE as below: 

Connected

AEON-81E # get router info routing-table all
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default

S* 0.0.0.0/0 [1/0] via 203.210.144.240, ppp2
[1/0] via 27.71.251.147, ppp1
C 10.2.1.0/24 is directly connected, vlan-10
C 10.2.2.0/24 is directly connected, vlan-20
C 10.2.3.0/24 is directly connected, vlan-30
C 10.2.7.0/24 is directly connected, LAG
C 10.2.8.0/24 is directly connected, vlan-80
C 10.2.9.0/24 is directly connected, vlan-90
C 10.2.18.0/24 is directly connected, vlan-180
C 10.2.70.0/24 is directly connected, vlan-70
C 10.2.71.0/24 is directly connected, vlan-71
C 10.2.72.0/24 is directly connected, vlan-72
C 10.2.99.0/24 is directly connected, vlan-99
C 27.71.251.147/32 is directly connected, ppp1