Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
JanDevos
New Contributor III

Link selection in SDWAN

Hi support forum, 

I am already for quite a time trying to influence SDWAN's path selection on my GNS3 simulation environment, without success.

I run FortiOS 7.0.9, for obvious reasons.

The topology is very easy : two FG's connected to each other via 'WAN1', 'WAN2' and 'back to back'.  WAN1 and WAN2 are routed networks (a couple of CIsco's).  Test flow is between loopback interface of both FG's.  The healthchecks use also these loopback IP's as probe addresses.

I have embraced the three phy interfaces (WAN1, 2 and back to back) by one SDWAN software interface.  In my simplest configuration, I don't add any cost and any SLA.  A ping health check monitors their availabilty.  No service is defined at this stage.

The three members just appear with similar attributes under SDWAN.  All 3 are alive, as long as the health-check states them available.

FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0

The actual path followed by the test flow is via member (3, port2). I suppose this is due to the fact that it appears as first in the physical routing table, automatically generated by the SDWAN gateway configuration.

FW-Gos # get router info routing-table stat
Routing table for VRF=0
S 10.10.10.2/32 [1/0] via 8.8.3.2, port2, [1/0]       <<<< first one in routing table 
[1/0] via 9.9.3.2, VLAN_999, [1/0]
[1/0] via 9.9.9.2, port6, [1/0]

 

Now, i want SDWAN to make an 'intelligent' decision, and overrule the above route selection.  Therefore, I assigned a differential cost (5-10-15) to the three members, favoring VLAN_999.  E.g. for VLAN_999:

config members
edit 2
set interface "VLAN_999"
set gateway 9.9.3.2
set cost 5
next

Verifying the member status, they are again all on-par, and there is no appearance of cost, nor of any other differentiating attribute that could influence interface selection by SDWAN.

FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0

... and the selected path stays member 3 = port 2.

Thereafter I tried by defining a service that takes in all traffic (dest = src='all'), aimed to select the link by 'quality' (= set mode priority), and factoring in the latency (= set link-cost-factor latency -- it does not appear in the below 'show', i suppose because latency is the default cost factor).

FW-Gos (service) # show
config service
edit 1
set name "anyService"
set mode priority
set dst "all"
set src "all"
set health-check "PingSLA"FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0

set priority-members 2 3 1
next
end

 This configuration is still not of any influence on the instataneous member link status

FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0

The service reveals the three member links as qualifying to forward the traffic within this service, with the lowest-latency link as first in order, which  seems logic.

Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Gen(3), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(priority), link-cost-factor(latency), link-cost-threshold(10), heath-check(PingSLA)
Members(3):
1: Seq_num(1 port6), alive, latency: 3.701, selected
2: Seq_num(2 VLAN_999), alive, latency: 63.098, selected
3: Seq_num(3 port2), alive, latency: 60.418, selected
Src address(1):
0.0.0.0-255.255.255.255

Dst address(1):
0.0.0.0-255.255.255.255

However, SDWAN still forwards via  port2.

Interesting to note is that failover from the forwarding interface to an alternate interface works as a charm (max. 5 pings lost, this is even faster than what I would expect by the default behavior of 30 packets having to time out over 500 millisec before failing over = 15 sec).  (port2 > VLAN_999 > port6).  When a failing interface restarts its life, it takes the flow back immediately.

Any help or sug

 

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
16 REPLIES 16
Julien87
Contributor II

Hi Jan,

 

What is your load balande mode in implicit rule ?

 

Have you Policy base route ?

 

Best regards,

 

 

Julien
Julien
JanDevos
New Contributor III

Julien,

 

Loadbalancing is per default (source IP based). Be aware I have only one single flow.  I have no PBR.  3 static routes are implicitly generated by SDWAN’s single statement ‘

config router static

    edit 1

        set dst 10.10.10.2 255.255.255.255

        set distance 1

        set sdwan-zone "virtual-wan-link"

    next

resulting in

FW-Gos # get router info routing-table static

Routing table for VRF=0

S       10.10.10.2/32 [1/0] via 8.8.3.2, port2, [1/0]

                      [1/0] via 9.9.3.2, VLAN_999, [1/0]

                      [1/0] via 9.9.9.2, port6, [1/0]

 

Here is the rest of the interesting configuration sections.  For the ‘originating’ node (I.e. the one originating the Ping test flow.  10.10.10.3 >10.10.10.2)

FW-Gos # config sys sdwan

 

FW-Gos (sdwan) # show

config system sdwan

    set status enable

    config zone

        edit "virtual-wan-link"

        next

    end

    config members

        edit 2

            set interface "VLAN_999"

            set gateway 9.9.3.2

            set cost 5

        next

        edit 1

            set interface "port6"

            set gateway 9.9.9.2

            set cost 15

        next

        edit 3

            set interface "port2"

            set gateway 8.8.3.2

            set cost 10

        next

    end

    config health-check

        edit "Default_DNS"

            set system-dns enable

            set interval 1000

        < ... and all the default gratuitous probes go here, not of any use ...  the used probe is here below; >

        edit "PingSLA"

            set server "10.10.10.2"

            set members 0

        next

    end

    config service

        edit 1

            set name "anyService"

            set mode priority

            set dst "all"

            set src "all"

            set health-check "PingSLA"

            set priority-members 2 3 1

        next

    end

end

 

FW-Gos (sdwan) #

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
Julien87

Hi Jan

 

Can you try with set mode sla in your config service.

Because, the cost  is using by the lowest cost as stragegy rules.

 

The mode priority use the latency.

 

Best regards,

 

 

Julien
Julien
JanDevos
New Contributor III

Julien, I did this already earlier, but with pleasure, I restore this 'mode sla' configuration.  Here it is.  

Service is now with mode sla 

FW-Gos (service) # show
config service
edit 1
set name "anyService"
set mode sla
set dst "all"
set src "all"
config sla
edit "PingSLA"
set id 1
next
end

set priority-members 2 3 1
next
end

It points to a modified health-check, that now contains latency as cost factor.

FW-Gos (PingSLA) # show
config health-check
edit "PingSLA"
set server "10.10.10.2"
set members 0
config sla
edit 1
set link-cost-factor latency
set latency-threshold 15
next

end

next 

end

The resulting service shows up like this

FW-Gos # diag sys sdwan servic

Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Gen(4), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(3):
1: Seq_num(1 port6), alive, sla(0x1), gid(0), cfg_order(2), cost(15), selected
2: Seq_num(2 VLAN_999), alive, sla(0x0), gid(0), cfg_order(0), cost(5), selected
3: Seq_num(3 port2), alive, sla(0x0), gid(0), cfg_order(1), cost(10), selected
Src address(1):
0.0.0.0-255.255.255.255

Dst address(1):
0.0.0.0-255.255.255.255

Sequence makes sense : the one with lowest latency is first.  The two others are close to each other in latency (within the bounds of threshold = 15), so there the one with lowest cost has taken precedence.  Note that all three are 'selected', but only the low-latency one is flagged sla '0x1'
This is confirmed by the appearance of the links in the health-check. Note that here as well, only the 'low latency' link is flagged with sla map '0x1'

FW-Gos # diag sys sdwan health
Health Check(PingSLA):
Seq(2 VLAN_999): state(alive), packet-loss(0.000%) latency(56.930), jitter(9.193), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x0
Seq(1 port6): state(alive), packet-loss(0.000%) latency(3.161), jitter(0.831), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x1
Seq(3 port2): state(alive), packet-loss(0.000%) latency(60.312), jitter(3.275), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x0

We would now expect that the low latency link takes in the test flow.  But it is not.  Test flow still goes on port2 (the third in rank).

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
JanDevos
New Contributor III

Julien, et al, 

I think I found the source of the problem.

When I explicitly ask 'use-sdwan yes' in the ping-options for the test flow, the test flow is now absorbed by the first-in-rank egress interface, i.e. the one with the lowest latency.  I am curious how SDWAN would behave for a flow other than ping.

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
Julien87

Hi Jan,

 

haaa ok, indeed if you launch the requests from the fortinet equipment, you have to define with which source you have to do it.

 

I prefer use a device PC in the LAN behind the Fortinet, for this reason.

You can perhaps use source-interface (LAN) for Telnet. I cannot check this one this day.

 

For information you have this option for multiple service (DNS, Fortiguard)

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Functionality-of-set-interface-select-meth... 

Julien
Julien
JanDevos
New Contributor III

That's what I did, Julien, see my results below (which are not consistent for both types of traffic, telnet and ICMP).  Anyway, I am now extending my test setup with two 'true' endpoints (cisco routers) upstream the two fortigates (diagram, where you also see the three 'WAN's).  I let you know the results.

Picture1.jpg

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
Julien87

Jan

I strongly recommend the self training nse 7 sd wan. All the parameters and how to calculate are well detailed.

 

I'm working on it , and it's informative.

 

Julien
Julien
JanDevos
New Contributor III

Julien, thank you for this recommendation, but I only find an 'OT' flavor of NSE7, not an SDWAN flavour.  NSE7 OT Security 7.2

Jan Devos
independent ICT infrastructure advisor
Jan Devosindependent ICT infrastructure advisor
Labels
Top Kudoed Authors