Hi support forum,
I am already for quite a time trying to influence SDWAN's path selection on my GNS3 simulation environment, without success.
I run FortiOS 7.0.9, for obvious reasons.
The topology is very easy : two FG's connected to each other via 'WAN1', 'WAN2' and 'back to back'. WAN1 and WAN2 are routed networks (a couple of CIsco's). Test flow is between loopback interface of both FG's. The healthchecks use also these loopback IP's as probe addresses.
I have embraced the three phy interfaces (WAN1, 2 and back to back) by one SDWAN software interface. In my simplest configuration, I don't add any cost and any SLA. A ping health check monitors their availabilty. No service is defined at this stage.
The three members just appear with similar attributes under SDWAN. All 3 are alive, as long as the health-check states them available.
FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0
The actual path followed by the test flow is via member (3, port2). I suppose this is due to the fact that it appears as first in the physical routing table, automatically generated by the SDWAN gateway configuration.
FW-Gos # get router info routing-table stat
Routing table for VRF=0
S 10.10.10.2/32 [1/0] via 8.8.3.2, port2, [1/0] <<<< first one in routing table
[1/0] via 9.9.3.2, VLAN_999, [1/0]
[1/0] via 9.9.9.2, port6, [1/0]
Now, i want SDWAN to make an 'intelligent' decision, and overrule the above route selection. Therefore, I assigned a differential cost (5-10-15) to the three members, favoring VLAN_999. E.g. for VLAN_999:
config members
edit 2
set interface "VLAN_999"
set gateway 9.9.3.2
set cost 5
next
Verifying the member status, they are again all on-par, and there is no appearance of cost, nor of any other differentiating attribute that could influence interface selection by SDWAN.
FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0
... and the selected path stays member 3 = port 2.
Thereafter I tried by defining a service that takes in all traffic (dest = src='all'), aimed to select the link by 'quality' (= set mode priority), and factoring in the latency (= set link-cost-factor latency -- it does not appear in the below 'show', i suppose because latency is the default cost factor).
FW-Gos (service) # show
config service
edit 1
set name "anyService"
set mode priority
set dst "all"
set src "all"
set health-check "PingSLA"FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0
set priority-members 2 3 1
next
end
This configuration is still not of any influence on the instataneous member link status
FW-Gos # diag sys sdwan memb
Member(2): interface: VLAN_999, flags=0x0 , gateway: 9.9.3.2, priority: 1 1024, weight: 0
Member(1): interface: port6, flags=0x0 , gateway: 9.9.9.2, priority: 1 1024, weight: 0
Member(3): interface: port2, flags=0x0 , gateway: 8.8.3.2, priority: 1 1024, weight: 0
The service reveals the three member links as qualifying to forward the traffic within this service, with the lowest-latency link as first in order, which seems logic.
Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Gen(3), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(priority), link-cost-factor(latency), link-cost-threshold(10), heath-check(PingSLA)
Members(3):
1: Seq_num(1 port6), alive, latency: 3.701, selected
2: Seq_num(2 VLAN_999), alive, latency: 63.098, selected
3: Seq_num(3 port2), alive, latency: 60.418, selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
0.0.0.0-255.255.255.255
However, SDWAN still forwards via port2.
Interesting to note is that failover from the forwarding interface to an alternate interface works as a charm (max. 5 pings lost, this is even faster than what I would expect by the default behavior of 30 packets having to time out over 500 millisec before failing over = 15 sec). (port2 > VLAN_999 > port6). When a failing interface restarts its life, it takes the flow back immediately.
Any help or sug
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Hi Jan,
What is your load balande mode in implicit rule ?
Have you Policy base route ?
Best regards,
Julien,
Loadbalancing is per default (source IP based). Be aware I have only one single flow. I have no PBR. 3 static routes are implicitly generated by SDWAN’s single statement ‘
config router static
edit 1
set dst 10.10.10.2 255.255.255.255
set distance 1
set sdwan-zone "virtual-wan-link"
next
resulting in
FW-Gos # get router info routing-table static
Routing table for VRF=0
S 10.10.10.2/32 [1/0] via 8.8.3.2, port2, [1/0]
[1/0] via 9.9.3.2, VLAN_999, [1/0]
[1/0] via 9.9.9.2, port6, [1/0]
Here is the rest of the interesting configuration sections. For the ‘originating’ node (I.e. the one originating the Ping test flow. 10.10.10.3 >10.10.10.2)
FW-Gos # config sys sdwan
FW-Gos (sdwan) # show
config system sdwan
set status enable
config zone
edit "virtual-wan-link"
next
end
config members
edit 2
set interface "VLAN_999"
set gateway 9.9.3.2
set cost 5
next
edit 1
set interface "port6"
set gateway 9.9.9.2
set cost 15
next
edit 3
set interface "port2"
set gateway 8.8.3.2
set cost 10
next
end
config health-check
edit "Default_DNS"
set system-dns enable
set interval 1000
< ... and all the default gratuitous probes go here, not of any use ... the used probe is here below; >
edit "PingSLA"
set server "10.10.10.2"
set members 0
next
end
config service
edit 1
set name "anyService"
set mode priority
set dst "all"
set src "all"
set health-check "PingSLA"
set priority-members 2 3 1
next
end
end
FW-Gos (sdwan) #
Hi Jan
Can you try with set mode sla in your config service.
Because, the cost is using by the lowest cost as stragegy rules.
The mode priority use the latency.
Best regards,
Julien, I did this already earlier, but with pleasure, I restore this 'mode sla' configuration. Here it is.
Service is now with mode sla
FW-Gos (service) # show
config service
edit 1
set name "anyService"
set mode sla
set dst "all"
set src "all"
config sla
edit "PingSLA"
set id 1
next
end
set priority-members 2 3 1
next
end
It points to a modified health-check, that now contains latency as cost factor.
FW-Gos (PingSLA) # show
config health-check
edit "PingSLA"
set server "10.10.10.2"
set members 0
config sla
edit 1
set link-cost-factor latency
set latency-threshold 15
next
end
next
end
The resulting service shows up like this
FW-Gos # diag sys sdwan servic
Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Gen(4), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(3):
1: Seq_num(1 port6), alive, sla(0x1), gid(0), cfg_order(2), cost(15), selected
2: Seq_num(2 VLAN_999), alive, sla(0x0), gid(0), cfg_order(0), cost(5), selected
3: Seq_num(3 port2), alive, sla(0x0), gid(0), cfg_order(1), cost(10), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
0.0.0.0-255.255.255.255
Sequence makes sense : the one with lowest latency is first. The two others are close to each other in latency (within the bounds of threshold = 15), so there the one with lowest cost has taken precedence. Note that all three are 'selected', but only the low-latency one is flagged sla '0x1'
This is confirmed by the appearance of the links in the health-check. Note that here as well, only the 'low latency' link is flagged with sla map '0x1'
FW-Gos # diag sys sdwan health
Health Check(PingSLA):
Seq(2 VLAN_999): state(alive), packet-loss(0.000%) latency(56.930), jitter(9.193), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x0
Seq(1 port6): state(alive), packet-loss(0.000%) latency(3.161), jitter(0.831), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x1
Seq(3 port2): state(alive), packet-loss(0.000%) latency(60.312), jitter(3.275), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996) sla_map=0x0
We would now expect that the low latency link takes in the test flow. But it is not. Test flow still goes on port2 (the third in rank).
Julien, et al,
I think I found the source of the problem.
When I explicitly ask 'use-sdwan yes' in the ping-options for the test flow, the test flow is now absorbed by the first-in-rank egress interface, i.e. the one with the lowest latency. I am curious how SDWAN would behave for a flow other than ping.
Hi Jan,
haaa ok, indeed if you launch the requests from the fortinet equipment, you have to define with which source you have to do it.
I prefer use a device PC in the LAN behind the Fortinet, for this reason.
You can perhaps use source-interface (LAN) for Telnet. I cannot check this one this day.
For information you have this option for multiple service (DNS, Fortiguard)
That's what I did, Julien, see my results below (which are not consistent for both types of traffic, telnet and ICMP). Anyway, I am now extending my test setup with two 'true' endpoints (cisco routers) upstream the two fortigates (diagram, where you also see the three 'WAN's). I let you know the results.
Jan
I strongly recommend the self training nse 7 sd wan. All the parameters and how to calculate are well detailed.
I'm working on it , and it's informative.
Julien, thank you for this recommendation, but I only find an 'OT' flavor of NSE7, not an SDWAN flavour. NSE7 OT Security 7.2
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1679 | |
1085 | |
752 | |
446 | |
226 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.