Description
This article describes the Integration of IPsec VPN with SD-WAN to manage IPsec traffic flow and Redundancy using the SDWAN rule.
Scope
FortiGate version 6.4 and above.
FortiGate version 7.0 and above.
Solution
To Manage the IPsec VPN with SD-WAN rather than using the route Priority.
Consider the Following Scenario.
Assuming the SD-WAN is already configured for the ISP, this is not a requirement for this setup but it's always good to have the ISP with SD-WAN.
Version 7.2.0 has been used to configure this setup.
1) Configure the VPN Interface but not from IPsec Wizard as the interface created from IPsec wizard cannot be called in the SD-WAN member or to be precise when the tunnel is created from IPsec wizard it creates routes, policy, addresses, etc. and to configure FortiGate interfaces as SD-WAN members, it necessary to remove or redirect existing configuration references.
So to avoid error-prone configuration a new wizard was added for SD-WAN VPN.
Go to Network -> SD-WAN, select 'Create New' -> SDWAN Zone, the name VPN has been used, do not add any members as of now.
Now create SD-WAN Member:
Go to Network -> SD-WAN, select 'Create New' -> SDWAN Member.
In the Interface drop-down, select +VPN. The Create IPsec VPN for SD-WAN members pane opens.
Enter the required information, then select 'Create'.
Name - Respected Tunnel Name (VPN_1).
Remote Device Ip address/ DDNS - The IP address has been used.
Outgoing Interface - The WAN 1 (For the setup it's port 3).
Authentication Mode - Pre-Shared Key/Signature (the pre-shared Key has been used).
Pre-Shared Key - (Define preshared key).
Review the setting and Create the Interface.
2) Now add the Interface in the respected zone if any zone has not been created for VPN create a new zone and add it.
Follow the same process and create the 2nd VPN tunnel and add it in the same zone.
3) Change the VPN traffic selector as per requirement, as with the SD-WAN Wizard it will create any and any.
Configure the Address:
Go to Policy & Object -> Addresses, select 'Create new' - Address (Configure the local and remote address as per requirement) In this setup local - 10.24.0.0/20 and Remote 10.25.0.0/20.
Now go to VPN -> Ipsec Tunnel -> Respected Tunnel and change the phase 2 selector.
Note.
Verify the Tunnel configuration by going to the VPN -> Ipsec Tunnel - > VPN_1 & VPN_2.
4) Go to the respected VPN Interface and assign an IP address to the Interface, any gateway has been defined when configuring the SD-WAN member as even if any gateway has been configured there it will again populate it with 0.0.0.0.
Go to Network -> Interface - > Expand the WAN 1 and edit the VPN_1 interface.
Define the local and remote interface IP, 1.1.1.1 and 1.1.1.2 have been used for VPN_1 & for VPN_2 -> 2.2.2.1 and 2.2.2.2
5) Create the Static Route for the VPN traffic using the VPN SD-WAN zone created if FortiOS is running v7.0 and above.
Note:
On FortiOS v6.4.x, Static routes can be created for individual VPN interfaces or for the entire SD-WAN interface but not for individual VPN SDWAN zones. Creating static routes for individual VPN SDWAN zones is supported only from FortiOS v7.0
Go to Network -> Static Route, select 'Create New' (Enter the required information) and select 'Ok'. Following snippet is from FortiOS v7.0.0
6) Create the Firewall policy for the respected VPN traffic using the VPN SDWAN Zone:
Go to Policy & Object - > Firewall Policy, select 'Create New', select the respected interface for LAN to VPN communication and add the required attribute.
Note.
Make sure Nat is disabled in the policy otherwise the traffic will use the tunnel Interface IP and if peer end Internal LAN has no route for tunnel Interface IP the traffic will be dropped at peer end Internal Lan.
Once the policy is created, it is possible to do a clone reverse of the policy, change the name and enable the policy.
7) Configure Performance SLA for VPN Interface in SD-WAN:
Go to SDWAN - > Performance SLA, select 'Create New', define the parameter (peer-end Local Lan machine IP has been used, as the server 10.25.12.3 ).
But after the configuration, the Performance SLA will still be down.
8) To get this SLA working it is necessary to add the source in the SD-WAN VPN member from CLI
# config system sdwan
# config members
edit 4
set interface "VPN_2"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
edit 5
set interface "VPN_1"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
end
Note 1.
Source IP can only be the interface IP, and it will work when the source IP is included in the VPN phase 2 traffic selector, In this configuration, local - 10.24.0.0/20 has been used and Remote 10.25.0.0/20 so 10.24.3.109 is in the respected range, that is why it is possible to use it.
Note 2.
If the peer end configuration is not done the SLA will not come up even after adding the source as the configuration is not complete and SLA relies on the tunnel configuration to reach the peer side server which is configured.
Create the SD-WAN Rule using the same performance SLA:
Go to Network - > SD-WAN Rules, select 'Create New', define the Parameter (the strategy as Lowest cost SLA has been used, it is possible to refer to the related articles and check which is suitable for the network, it is possible to configure either redundant or else load-balanced).
Latency, Best Quality, and Lowest cost SLA will use one interface at a time but with Maximum bandwidth the Traffic is load-balanced among interfaces.
Make sure the VPN rule is above the all to all rule as the rule works top to bottom.
9) It is necessary to create a blackhole route for the destination subnets as if it is not, and assume the scenario both tunnels are down the traffic will be routed to the ISP link if there's a policy match and even the tunnel comes back up the traffic will still use the existing session rather than using the tunnel until and unless the session is timed out or clear it manually.
Use CLI to configure the blackhole route, entry 4 was not there so it has been used, make sure the AD value which is used in blackhole route is higher than the static which has been configured:
# config router static
edit 4
set dst 10.25.0.0 255.255.240.0
set distance 15
set blackhole enable
end
To check whether the route is working or not bring down both the tunnel try initiating some traffic from the LAN, use the below command to check the route.
It is possible to see it shows the root as the next hop, also in debug flow filter, it is possible to see it routing it towards the root.
Firewall_Dual_ISP # diagnose ip rtcache list | grep 10.25
10.24.5.179@3(port1)->10.25.12.10@13(root) gwy=0.0.0.0 prefsrc=10.24.3.109
Flow filter:
id=65308 trace_id=149 func=init_ip_session_common line=6076 msg="allocate a new session-00f532ea, tun_id=0.0.0.0"
id=65308 trace_id=149 func=iprope_dnat_check line=5331 msg="in-[port1], out-[]"
id=65308 trace_id=149 func=iprope_dnat_tree_check line=823 msg="len=0"
id=65308 trace_id=149 func=iprope_dnat_check line=5343 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=65308 trace_id=149 func=vf_ip_route_input_common line=2605 msg="find a route: flag=00000000 gw-10.25.12.10 via root"
Firewall_Dual_ISP # diagnose ip address list
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=13 devname=root
10) Configure the Peed end Device, if it is not already done (the normal Route Priority has been used at the peer side for failover, it is also possible to configure SD-WAN at this side using the above example).
- Configure the two different tunnels for both peer IPs, using the Wan interface if user is not configuring SD-WAN then use the IPsec wizard to configure it.
- If the wizard has been used, it will create the address, policy, static route, blackhole route.
- It is necessary to change the VPN Interface IP.
- Once everything is created, it is possible to change the Priority of the route as per requirement, it is also possible toleave it as it is, to manage the tunnel from the Firewall which is having the dual-link only.
To check the SDWAN:
Firewall_Dual_ISP # diagnose sys sdwan member
Member(1): interface: port3, flags=0x0 , gateway: 10.27.11.206, priority: 1 1024, weight: 0
Member(2): interface: port4, flags=0x0 , gateway: 10.26.11.206, priority: 1 1024, weight: 0
Member(4): interface: VPN_2, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Member(5): interface: VPN_1, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Firewall_Dual_ISP # diagnose sys sdwan service
Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(2):
1: Seq_num(5 VPN_1), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(4 VPN_2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
Src address(1):
10.24.0.0-10.24.15.255
Dst address(1):
10.25.0.0-10.25.15.255
Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(load-balance hash-mode=round-robin)
Members(2):
1: Seq_num(1 port3), alive, sla(0x0), gid(0), num of pass(0), selected
2: Seq_num(2 port4), alive, sla(0x0), gid(0), num of pass(0), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
0.0.0.0-255.255.255.255
Firewall_Dual_ISP # diagnose sys sdwan health-check
Health Check(ping):
Seq(5 VPN_1): state(alive), packet-loss(0.000%) latency(1.476), jitter(0.197), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Seq(4 VPN_2): state(alive), packet-loss(0.000%) latency(1.340), jitter(0.244), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Firewall_Dual_ISP # diagno firewall proute list
list route policy info(vf=root):
id=2140340226(0x7f930002) vwl_service=2(VPN_SDWAN) vwl_mbr_seq=5 4 dscp_tag=0xfc 0xfc flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=28(VPN_1) oif=25(VPN_2)
source(1): 10.24.0.0-10.24.15.255
destination(1): 10.25.0.0-10.25.15.255
hit_count=0 last_used=2022-04-19 21:56:27
id=2140340225(0x7f930001) vwl_service=1(Internet) vwl_mbr_seq=1 2 dscp_tag=0xfc 0xfc flags=0x10 load-balance hash-mode=round-robin tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=5(port3) num_pass=-1 oif=6(port4) num_pass=-1
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255
hit_count=117383 last_used=2022-04-19 22:20:30
To check the VPN:
https://community.fortinet.com/t5/FortiGate/Technical-Tip-Troubleshooting-IPsec-VPN-tunnel-errors-wi...
https://community.fortinet.com/t5/FortiGate/Technical-Tip-Troubleshooting-IPsec-VPNs/ta-p/195955
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-IPsec-Site-to-Site-Tunnel-Connectivi...
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-IPSEC-Tunnel-debugging-IKE/ta-p/1900...
Related KB articles:
https://community.fortinet.com/t5/FortiGate/Technical-Tip-Configure-FortiGate-SD-WAN-with-an-IPSEC-V...
https://community.fortinet.com/t5/FortiGate/Technical-Note-How-to-use-BGP-and-SD-WAN-for-advertising...
https://community.fortinet.com/t5/FortiGate/Technical-Tip-SD-WAN-performance-SLA-for-IPsec-interface...
https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-use-IPsec-tunnel-interface-on-Perfo...
https://docs.fortinet.com/document/fortigate/7.2.0/administration-guide/942095/sd-wan-zones
https://docs.fortinet.com/document/fortigate/6.2.0/new-features/403128/dual-vpn-tunnel-wizard
https://docs.fortinet.com/document/fortigate/7.2.0/administration-guide/716691/sd-wan-rules
https://docs.fortinet.com/document/fortigate/7.2.0/administration-guide/19246/sd-wan
