Description
This article describes the Integration of IPsec VPN with SD-WAN to manage IPsec traffic flow and Redundancy using the SD-WAN rule.
Scope
FortiGate version 6.4 and above.
FortiGate version 7.0 and above.
Solution
To Manage the IPsec VPN with SD-WAN rather than using the route Priority.
Consider the Following Scenario.
Assume the SD-WAN is already configured for the ISP. This is not a requirement for this setup but it is always good to have the ISP with SD-WAN.
Version 7.2.0 has been used to configure this setup.
- Configure the VPN Interface, but not from IPsec Wizard, as the interface created from IPsec wizard cannot be called in the SD-WAN member. When the tunnel is created from IPsec wizard, it creates routes, policy, addresses, etc., and to configure FortiGate interfaces as SD-WAN members, it is necessary to remove or redirect existing configuration references.
A new wizard was added for SD-WAN VPN to avoid error-prone configuration.
Navigate to Network -> SD-WAN, and select 'Create New' -> SDWAN Zone, the name VPN has been used, do not add any members as of now.
Create an SD-WAN Member:
Navigate to Network -> SD-WAN, select 'Create New' -> SDWAN Member.
Select +VPN in the Interface drop-down to open the Create IPsec VPN for SD-WAN members pane.
Enter the required information, then select 'Create'.
Name - Respected Tunnel Name (VPN_1).
Remote Device Ip address/ DDNS - The IP address has been used.
Outgoing Interface - The WAN 1 (For the setup it's port 3).
Authentication Mode - Pre-Shared Key/Signature (the pre-shared Key has been used).
Pre-Shared Key - (Define preshared key).
Review the settings and Create the Interface.
-
Add the Interface in the respected zone. If a zone has not been created for VPN, create a new zone and add it.
Follow the same process create the second VPN tunnel and add it in the same zone.
-
Change the VPN traffic selector as per requirement, as with the SD-WAN Wizard it will create any and any.
Configure the Address:
Go to Policy & Object -> Addresses, select 'Create new' - Address (Configure the local and remote address as per requirement) In this setup local - 10.24.0.0/20 and Remote 10.25.0.0/20.
If it is not desired to use this IPsec connection to go to Internet, go to VPN -> Ipsec Tunnel -> Respected Tunnel and change the phase 2 selector.
Note: Verify the Tunnel configuration by going to the VPN -> Ipsec Tunnel - > VPN_1 & VPN_2.
To make changes to algorithm/encryption in phase-1/ phase-2 or ike version, select 'Convert To Custom Tunnel' under the Tunnel Template as shown in the following figure. -
Go to the respected VPN Interface and assign an IP address to the Interface, any gateway has been defined when configuring the SD-WAN member as even if any gateway has been configured there it will again populate it with 0.0.0.0.
Go to Network -> Interface - > Expand the WAN 1 and edit the VPN_1 interface.
Define the local and remote interface IP, 1.1.1.1 and 1.1.1.2 have been used for VPN_1 & VPN_2 -> 2.2.2.1 and 2.2.2.2.
-
Create the Static Route for the VPN traffic using the VPN SD-WAN zone created if FortiOS is running v7.0 and above.
Note:
On FortiOS v6.4.x, Static routes can be created for individual VPN interfaces or the entire SD-WAN interface but not for individual VPN SD-WAN zones. Creating static routes for individual VPN SD-WAN zones is supported only by FortiOS v7.0.
Go to Network -> Static Route, select 'Create New' (Enter the required information) and select 'Ok'. The following snippet is from FortiOS v7.0.0.
-
Create the Firewall policy for the respected VPN traffic using the VPN SD-WAN Zone:
Go to Policy & Object -> Firewall Policy, select 'Create New', select the respected interface for LAN to VPN communication, and add the required attribute.
Note: Ensure NAT is disabled in the policy, otherwise the traffic will use the tunnel Interface IP. If the peer end Internal LAN has no route for tunnel Interface IP the traffic will be dropped at the peer end Internal LAN.
Once the policy is created, it is possible to do a clone reverse of the policy, change the name, and enable the policy.
-
Configure Performance SLA for VPN Interface in SD-WAN:
Go to SDWAN - > Performance SLA, select 'Create New', and define the parameter (peer-end Local LAN machine IP has been used, as the server 10.25.12.3 ).
However, after the configuration, the Performance SLA will still be down.
-
Add the source in the SD-WAN VPN member from CLI to get this SLA working.
config system sdwan
config members
edit 4
set interface "VPN_2"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
edit 5
set interface "VPN_1"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
end
Note 1.Source IP can only be the interface IP, and it will work when the source IP is included in the VPN phase 2 traffic selector. In this configuration, local - 10.24.0.0/20 has been used and Remote 10.25.0.0/20 so 10.24.3.109 is in the respected range, that is why it is possible to use it.
Note 2.
If the peer end configuration is not done, the SLA will not come up even after adding the source, as the configuration is not complete and SLA relies on the tunnel configuration to reach the peer side server which is configured.
Create the SD-WAN Rule using the same performance SLA:
Go to Network - > SD-WAN Rules, select 'Create New', define the Parameter (the strategy as Lowest cost SLA has been used, it is possible to refer to the related articles and check which is suitable for the network, it is possible to configure either redundant or else load-balanced).
Latency, Best Quality, and Lowest cost SLA will use one interface at a time but with Maximum bandwidth the Traffic is load-balanced among interfaces.
Verify the VPN rule is above the all to all rule as the rule works top to bottom.
-
It is necessary to create a blackhole route for the destination subnets as if it is not and assume the scenario both tunnels are down the traffic will be routed to the ISP link if there is a policy match, and even if the tunnel comes back up the traffic will still use the existing session rather than using the tunnel until and unless the session is timed out or clear it manually.
Use CLI to configure the blackhole route, entry 4 was not there so it has been used, make sure the AD value that is used in the blackhole route is higher than the static which has been configured:
config router static
edit 4
set dst 10.25.0.0 255.255.240.0
set distance 15
set blackhole enable
endTo check whether the route is working or not bring down both tunnels and try initiating some traffic from the LAN, use the below command to check the route.
It is possible to see it shows the root as the next hop, also in debug flow filter, it is possible to see it routing it towards the root.
Firewall_Dual_ISP # diagnose ip rtcache list | grep 10.25
10.24.5.179@3(port1)->10.25.12.10@13(root) gwy=0.0.0.0 prefsrc=10.24.3.109Flow filter:
id=65308 trace_id=149 func=init_ip_session_common line=6076 msg="allocate a new session-00f532ea, tun_id=0.0.0.0"
id=65308 trace_id=149 func=iprope_dnat_check line=5331 msg="in-[port1], out-[]"
id=65308 trace_id=149 func=iprope_dnat_tree_check line=823 msg="len=0"
id=65308 trace_id=149 func=iprope_dnat_check line=5343 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=65308 trace_id=149 func=vf_ip_route_input_common line=2605 msg="find a route: flag=00000000 gw-10.25.12.10 via root"Firewall_Dual_ISP # diagnose ip address list
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=13 devname=root - Configure the Peer end Device, if it is not already done (the normal Route Priority has been used at the peer side for failover, it is also possible to configure SD-WAN on this side using the above example).
- Configure the two different tunnels for both peer IPs, using the Wan interface if the user is not configuring SD-WAN then use the IPsec wizard to configure it.
- If the wizard has been used, it will create the address, policy, static route, and blackhole route.
- It is necessary to change the VPN Interface IP.
- Once everything is created, it is possible to change the Priority of the route as per requirement, it is also possible to leave it as it is, to manage the tunnel from the Firewall which has the dual-link only.
To check the SD-WAN:
Firewall_Dual_ISP # diagnose sys sdwan member
Member(1): interface: port3, flags=0x0 , gateway: 10.27.11.206, priority: 1 1024, weight: 0
Member(2): interface: port4, flags=0x0 , gateway: 10.26.11.206, priority: 1 1024, weight: 0
Member(4): interface: VPN_2, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Member(5): interface: VPN_1, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Firewall_Dual_ISP # diagnose sys sdwan service
Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(2):
1: Seq_num(5 VPN_1), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(4 VPN_2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
Src address(1):
10.24.0.0-10.24.15.255
Dst address(1):
10.25.0.0-10.25.15.255
Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(load-balance hash-mode=round-robin)
Members(2):
1: Seq_num(1 port3), alive, sla(0x0), gid(0), num of pass(0), selected
2: Seq_num(2 port4), alive, sla(0x0), gid(0), num of pass(0), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
0.0.0.0-255.255.255.255
Firewall_Dual_ISP # diagnose sys sdwan health-check
Health Check(ping):
Seq(5 VPN_1): state(alive), packet-loss(0.000%) latency(1.476), jitter(0.197), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Seq(4 VPN_2): state(alive), packet-loss(0.000%) latency(1.340), jitter(0.244), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Firewall_Dual_ISP # diagno firewall proute list
list route policy info(vf=root):
id=2140340226(0x7f930002) vwl_service=2(VPN_SDWAN) vwl_mbr_seq=5 4 dscp_tag=0xfc 0xfc flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=28(VPN_1) oif=25(VPN_2)
source(1): 10.24.0.0-10.24.15.255
destination(1): 10.25.0.0-10.25.15.255
hit_count=0 last_used=2022-04-19 21:56:27
id=2140340225(0x7f930001) vwl_service=1(Internet) vwl_mbr_seq=1 2 dscp_tag=0xfc 0xfc flags=0x10 load-balance hash-mode=round-robin tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=5(port3) num_pass=-1 oif=6(port4) num_pass=-1
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255
hit_count=117383 last_used=2022-04-19 22:20:30
To check the VPN:
Technical Tip: Troubleshooting IPsec VPN tunnel errors with large size packets
Troubleshooting Tip: IPsec VPNs tunnels
Troubleshooting Tip: Troubleshooting IPsec Site-to-Site Tunnel Connectivity
Troubleshooting Tip: IPSEC Tunnel (debugging IKE)
Related documents:
Technical Tip: Configure FortiGate SD-WAN with an IPSEC VPN
Technical Note: How to use BGP and SD-WAN for advertising routes and path selection in FortiGate
Troubleshooting Tip: SD-WAN performance SLA for IPsec interface shows as 'down'
Technical Tip: How to use IPsec tunnel interface on Performance SLA
SD-WAN zones
Dual VPN Tunnel Wizard
SD-WAN rules
SD-WAN
