Description
This article describes the Integration of IPsec VPN with SD-WAN to manage IPsec traffic flow and Redundancy using the SDWAN rule.
Scope
FortiGate version 6.4 and above.
FortiGate version 7.0 and above.
Solution
To Manage the IPsec VPN with SD-WAN rather than using the route Priority.
Consider the Following Scenario.
Assuming the SD-WAN is already configured for the ISP, this is not a requirement for this setup but it is always good to have the ISP with SD-WAN.
Version 7.2.0 has been used to configure this setup.
- Configure the VPN Interface but not from IPsec Wizard as the interface created from IPsec wizard cannot be called in the SD-WAN member or to be precise when the tunnel is created from IPsec wizard it creates routes, policy, addresses, etc., and to configure FortiGate interfaces as SD-WAN members, it is necessary to remove or redirect existing configuration references.
So to avoid error-prone configuration a new wizard was added for SD-WAN VPN.
Go to Network -> SD-WAN, and select 'Create New' -> SDWAN Zone, the name VPN has been used, do not add any members as of now.
Now create an SD-WAN Member:
Go to Network -> SD-WAN, select 'Create New' -> SDWAN Member.
In the Interface drop-down, select +VPN. The Create IPsec VPN for SD-WAN members pane opens.
Enter the required information, then select 'Create'.
Name - Respected Tunnel Name (VPN_1).
Remote Device Ip address/ DDNS - The IP address has been used.
Outgoing Interface - The WAN 1 (For the setup it's port 3).
Authentication Mode - Pre-Shared Key/Signature (the pre-shared Key has been used).
Pre-Shared Key - (Define preshared key).
Review the settings and Create the Interface.
-
Now add the Interface in the respected zone if any zone has not been created for VPN create a new zone and add it.
Follow the same process create the 2nd VPN tunnel and add it in the same zone.
-
Change the VPN traffic selector as per requirement, as with the SD-WAN Wizard it will create any and any.
Configure the Address:
Go to Policy & Object -> Addresses, select 'Create new' - Address (Configure the local and remote address as per requirement) In this setup local - 10.24.0.0/20 and Remote 10.25.0.0/20.
If it is not desired to use this IPsec connection to go to Internet, go to VPN -> Ipsec Tunnel -> Respected Tunnel and change the phase 2 selector.
Note.
Verify the Tunnel configuration by going to the VPN -> Ipsec Tunnel - > VPN_1 & VPN_2.
-
Go to the respected VPN Interface and assign an IP address to the Interface, any gateway has been defined when configuring the SD-WAN member as even if any gateway has been configured there it will again populate it with 0.0.0.0.
Go to Network -> Interface - > Expand the WAN 1 and edit the VPN_1 interface.
Define the local and remote interface IP, 1.1.1.1 and 1.1.1.2 have been used for VPN_1 & VPN_2 -> 2.2.2.1 and 2.2.2.2.
-
Create the Static Route for the VPN traffic using the VPN SD-WAN zone created if FortiOS is running v7.0 and above.
Note:
On FortiOS v6.4.x, Static routes can be created for individual VPN interfaces or the entire SD-WAN interface but not for individual VPN SD-WAN zones. Creating static routes for individual VPN SD-WAN zones is supported only by FortiOS v7.0.
Go to Network -> Static Route, select 'Create New' (Enter the required information) and select 'Ok'. The following snippet is from FortiOS v7.0.0.
-
Create the Firewall policy for the respected VPN traffic using the VPN SD-WAN Zone:
Go to Policy & Object -> Firewall Policy, select 'Create New', select the respected interface for LAN to VPN communication, and add the required attribute.
Note.
Make sure NAT is disabled in the policy otherwise the traffic will use the tunnel Interface IP and if the peer end Internal LAN has no route for tunnel Interface IP the traffic will be dropped at the peer end Internal LAN.
Once the policy is created, it is possible to do a clone reverse of the policy, change the name, and enable the policy.
-
Configure Performance SLA for VPN Interface in SD-WAN:
Go to SDWAN - > Performance SLA, select 'Create New', and define the parameter (peer-end Local LAN machine IP has been used, as the server 10.25.12.3 ).
However, after the configuration, the Performance SLA will still be down.
-
To get this SLA working it is necessary to add the source in the SD-WAN VPN member from CLI
config system sdwan
config members
edit 4
set interface "VPN_2"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
edit 5
set interface "VPN_1"
set zone "VPN"
set source 10.24.3.109 <----- Added LAN interface IP.
next
end
Note 1.Source IP can only be the interface IP, and it will work when the source IP is included in the VPN phase 2 traffic selector, In this configuration, local - 10.24.0.0/20 has been used and Remote 10.25.0.0/20 so 10.24.3.109 is in the respected range, that is why it is possible to use it.
Note 2.
If the peer end configuration is not done the SLA will not come up even after adding the source as the configuration is not complete and SLA relies on the tunnel configuration to reach the peer side server which is configured.
Create the SD-WAN Rule using the same performance SLA:
Go to Network - > SD-WAN Rules, select 'Create New', define the Parameter (the strategy as Lowest cost SLA has been used, it is possible to refer to the related articles and check which is suitable for the network, it is possible to configure either redundant or else load-balanced).
Latency, Best Quality, and Lowest cost SLA will use one interface at a time but with Maximum bandwidth the Traffic is load-balanced among interfaces.
Make sure the VPN rule is above the all to all rule as the rule works top to bottom.
-
It is necessary to create a blackhole route for the destination subnets as if it is not and assume the scenario both tunnels are down the traffic will be routed to the ISP link if there is a policy match, and even if the tunnel comes back up the traffic will still use the existing session rather than using the tunnel until and unless the session is timed out or clear it manually.
Use CLI to configure the blackhole route, entry 4 was not there so it has been used, make sure the AD value that is used in the blackhole route is higher than the static which has been configured:
config router static
edit 4
set dst 10.25.0.0 255.255.240.0
set distance 15
set blackhole enable
endTo check whether the route is working or not bring down both tunnels and try initiating some traffic from the LAN, use the below command to check the route.
It is possible to see it shows the root as the next hop, also in debug flow filter, it is possible to see it routing it towards the root.
Firewall_Dual_ISP # diagnose ip rtcache list | grep 10.25
10.24.5.179@3(port1)->10.25.12.10@13(root) gwy=0.0.0.0 prefsrc=10.24.3.109Flow filter:
id=65308 trace_id=149 func=init_ip_session_common line=6076 msg="allocate a new session-00f532ea, tun_id=0.0.0.0"
id=65308 trace_id=149 func=iprope_dnat_check line=5331 msg="in-[port1], out-[]"
id=65308 trace_id=149 func=iprope_dnat_tree_check line=823 msg="len=0"
id=65308 trace_id=149 func=iprope_dnat_check line=5343 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=65308 trace_id=149 func=vf_ip_route_input_common line=2605 msg="find a route: flag=00000000 gw-10.25.12.10 via root"Firewall_Dual_ISP # diagnose ip address list
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=13 devname=root - Configure the Peer end Device, if it is not already done (the normal Route Priority has been used at the peer side for failover, it is also possible to configure SD-WAN on this side using the above example).
- Configure the two different tunnels for both peer IPs, using the Wan interface if the user is not configuring SD-WAN then use the IPsec wizard to configure it.
- If the wizard has been used, it will create the address, policy, static route, and blackhole route.
- It is necessary to change the VPN Interface IP.
- Once everything is created, it is possible to change the Priority of the route as per requirement, it is also possible to leave it as it is, to manage the tunnel from the Firewall which has the dual-link only.
To check the SD-WAN:
Firewall_Dual_ISP # diagnose sys sdwan member
Member(1): interface: port3, flags=0x0 , gateway: 10.27.11.206, priority: 1 1024, weight: 0
Member(2): interface: port4, flags=0x0 , gateway: 10.26.11.206, priority: 1 1024, weight: 0
Member(4): interface: VPN_2, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Member(5): interface: VPN_1, flags=0xc , gateway: 10.28.4.137, priority: 1 1024, weight: 0
Firewall_Dual_ISP # diagnose sys sdwan service
Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(2):
1: Seq_num(5 VPN_1), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(4 VPN_2), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
Src address(1):
10.24.0.0-10.24.15.255
Dst address(1):
10.25.0.0-10.25.15.255
Service(1): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(load-balance hash-mode=round-robin)
Members(2):
1: Seq_num(1 port3), alive, sla(0x0), gid(0), num of pass(0), selected
2: Seq_num(2 port4), alive, sla(0x0), gid(0), num of pass(0), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
0.0.0.0-255.255.255.255
Firewall_Dual_ISP # diagnose sys sdwan health-check
Health Check(ping):
Seq(5 VPN_1): state(alive), packet-loss(0.000%) latency(1.476), jitter(0.197), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Seq(4 VPN_2): state(alive), packet-loss(0.000%) latency(1.340), jitter(0.244), mos(4.403), bandwidth-up(9999999), bandwidth-dw(9999999), bandwidth-bi(19999998) sla_map=0x1
Firewall_Dual_ISP # diagno firewall proute list
list route policy info(vf=root):
id=2140340226(0x7f930002) vwl_service=2(VPN_SDWAN) vwl_mbr_seq=5 4 dscp_tag=0xfc 0xfc flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=28(VPN_1) oif=25(VPN_2)
source(1): 10.24.0.0-10.24.15.255
destination(1): 10.25.0.0-10.25.15.255
hit_count=0 last_used=2022-04-19 21:56:27
id=2140340225(0x7f930001) vwl_service=1(Internet) vwl_mbr_seq=1 2 dscp_tag=0xfc 0xfc flags=0x10 load-balance hash-mode=round-robin tos=0x00 tos_mask=0x00 protocol=0 sport=0-65535 iif=0 dport=1-65535 path(2) oif=5(port3) num_pass=-1 oif=6(port4) num_pass=-1
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255
hit_count=117383 last_used=2022-04-19 22:20:30
To check the VPN:
Technical Tip: Troubleshooting IPsec VPN tunnel errors with large size packets
Troubleshooting Tip: IPsec VPNs tunnels
Troubleshooting Tip: Troubleshooting IPsec Site-to-Site Tunnel Connectivity
Troubleshooting Tip: IPSEC Tunnel (debugging IKE)
Related documents:
Technical Tip: Configure FortiGate SD-WAN with an IPSEC VPN
Technical Note: How to use BGP and SD-WAN for advertising routes and path selection in FortiGate
Troubleshooting Tip: SD-WAN performance SLA for IPsec interface shows as 'down'
Technical Tip: How to use IPsec tunnel interface on Performance SLA
SD-WAN zones
Dual VPN Tunnel Wizard
SD-WAN rules
SD-WAN
