Technical Tip: Configure FortiGate SD-WAN with an IPSEC VPN

FortiKoala · ‎04-12-2018

Description
This article describes how to configure SD-WAN in combination with IPSEC VPN tunnels. From FortiOS 6.0 the SD-WAN feature is more granular and allows the combination of IPSEC tunnel interfaces with regular interfaces.

Scope

The scope of this article considers that HQ has 2 Internet connections called ISP1 and ISP2 shown in the diagram with Blue and Green lines describing the different paths
    - ISP1 is connected to port1.
    - ISP2 is connected to port2.
    - Branch has 1 internet connection with 2 IPSEC tunnels toward HQ ISP1 and HQ ISP2.
    - All traffic from the Branch has to go through HQ, i.e. Branch users connect to the internet through HQ.
    - Internet security policies are applied to HQ.

The routing preferences are :
    - Branch to HQ : 1 Blue VPN, 2 Green VPN
    - Branch to Internet (through HQ) : 1 Green VPN, 2 Blue VPN
    - HQ to Branch : 1 Blue VPN, 2 Green VPN
    - HQ and Branch to Internet : 1 ISP1, 2 ISP2

The best path selection is based on the configured service level agreement (SLA).

Solution
The following solution explains how to configure the HQ and Branch FortiGates

HQ Configuration:

VPN’s toward Branch are already configured as follows:
    - port1_p1 : VPN over ISP1
    - port2_p1 : VPN over ISP2

1. Configure IP addresses on tunnel interfaces for health checks:

config system interface
    edit "port1_p1"
        set ip 1.1.1.1 255.255.255.255
        set remote-ip 1.1.1.2 255.255.255.255
    next
    edit "port2_p1"
        set ip 2.2.2.1 255.255.255.255
        set remote-ip 2.2.2.2 255.255.255.255
    next
end

2. Add port1 (ISP1), port2 (ISP2), port1_p1, port2_p1 to SD-WAN:

Note: port1_p1, port2_p1 have no gateway since they are tunnel interfaces

config system virtual-wan-link
    set status enable
    config members
        edit 1
            set interface "port1"
            set gateway 172.31.143.254
        next
        edit 2
            set interface "port2"
            set gateway 172.31.201.13
        next
        edit 3
            set interface "port1_p1"
        next
        edit 4
            set interface "port2_p1"
        next
    end

3. Configure separate health-checks for the internet connection and IPSEC VPNs:

config system virtual-wan-link
config health-check
        edit "PingGoogle"
            set server "8.8.8.8"
            set members 1 2
            config sla
                edit 1
                    set latency-threshold 20
                    set packetloss-threshold 1
                next
            end
        next
        edit "PingRemoteHost"
            set server "10.119.11.187"
            set members 3 4
            config sla
                edit 1
                    set packetloss-threshold 1
                next
            end
        next
    end
end

4. Configure SD-WAN rules and address objects:

config firewall address
    edit "Tunnel_port1"
        set subnet 1.1.1.0 255.255.255.252
    next
    edit "Tunnel_port2"
        set subnet 2.2.2.0 255.255.255.252
    next
end

config system virtual-wan-link
config service
        edit 4
            set name "To_Branch"
            set mode sla
            set dst "Remote_subnet"
            set src "all"
            config sla
                edit "PingRemoteHost"
                    set id 1
                next
            end
            set priority-members 4 3
        next
        edit 2
            set name "To_internet"
            set mode sla
            set dst "all"
            set src "all"
            config sla
                edit "PingGoogle"
                    set id 1
                next
            end
            set priority-members 1 2
        next
    end
end

5. The default route points towards the virtual-wan-link (SD-WAN) interface:

config router static
    edit 1
        set distance 1
        set virtual-wan-link enable
    next
end

6. Four Firewall policies are created to allow the traffic as follows:
    - from LAN HQ to Branch (ID 5)
    - from Branch to LAN HQ (ID 4)
    - from LAN HQ to Internet (ID 3)
    - from Branch to Internet (ID 6)

config firewall policy
    edit 5
        set name "LAN_HQ->BR"
        set srcintf "port5"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "Remote_subnet"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 4
        set name "BR->LAN_HQ"
        set srcintf "virtual-wan-link"
        set dstintf "port5"
        set srcaddr "Remote_subnet" "Tunnel_port1" "Tunnel_port2"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 3
        set name "LAN_HQ->Internet"
        set srcintf "port5"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set logtraffic all
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
    edit 6
        set name "BR->Internet"
        set srcintf "virtual-wan-link"
        set dstintf "virtual-wan-link"
        set srcaddr "Remote_subnet"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
end

Branch configuration:

HQ VPNs towards the Branch are already configured as follows:
- to_port1_p1 : VPN toward HQ ISP1
- to_port2_p1 : VPN toward HQ ISP2

1. Configure IP addresses on the tunnel interfaces for health checks:

config system interface
    edit "to_port1_p1"
        set ip 1.1.1.2 255.255.255.255
        set remote-ip 1.1.1.1 255.255.255.255
    next
    edit "to_port2_p1"
        set ip 2.2.2.2 255.255.255.255
        set remote-ip 2.2.2.1 255.255.255.255
    next
end

2. Add to_port1_p1, to_port2_p1 to SD-WAN:

config system virtual-wan-link
    set status enable
    config members
        edit 1
            set interface "to_port1_p1"
        next
        edit 2
            set interface "to_port2_p1"
        next
    end

3. Configure health-checks for IPSEC VPNs:

config system virtual-wan-link
    config health-check
        edit "PingHQFGT"
            set server "10.118.3.5"
            set members 1 2
            config sla
                edit 1
                    set packetloss-threshold 1
                next
            end
        next
    end
end

4. Configure SD-WAN rules and address object:

config firewall address
    edit "HQ_Subnet"
        set subnet 10.118.0.0 255.255.240.0
    next
end

config system virtual-wan-link
    config service
        edit 2
            set name "To_HQ"
            set mode sla
            set dst "HQ_Subnet"
            set src "all"
            config sla
                edit "PingHQFGT"
                    set id 1
                next
            end
            set priority-members 2 1
        next
        edit 1
            set name "To_Internet"
            set mode sla
            set dst "all"
            set src "all"
            config sla
                edit "PingHQFGT"
                    set id 1
                next
            end
            set priority-members 1 2
        next
    end
end

5. The default route points towards the virtual-wan-link (SD-WAN) interface. Routes toward the remote VPN gateway are added on wan1 in order to establish the VPN tunnels:

config router static
    edit 2
        set dst 172.31.195.5 255.255.255.255
        set gateway 10.5.31.254
        set device "wan1"
    next
    edit 3
        set dst 172.31.131.5 255.255.255.255
        set gateway 10.5.31.254
        set device "wan1"
    next
    edit 4
        set distance 1
        set virtual-wan-link enable
    next
end

6. Two Firewall policies are created on SD-WAN interface to allow the traffic as follows:
- from Branch to “any” (ID 1)
- from “any” to branch (ID 2)

config firewall policy
    edit 1
        set name "BR->HQ"
        set uuid 2e7f39aa-3351-51e8-97c8-a3aa295355ba
        set srcintf "internal"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 2
        set name "HQ->BR"
        set uuid 0348d2a4-3352-51e8-9fe3-d5e6c49308e6
        set srcintf "virtual-wan-link"
        set dstintf "internal"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
end

Verify the configuration

Health check status:

FG300E-HQ # diagnose sys virtual-wan-link health-check
Health Check(PingGoogle):
Seq(1): state(alive), packet-loss(0.000%) latency(4.140), jitter(0.138) sla_map=0x1
Seq(2:( state(alive), packet-loss(0.000%) latency(4.082), jitter(0.032) sla_map=0x1
Health Check(PingRemoteHost):
Seq(3): state(alive), packet-loss(0.000%) latency(1.699), jitter(0.080) sla_map=0x1
Seq(4:( state(alive), packet-loss(0.000%) latency(1.843), jitter(0.376) sla_map=0x1

SD-WAN rules path selection:

FG300E-HQ # diagnose sys virtual-wan-link service
Service(4): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(4), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(3), alive, sla(0x1),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 10.119.0.0-10.119.15.255

Service(2): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 0.0.0.0-255.255.255.255

FG300E-HQ # diagnose firewall proute list
list route policy info(vf=root):

id=4278648836 vwl_service=4(To_Branch) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=45 oif=44
source(1): 0.0.0.0-255.255.255.255
destination(1): 10.119.0.0-10.119.15.255

id=4278648834 vwl_service=2(To_internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=9 oif=10
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255

Branch:

FG60D-BR # diagnose sys virtual-wan-link health-check
Health Check(PingHQFGT):
Seq(1): state(alive), packet-loss(0.000%) latency(1.221), jitter(0.241) sla_map=0x1
Seq(2): state(alive), packet-loss(0.000%) latency(1.188), jitter(0.255) sla_map=0x1

FG60D-BR # diagnose sys virtual-wan-link service
Service(2): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(2), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(1), alive, sla(0x1),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 10.118.0.0-10.118.15.255

Service(1): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 0.0.0.0-255.255.255.255

FG60D-BR # diagnose firewall proute list
list route policy info(vf=root):

id=4278517762 vwl_service=2(To_HQ) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=20 oif=19
source(1): 0.0.0.0-255.255.255.255
destination(1): 10.118.0.0-10.118.15.255

id=4278517761 vwl_service=1(To_Internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=19 oif=20
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255

Problem: Packet loss occurs on HQ ISP2 and HQ -> Branch communication goes to green VPN

HQ:

FG300E-HQ # diagnose sys virtual-wan-link health-check
Health Check(PingGoogle):
Seq(1): state(alive), packet-loss(0.000%) latency(4.997), jitter(1.210) sla_map=0x1
Seq(2:( state(alive), packet-loss(20.000%) latency(4.799), jitter(0.975) sla_map=0x0
Health Check(PingRemoteHost):
Seq(3): state(alive), packet-loss(0.000%) latency(1.675), jitter(0.129) sla_map=0x1
Seq(4:( state(alive), packet-loss(23.333%) latency(1.862), jitter(0.483) sla_map=0x0

FG300E-HQ # diagnose sys virtual-wan-link service

Service(4): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(3), alive, sla(0x1),cfg_order(1), selected
    2: Seq_num(4), alive, sla(0x0),cfg_order(0), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 10.119.0.0-10.119.15.255

Service(2): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x0),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 0.0.0.0-255.255.255.255

FG300E-HQ # diagnose firewall proute list
list route policy info(vf=root):

id=4278648836 vwl_service=4(To_Branch) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=44 oif=45
source(1): 0.0.0.0-255.255.255.255
destination(1): 10.119.0.0-10.119.15.255

id=4278648834 vwl_service=2(To_internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=9 oif=10
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255

Branch:

FG60D-BR # diag sys virtual-wan-link health-check
Health Check(PingHQFGT):
Seq(1): state(alive), packet-loss(0.000%) latency(1.241), jitter(0.175) sla_map=0x1
Seq(2): state(alive), packet-loss(26.667%) latency(1.166), jitter(0.226) sla_map=0x0

FG60D-BR # diag sys virtual-wanabou-link service

Service(2): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(1), selected
    2: Seq_num(2), alive, sla(0x0),cfg_order(0), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 10.118.0.0-10.118.15.255

Service(1): Address Mode(IPV4) flags=0x0
TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x0),cfg_order(1), selected
Src address: 0.0.0.0-255.255.255.255
Dst address: 0.0.0.0-255.255.255.255

FG60D-BR # diagnose firewall proute list
list route policy info(vf=root):

id=4278517762 vwl_service=2(To_HQ) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=19 oif=20
source(1): 0.0.0.0-255.255.255.255
destination(1): 10.118.0.0-10.118.15.255

id=4278517761 vwl_service=1(To_Internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=19 oif=20
source(1): 0.0.0.0-255.255.255.255
destination(1): 0.0.0.0-255.255.255.255

Related Articles

Technical Note: How to use BGP and SD-WAN for advertising routes and path selection in FortiGate