FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
nvisentin_FTNT
Article Id 190055

Purpose
This article describes how to use BGP to advertise routes and SD-WAN for path selection.

Scope
From FortiOS 6.0, the SD-WAN feature supports dynamic routing.
Diagram


Expectations, Requirements

Scenario : HQ has 2 Internet connections called ISP1 and ISP2.
 
Branch has 1 Internet connection with 2 Ipsec tunnels toward HQ ISP1 and HQ ISP2.
    - VPN 1 on ISP1: HQ Tunnel IP 1.1.1.1; Branch Tunnel IP 1.1.1.2
    - VPN 2 on ISP2: HQ Tunnel IP 2.2.2.1; Branch Tunnel IP 2.2.2.2

iBGP peering is configured on each VPN.

HQ routing logic :
Branch advertises its subnet with a BGP community:
10.119.0.0/20 → BGP Community 2:1

HQ applies a route tag depending the received BGP community:
BGP community 2:1 → route tag 21

HQ will then apply SD-WAN rules based on the source address and route tag:
    - route tag 21 and source 10.118.0.0/20 → VPN 1 as primary path, VPN 2 as secondary path
    - route tag 21 and source 10.111.0.0/20 → VPN 2 as primary path, VPN 1 as secondary path
    - routes without tag (Internet access) →  port1 as primary path, port2 as secondary path

Branch routing logic :
HQ advertises each subnet with a unique BGP community:
10.118.0.0/20 -> BGP community 1:1
10.111.0.0/20 -> BGP community 1:2

Branch applies a route tag depending on the received BGP community:
BGP community 1:1 → route tag 11
BGP community 1:2 → route tag 12

Branch will then apply SD-WAN rules based on route tag:
    - route tag 11 → VPN 1 as primary path, VPN 2 as secondary path
    - route tag 12 → VPN 2 as primary path, VPN 1 as secondary path
    - routes without tag (Internet through HQ) → VPN 1 & VPN 2 (default ECMP rule)

SD-WAN path selection is based on the configured SLA.


Configuration
HQ:

VPN’s toward Branch are already configured as follows:
    - port1_p1 : VPN over ISP1
    - port2_p1 : VPN over ISP2

1. Configure IP addresses on tunnel interfaces for health checks and iBGP peerings:

config system interface
    edit "port1_p1"
        set ip 1.1.1.1 255.255.255.255
        set remote-ip 1.1.1.2 255.255.255.255
    next
    edit "port2_p1"
        set ip 2.2.2.1 255.255.255.255
        set remote-ip 2.2.2.2 255.255.255.255
    next
end

2. port1 (ISP1), port2 (ISP2), port1_p1, port2_p1 are added to SD-WAN.
Note : port1_p1, port2_p1 have no gateway since they are tunnel interfaces

config system virtual-wan-link
    set status enable
    config members
        edit 1
            set interface "port1"
            set gateway 172.31.143.254         
        next
        edit 2
            set interface "port2"
            set gateway 172.31.201.13            
        next
        edit 3
            set interface "port1_p1"
        next
        edit 4
            set interface "port2_p1"
        next
    end

3. Configure separate health-checks for Internet connection and Ipsec VPN’s:

config system virtual-wan-link
config health-check
        edit "PingGoogle"
            set server "8.8.8.8"
            set members 1 2
            config sla
                edit 1
                next
            end
        next
        edit "PingBR"
            set server "10.119.2.36"
            set members 3 4
            config sla
                edit 1
                next
            end
        next
    end
end

4. Configure prefix-lists, route-map and BGP in order to set the BGP communities on the advertised routes:

config router prefix-list
    edit "HQ_LAN_1"
        config rule
            edit 1
                set prefix 10.118.0.0 255.255.240.0
                unset ge
                unset le
            next
        end
    next
    edit "HQ_LAN_2"
        config rule
            edit 1
                set prefix 10.111.0.0 255.255.240.0
                unset ge
                unset le
            next
        end
    next
end

config router route-map
    edit "Set_community"
        config rule
            edit 1
                set match-ip-address "HQ_LAN_1"
                set set-community "1:1"
            next
            edit 2
                set match-ip-address "HQ_LAN_2"
                set set-community "1:2"
            next
            edit 3
            next
        end
    next
end

config router bgp
    set as 1
    config neighbor
        edit "1.1.1.2"
            set remote-as 1
            set route-map-out "Set_community"
        next
        edit "2.2.2.2"
            set remote-as 1
            set route-map-out "Set_community"
        next
    end
    config network
        edit 1
            set prefix 10.118.0.0 255.255.240.0
        next
        edit 2
            set prefix 10.111.0.0 255.255.240.0
        next
    end
end

5. Configure community-lists, route-map and BGP in order to set the route-tag depending on the received BGP community:

config router community-list
    edit "Branch_LAN"
        config rule
            edit 1
                set action permit
                set match "2:1"
            next
        end
    next
end
config router route-map
    edit "Community_to_tag"
        config rule
            edit 1
                set match-community "Branch_LAN"
                set set-route-tag 21
            next
            edit 2
            next
        end
    next
end

config router bgp
    config neighbor
        edit "1.1.1.2"
            set route-map-in "Community_to_tag"
        next
        edit "2.2.2.2"
            set route-map-in "Community_to_tag"
        next
    end

6. Configure SD-WAN rules and address objects:

config firewall address
    edit "Remote_subnet"
        set subnet 10.119.0.0 255.255.240.0
    next
    edit "Tunnel_port1"
        set subnet 1.1.1.0 255.255.255.252
    next
    edit "Tunnel_port2"
        set subnet 2.2.2.0 255.255.255.252
    next
    edit "HQ_Subnet_1"
        set subnet 10.118.0.0 255.255.240.0
    next
    edit "HQ_Subnet_2"
        set subnet 10.111.0.0 255.255.240.0
    next
end

config system virtual-wan-link
config service
        edit 4
            set name "To_Branch_1"
            set mode sla
            set route-tag 21
            set src "HQ_Subnet_1"
            config sla
                edit "PingBR"
                    set id 1
                next
            end
            set priority-members 3 4
        next
        edit 3
            set name "To_Branch_2"
            set mode sla
            set route-tag 21
            set src "HQ_Subnet_2"
            config sla
                edit "PingBR"
                    set id 1
                next
            end
            set priority-members 4 3
        next
        edit 2
            set name "To_Internet"
            set mode sla
            set dst "all"
            config sla
                edit "PingGoogle"
                    set id 1
                next
            end
            set priority-members 1 2
        next
    end
end

7. The default route points to the virtual-wan-link (SD-WAN) interface:

config router static
    edit 1
        set distance 1
        set virtual-wan-link enable
    next
end

8. 4 Firewall policies are created to allow the traffic as follows:
    - from LAN HQ to Branch (ID 5)
    - from Branch to LAN HQ 1 (ID 4)
    - from Branch to LAN HQ 2 (ID 7)
    - from LAN HQ to Internet (ID 3)
    - from Branch to Internet (ID 6)

config firewall policy
    edit 5
        set name "LAN_HQ->BR"
        set srcintf "port5" "port6"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "Remote_subnet"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 4
        set name "BR->LAN_HQ"
        set srcintf "virtual-wan-link"
        set dstintf "port5"
        set srcaddr "Remote_subnet" "Tunnel_port1" "Tunnel_port2"
        set dstaddr "HQ_Subnet_1"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 7
        set name "BR->LAN_HQ_2"
        set srcintf "virtual-wan-link"
        set dstintf "port6"
        set srcaddr "Remote_subnet"
        set dstaddr "HQ_Subnet_2"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 3
        set name "LAN_HQ->Internet"
        set srcintf "port5" "port6"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set logtraffic all
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
    edit 6
        set name "BR->Internet"
        set srcintf "virtual-wan-link"
        set dstintf "virtual-wan-link"
        set srcaddr "Remote_subnet"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set av-profile "default"
        set webfilter-profile "monitor-all"
        set ips-sensor "default"
        set application-list "default"
        set profile-protocol-options "default"
        set ssl-ssh-profile "certificate-inspection"
        set nat enable
    next
end

Branch:

VPN’s toward HQ are already configured as follows:
to_port1_p1 : VPN toward HQ ISP1
to_port2_p1 : VPN toward HQ ISP2

1. Configure IP addresses on tunnel interfaces for health checks:

config system interface
    edit "to_port1_p1"
        set ip 1.1.1.2 255.255.255.255
        set remote-ip 1.1.1.1 255.255.255.255
    next
    edit "to_port2_p1"
        set ip 2.2.2.2 255.255.255.255
        set remote-ip 2.2.2.1 255.255.255.255
    next
end

2. to_port1_p1, to_port2_p1 are added to SD-WAN:

config system virtual-wan-link
    set status enable
    config members
        edit 1
            set interface "to_port1_p1"
        next
        edit 2
            set interface "to_port2_p1"
        next
    end

3. Configure health-checks for Ipsec VPN’s:

config system virtual-wan-link
    config health-check
        edit "PingHQFGT"
            set server "10.118.3.5"
            set members 1 2
            config sla
                edit 1
                    set packetloss-threshold 1
                next
            end
        next
    end
end

4. Configure prefix-lists, route-map and BGP in order to set the BGP communities on the advertised routes:

config router prefix-list
    edit "Branch_LAN"
        config rule
            edit 1
                set prefix 10.119.0.0 255.255.240.0
                unset ge
                unset le
            next
        end
    next
end

config router route-map
    edit "Set_community"
        config rule
            edit 1
                set match-ip-address "Branch_LAN"
                set set-community "2:1"
            next
            edit 2
            next
        end
    next
end

config router bgp
    set as 1
    config neighbor
        edit "1.1.1.1"
            set remote-as 1
            set route-map-out "Set_community"
        next
        edit "2.2.2.1"
            set remote-as 1
            set route-map-out "Set_community"
        next
    end
    config network
        edit 1
            set prefix 10.119.0.0 255.255.240.0
        next
    end

5. Configure community-lists, route-map and BGP in order to set the route-tag depending on the received BGP community:

config router community-list
    edit "HQ_LAN_1"
        config rule
            edit 1
                set action permit
                set match "1:1"
            next
        end
    next
    edit "HQ_LAN_2"
        config rule
            edit 1
                set action permit
                set match "1:2"
            next
        end
    next
end

config router route-map
    edit "Community_to_tag"
        config rule
            edit 1
                set match-community "HQ_LAN_1"
                set set-route-tag 11
            next
            edit 2
                set match-community "HQ_LAN_2"
                set set-route-tag 12
            next
            edit 3
            next
        end
    next
end

config router bgp
    config neighbor
        edit "1.1.1.1"
            set route-map-in "Community_to_tag"
        next
        edit "2.2.2.1"
            set route-map-in "Community_to_tag"
        next
    end

6. Configure SD-WAN rules:

config system virtual-wan-link
    config service
        edit 2
            set name "To_HQ_2"
            set mode sla
            set route-tag 12
            config sla
                edit "PingHQFGT"
                    set id 1
                next
            end
            set priority-members 2 1
        next
        edit 1
            set name "To_HQ_1"
            set mode sla
            set route-tag 11
            config sla
                edit "PingHQFGT"
                    set id 1
                next
            end
            set priority-members 1 2
        next
    end
end

7. The default route points the virtual-wan-link (SD-WAN) interface. Routes toward the remote VPN gateway are added on wan1 in order to establish the VPN tunnels:

config router static
    edit 2
        set dst 172.31.195.5 255.255.255.255
        set gateway 10.5.31.254
        set device "wan1"
    next
    edit 3
        set dst 172.31.131.5 255.255.255.255
        set gateway 10.5.31.254
        set device "wan1"
    next
    edit 4
        set distance 1
        set virtual-wan-link enable
    next
end

8. 2 Firewall policies are created on SD-WAN interface to allow the traffic as follows:

    - from Branch to “any” (ID 1)
    - from “any internal IPs” to branch (ID 2)

config firewall address
    edit "HQ_Subnet_1"
        set subnet 10.118.0.0 255.255.240.0
    next
    edit "Tunnel_1"
        set subnet 1.1.1.0 255.255.255.252
    next
    edit "Tunnel_2"
        set subnet 2.2.2.0 255.255.255.252
    next
    edit "HQ_Subnet_2"
        set subnet 10.111.0.0 255.255.240.0
    next
end

config firewall policy
    edit 1
        set name "BR->HQ"
        set srcintf "internal"
        set dstintf "virtual-wan-link"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
    edit 2
        set name "HQ->BR"
        set srcintf "virtual-wan-link"
        set dstintf "internal"
        set srcaddr "HQ_Subnet_1" "Tunnel_1" "Tunnel_2" "HQ_Subnet_2"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
end


Verification
HQ:

Received BGP networks are properly tagged:

FG300E-HQ # get router info bgp network 10.119.0.0
BGP routing table entry for 10.119.0.0/20
Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.2 from 1.1.1.2 (10.119.2.36)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 2:1
      Last update: Mon Apr 16 09:50:35 2018

  Local
    2.2.2.2 from 2.2.2.2 (10.119.2.36)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 2:1
      Last update: Mon Apr 16 09:50:39 2018

FG300E-HQ # get router info bgp network
BGP table version is 22, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*> 10.111.0.0/20    0.0.0.0                       100  32768        0 i
*> 10.118.0.0/20    0.0.0.0                       100  32768        0 i
*>i10.119.0.0/20    1.1.1.2                  0    100      0       21 i
* i                 2.2.2.2                  0    100      0       21 i

Total number of prefixes 3

SD-WAN rules path selection:

FG300E-HQ # diag sys virtual-wan-link service
Service(4): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(3), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(4), alive, sla(0x1),cfg_order(1), selected
  Src address: 10.118.0.0-10.118.15.255
  Route tag address: 10.119.0.0/255.255.240.0

Service(3): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(4), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(3), alive, sla(0x1),cfg_order(1), selected
  Src address: 10.111.0.0-10.111.15.255
  Route tag address: 10.119.0.0/255.255.240.0

Service(2): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
  Dst address: 0.0.0.0-255.255.255.255

FG300E-HQ # diagnose firewall proute list
list route policy info(vf=root):

id=4280680452 vwl_service=4(To_Branch_1) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=44 oif=45
source(1): 10.118.0.0-10.118.15.255
destination wildcard(1): 10.119.0.0/255.255.240.0

id=4280680451 vwl_service=3(To_Branch_2) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=45 oif=44
source(1): 10.111.0.0-10.111.15.255
destination wildcard(1): 10.119.0.0/255.255.240.0


id=4280680450 vwl_service=2(To_Internet) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=9 oif=10
destination(1): 0.0.0.0-255.255.255.255
source wildcard(1): 0.0.0.0/0.0.0.0

Branch:

Received BGP networks are properly tagged:

FG60D-BR # get router info bgp network 10.118.0.0
BGP routing table entry for 10.118.0.0/20

Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.1 from 1.1.1.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 1:1
      Last update: Mon Apr 16 09:21:17 2018

  Local
    2.2.2.1 from 2.2.2.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 1:1
      Last update: Mon Apr 16 09:21:31 2018

FG60D-BR # get router info bgp network 10.111.0.0
BGP routing table entry for 10.111.0.0/20
Paths: (2 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  Local
    1.1.1.1 from 1.1.1.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal, best
      Community: 1:2
      Last update: Mon Apr 16 09:21:17 2018

  Local
    2.2.2.1 from 2.2.2.1 (10.10.10.10)
      Origin IGP metric 0, localpref 100, valid, internal
      Community: 1:2
      Last update: Mon Apr 16 09:21:31 2018

FG60D-BR # get router info bgp network
BGP table version is 8, local router ID is 10.119.2.36
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*>i10.111.0.0/20    1.1.1.1                  0    100      0       12 i
* i                 2.2.2.1                  0    100      0       12 i
*>i10.118.0.0/20    1.1.1.1                  0    100      0       11 i
* i                 2.2.2.1                  0    100      0       11 i
*> 10.119.0.0/20    0.0.0.0                       100  32768        0 i



FG60D-BR # diagnose sys virtual-wan-link service
Service(2): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(2), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(1), alive, sla(0x1),cfg_order(1), selected
  Route tag address: 10.111.0.0/255.255.240.0

Service(1): Address Mode(IPV4) flags=0x0
  TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla)
  Members:
    1: Seq_num(1), alive, sla(0x1),cfg_order(0), selected
    2: Seq_num(2), alive, sla(0x1),cfg_order(1), selected
  Route tag address: 10.118.0.0/255.255.240.0

FG60D-BR # diagnose firewall proute list
list route policy info(vf=root):

id=4279697410 vwl_service=2(To_HQ_2) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=20 oif=19
source wildcard(1): 0.0.0.0/0.0.0.0
destination wildcard(1): 10.111.0.0/255.255.240.0

id=4279697409 vwl_service=1(To_HQ_1) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sport=0:65535 iif=0 dport=1-65535 oif=19 oif=20
source wildcard(1): 0.0.0.0/0.0.0.0
destination wildcard(1): 10.118.0.0/255.255.240.0

Related Articles

Technical Tip: Configure FortiGate SD-WAN with an IPSEC VPN

Contributors