FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Jite
Staff
Staff
Article Id 249071
Description This article explains how to use a link monitor to trigger full BGP traffic failover to a secondary ISP.
Scope All FortiGate firmware.
Solution

This article will use the following scenario as an example

 

There are 2 service providers (ISP_1 and ISP_2) who provide internet service over BGP peers. ISP_1 is the primary ISP and ISP_2 is the secondary ISP. Due to memory constraints, the current configuration has been set up to only receive default routes from the ISPs. The organization's prefix is advertised to the internet using one ISP at a time to avoid asymmetric routing and to have proper stateful inspection. 

 

Jite_0-1678798746257.png

 

If ISP_1 experiences a problem on the path to the internet behind the directly connected edge, the router will be unaware that ISP_1 has lost internet connectivity and will still make use of the default route it receives from the ISP_1 Provider Edge Router. The router will also continue advertising the organization's prefix to this faulty upstream. 

 

Jite_1-1678798746258.png

 

Traditional conditional advertisement will not work in this scenario as the only prefix learned over BGP from the ISPs is the default route (0.0.0.0/0). 

 

Link monitor is a tool that can be used to monitor availability of an IP address over a link. Link monitor is generally used to trigger and/or withdraw static routes. This article explores how to use this tool to monitor and trigger BGP routes and advertisements. 

 

 

Process: 

1) Create 2 default routes, pointing to the primary and the secondary ISPs, with primary having lower priority:

 

# config router static 

    edit 6 

        set gateway 10.10.10.1 

        set device "port3" 

    next 

    edit 7 

        set gateway 20.20.20.1 

        set priority 2 

        set device "port4" 

    next 

end 

 

2) Create a ‘dummy’ static route, pointing to ISP_1 on the FortiGate:

 

# config router static 

    edit 5 

        set dst 169.254.0.1 255.255.255.255 

        set gateway 10.10.10.1 

        set device "port3" 

    next 

end 

 

3) Configure a link monitor for a reliable IP over the internet. 

 

# config system link-monitor 

    edit "1" 

        set srcintf "port3" 

        set server "40.40.40.40" 

        set gateway-ip 10.10.10.1   

    next 

end 

 

4) Insert a dummy route into the BGP network table. 

 

# config router bgp 

    set as 4000 

    config network 

        edit 1 

            set prefix 1.1.1.0 255.255.255.0 

        next 

        edit 2 

            set prefix 169.254.0.1 255.255.255.255 

        next 

    end 

end 

 

 

5) Configure BGP conditional advertisement to advertise the prefix to ISP_1 when the dummy route is present in the BGP network table. This process is broken down into 4 steps. 

 

5.a) Prepare a prefix-list and route-map for the prefixes:

 

# config router prefix-list 

    edit "Public_internal" 

        config rule 

            edit 1 

                set prefix 1.1.1.0 255.255.255.0 

                unset ge 

                unset le 

            next 

        end 

    next 

end 

 

# config router route-map 

    edit "My_Prefix" 

        config rule 

            edit 1 

                set match-ip-address "Public_internal" 

            next 

        end 

    next 

end 

 

5.b) Create a prefix-list and route-map for the ‘dummy’ prefix. 

 

# config router prefix-list 

    edit "dummy_list" 

        config rule 

            edit 1 

                set prefix 169.254.0.1 255.255.255.255 

                unset ge 

                unset le 

            next 

        end 

    next 

end 

 

# config router route-map 

    edit "dummy_map" 

        config rule 

            edit 1 

                set match-ip-address "dummy_list" 

            next 

        end 

    next 

end 

 

5.c) Advertise the route-map for the prefix to the primary ISP only when the ‘dummy’ route is present from BGP network table:

 

# config router bgp 

set as 4000 

config neighbor 

edit "10.10.10.1" 

set capability-graceful-restart enable 

set soft-reconfiguration enable 

set interface "port3" 

set remote-as 1111 

set route-map-out "My_Prefix"  <=== This ensures the dummy route is not advertised to a neighbor.

config conditional-advertise 

edit "My_Prefix" 

set condition-routemap "dummy_map" 

set condition-type exist  <=== Prefix is advertised when the condition exists. This is the default. 

next

end 

next 

end 

end 

 

 

5.d) Advertise the route-map for the prefix to the secondary ISP only when the ‘dummy’ route is absent from BGP network table. 

 

# config router bgp 

set as 4000 

config neighbor 

edit "20.20.20.1" 

set capability-graceful-restart enable

set soft-reconfiguration enable 

set interface "port4" 

set remote-as 2222

set route-map-out "My_Prefix"  <=== This ensures dummy route is not advertised to neighbor

config conditional-advertise 

edit "My_Prefix" 

set condition-routemap "dummy_map"

set condition-type non-exist    <=== Prefix is advertised when condition does not exist.

next 

end 

next

end 

end 

 

Confirmation 

 

When ISP_1 is up, run the following commands to confirm the configuration was successful (example output is attached):

 

Client_Router # get router info routing-table all 

Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP 

       O - OSPF, IA - OSPF inter area 

       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 

       E1 - OSPF external type 1, E2 - OSPF external type 2 

       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area 

       V - BGP VPNv4 

       * - candidate default 

 

Routing table for VRF=0 

S*      0.0.0.0/0 [10/0] via 10.10.10.1, port3, [1/0] 

                  [10/0] via 20.20.20.1, port4, [2/0] 

C       1.1.1.0/24 is directly connected, port5 

S       10.0.0.0/8 [10/0] via 10.9.15.254, port1, [1/0] 

C       10.9.0.0/20 is directly connected, port1 

C       10.9.16.0/20 is directly connected, port2 

C       10.10.10.0/30 is directly connected, port3 

C       20.20.20.0/30 is directly connected, port4 

S       169.254.0.1/32 [10/0] via 10.10.10.1, port3, [1/0] 

S       172.16.0.0/12 [10/0] via 10.9.15.254, port1, [1/0] 

S       192.168.0.0/16 [10/0] via 10.9.15.254, port1, [1/0] 

 

 

Client_Router # get router info bgp network  

VRF 0 BGP table version is 11, local router ID is 20.20.20.2 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              S Stale 

Origin codes: i - IGP, e - EGP, ? - incomplete 

  

   Network          Next Hop            Metric     LocPrf Weight RouteTag Path 

*> 0.0.0.0/0        20.20.20.1      0             150      0        0 2222 i <-/1> 

*                   10.10.10.1      0                      0        0 1111 i <-/-> 

*> 1.1.1.0/24       0.0.0.0                       100  32768        0 i <-/1> 

*> 169.254.0.1/32   0.0.0.0                       100  32768        0 i <-/1> 

  

Total number of prefixes 3  

 

Client_Router # get router info bgp summary 

 

VRF 0 BGP router identifier 20.20.20.2, local AS number 4000 

BGP table version is 9 

4 BGP AS-PATH entries 

0 BGP community entries 

 

Neighbor   V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd 

10.10.10.1 4       1111    1506    1520        9    0    0 00:40:59        1 

20.20.20.1 4       2222    1534    1548        9    0    0 00:35:48        1 

 

Total number of neighbors 2 

 

Condition route map: 

  dummy_map, state 1, use 5 

 

 

Client_Router # get router info bgp nei 10.10.10.1 advertised-routes 

VRF 0 BGP table version is 9, local router ID is 20.20.20.2 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal 

Origin codes: i - IGP, e - EGP, ? - incomplete 

 

   Network          Next Hop            Metric     LocPrf Weight RouteTag Path 

*> 1.1.1.0/24       10.10.10.2                    100  32768        0 i <-/-> 

 

Total number of prefixes 1 

 

 

Client_Router # get router info bgp nei 20.20.20.1 advertised-routes 

% No prefix for neighbor 20.20.20.1 

 

 

When ISP_1 loses connection to internet behind the client and neighborship is still active:

 

Client_Router # get router info routing-table all 

Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP 

       O - OSPF, IA - OSPF inter area 

       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 

       E1 - OSPF external type 1, E2 - OSPF external type 2 

       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area 

       V - BGP VPNv4 

       * - candidate default 

 

Routing table for VRF=0 

S*      0.0.0.0/0 [10/0] via 20.20.20.1, port4, [2/0] 

C       1.1.1.0/24 is directly connected, port5 

S       10.0.0.0/8 [10/0] via 10.9.15.254, port1, [1/0] 

C       10.9.0.0/20 is directly connected, port1 

C       10.9.16.0/20 is directly connected, port2 

C       10.10.10.0/30 is directly connected, port3 

C       20.20.20.0/30 is directly connected, port4 

S       172.16.0.0/12 [10/0] via 10.9.15.254, port1, [1/0] 

S       192.168.0.0/16 [10/0] via 10.9.15.254, port1, [1/0] 

 

 

Client_Router # get router info bgp network 

VRF 0 BGP table version is 12, local router ID is 20.20.20.2 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              S Stale 

Origin codes: i - IGP, e - EGP, ? - incomplete 

  

   Network          Next Hop            Metric     LocPrf Weight RouteTag Path 

*> 0.0.0.0/0        20.20.20.1      0             150      0        0 2222 i <-/1> 

*                   10.10.10.1      0                      0        0 1111 i <-/-> 

*> 1.1.1.0/24       0.0.0.0                       100  32768        0 i <-/1> 

  

Total number of prefixes 2  

 

Client_Router # get router info bgp summary 

 

VRF 0 BGP router identifier 20.20.20.2, local AS number 4000 

BGP table version is 10 

4 BGP AS-PATH entries 

0 BGP community entries 

 

Neighbor   V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd 

10.10.10.1 4       1111    1508    1521        9    0    0 00:42:20        1 

20.20.20.1 4       2222    1535    1550        9    0    0 00:37:09        1 

 

Total number of neighbors 2 

 

Condition route map: 

  dummy_map, state 0, use 5 

 

 

Client_Router # get router info bgp nei 10.10.10.1 advertised-routes 

% No prefix for neighbor 10.10.10.1 

 

 

Client_Router # get router info bgp nei 20.20.20.1 advertised-routes 

VRF 0 BGP table version is 10, local router ID is 20.20.20.2 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal 

Origin codes: i - IGP, e - EGP, ? - incomplete 

 

   Network          Next Hop            Metric     LocPrf Weight RouteTag Path 

*> 1.1.1.0/24       20.20.20.2                    100  32768        0 i <-/-> 

 

Total number of prefixes 1 

 

 

Client_Router # 

 

Notes: 

 

For this setup to work, it is important not to configure ‘network-import-check disable’ globally in BGP. If ‘network-import-check disable’ is being used, configure the following under the dummy route in BGP: 
 

# config router bgp 

    config network 

        edit 2 

            set prefix 169.254.0.1 255.255.255.255 

            set network-import-check enable 

        next 

    end 

end 

 

The ‘dummy’ route is only used to trigger BGP conditional routing and should not be an active route to a used/reachable IP or subnet. It is important to insert the ‘dummy’ route into the BGP network table as the FortiGate conditional advertisement is triggered by the BGP network table and not the routing table (which contains the static route). View the bgp network table using the following command: 

 

# get router info bgp network 

 

Related articles:

- https://community.fortinet.com/t5/FortiGate/Technical-Tip-Selective-route-removal-using-link-monitor... 

- https://community.fortinet.com/t5/FortiGate/Technical-Tip-BGP-conditional-advertisement-example-in-a... 

Contributors