FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Xav_FTNT
Staff
Staff

Purpose


The article describes the configuration that needs to be applied to a FortiGate HA cluster and the BGP settings so that each router (the FortiGate and its peer(s)) will keep the BGP routes in their routing table(s) to avoid traffic interruption during an HA failover.

Scope

  • All FortiOS
  • FortiGate running in NAT and HA mode


Expectations, Requirements


HA cluster with one or more BGP peers will failover without traffic interruption.

Configuration


On a FortiGate HA cluster, the BGP router daemon process is only running on the Primary (Master) unit. When there is an HA failover a new BGP process will be launched on the newly elected master.

Even though the FortiGate has all the routes, if the peer sees the FortiGate as unresponsive, it will remove all the route from its routing table and traffic will be interrupted :

Therefore what needs to be done to avoid traffic interruption  can be divided in three parts, which are detailed later :

  •  Check that remote peer will not delete the routes
  •  Check that the Fortigate cluster will keep the BGP routes in the routing table.
  •  Fine tune timers

 

1) Check that remote peer will not delete the routes

This can be achieved with BGP graceful restart. 'Graceful Restart' is a BGP capability. It is an internet standard defined in RFC 4724. This capability needs to be configured on both peers.

In order to be effective on  FortiGate, Graceful Restart needs to be enabled at both :
- Global configuration level
- Peer level

Configuration snapshot :


# config router bgp
    set as 65111
    set graceful-restart enable
        config neighbor
            edit "10.2.3.4"
                set capability-graceful-restart enable
                set remote-as 65000
                set weight 20
            next
        end
        config network
            edit 1
                set prefix 172.31.0.0 255.255.0.0
            next
        end
end



2) Check that the FortiGate cluster will keep the BGP route in the routing table

When the FortiGate is configured in an HA cluster, all the routes will be synchronized to the slave devices. The synchronized routes on the slave will have a limited lifetime and a lower priority.
The lifetime of these routes can be configured through the 'route-ttl' parameter in system ha configuration :

# config system ha
    set route-ttl 30
end


Default value is 10.

3) Fine tuning timers


There are three main timers that can be tuned :

holdtime-timer (default 180) : Number of seconds to mark peer as dead
This is the number of seconds to wait between keepalive, update or notification message before considering the connection to the peer as closed.

graceful-restart-time(default 120): Time needed for neighbours to restart(sec)
This is the number of seconds to  wait for the OPEN message before removing the stale routes
graceful-restart-time should be less or equal to the holdtime-timer

graceful-stalepath-time : Time to hold stale paths of restarting neighbour(sec)
Total maximum time that a stale route should be kept before being deleted.


Consider tuning  these counters with the two following criteria :

- Time you want to detect a real BGP peer failure
- Maximum time allowed for a restart time

Note.

When graceful-restart is enabled it will delay the time at which a real network/peer failure will be detected, and as a consequence this will end up in a down time that can be as long as the graceful-restart-time
Therefore it is important that those timers be configured to a value that suits to the network requirements.


Verification
In output of  the CLI commands :

    # FGT# get router info bgp neighbor a.b.c.d

Check the graceful restart capabilities :

For address family: IPv4 Unicast
  BGP table version 1, neighbor version 0
  Index 1, Offset 0, Mask 0x2
  AF-dependant capabilities:
    Graceful restart: advertised, received


 

Check timers in the CLI command  :

    # FGT# get router info bgp neighbor a.b.c.d


Troubleshooting
Check other BGP related information with :

    # FGT# get router info bgp neighbor
    FGT# get router info bgp summary
    FGT# get router info bgp network
    FGT# get router info routing-table database

Contributors