FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
sagha
Staff
Staff
Article Id 338330
Description This article describes the Graceful restart capability with OSPF and the forwarding of traffic with FortiGate HA.
Scope FortiGate running in NAT and HA mode.
Solution

To avoid packet loss and achieve nonstop forwarding, FortiGate employs HA and graceful restart capability with OSPF.
When one of FortiGate's HA clusters has any issues or failover is taking place between FortiGates, it is possible to use the Graceful restart if there is OSPF neighborship established between FortiGate and its OSPF neighbor.


When a failover happens on the FortiGate side, FortiGate will send a packet telling the OSPF neighbor router that there is a FortiGate restart event.


Because of this, the GR-capable neighbor router is required to keep the FIB information and continue forwarding traffic for the configured graceful-restart-timer. The new Primary for FortiGate HA can use this time to bring up the OSPF neighborship in the meantime and switch to using the newly learned routes.

                                                           sagha_0-1725268857834.png
  1. The secondary FortiGate in HA receives all FIB information from the Primary FortiGate. This information is visible in the kernel table as routes with high priority.

FGT2 # get router info kernel | grep 0.9
tab=254 vf=0 scope=0 type=1 proto=18 prio=2147549185 0.0.0.0/0.0.0.0/0->10.163.0.9/32 pref=0.0.0.0 gwy=10.199.1.18 dev=41(port4)

  1. The secondary FortiGate holds onto these routes and this depends on the settings that are configured under HA. Details here

     

    config system ha
         set route-ttl 60
    end

     

    As an example: If configuring route-ttl as 60, it will hold the routes for 60 seconds after a failover on the New Primary FortiGate after failover from the old Primary FortiGate.

     

  2. With Graceful restart enabled, upon a failover, FortiGate sends an LS update packet with Graceful Restart to the OSPF neighbor.


    config router ospf
        set router-id 1.1.1.1
        set restart-mode graceful-restart  <--
        set restart-period 30 <--
    end

     

    Frame 72: 122 bytes on wire (976 bits), 122 bytes captured (976 bits)

    Ethernet II, Src: Fortinet_09:00:12 (00:09:0f:09:00:12), Dst: IPv4mcast_05 (01:00:5e:00:00:05)

    Internet Protocol Version 4, Src: 10.199.1.17, Dst: 224.0.0.5

    Open Shortest Path First

        OSPF Header

        LS Update Packet

            Number of LSAs: 1

            LSA-type 9 (Opaque LSA, Link-local scope), len 44

                .000 0000 0000 0001 = LS Age (seconds): 1

                0... .... .... .... = Do Not Age Flag: 0

                Options: 0x02, (E) External Routing

                LS Type: Opaque LSA, Link-local scope (9)

                Link State ID Opaque Type: Grace-LSA (3)

                Link State ID Opaque ID: 0

                Advertising Router: 1.1.1.1

                Sequence Number: 0x80000001

                Checksum: 0x9b59

                Length: 44

                Grace Period: 30 seconds

                    TLV Type: grace-LSA Grace Period (1)

                    TLV Length: 4

                    Grace Period: 30 seconds

                Restart Reason: Processor Switchover (3)

                    TLV Type: grace-LSA Restart Reason (2)

                    TLV Length: 1

                    Restart Reason: Processor Switchover (3)

                    Pad Bytes: 000000

                Restart IP: 10.199.1.17

                    TLV Type: grace-LSA Restart IP (3)

                    TLV Length: 4

                    Restart IP: 10.199.1.17

     

  3. On the FortiGate cluster, nothing is shown in the OSPF debugs regarding graceful restart. However, the GR-capable OSPF neighbor would see this in the debugs. In this case, another standalone FortiGate is the OSPF neighbor, the following can be seen in OSPF debugs:

     

    2024-09-02 10:07:18 OSPF: RECV[LS-Upd]: From 1.1.1.1 via port4:10.199.1.17 (10.199.1.17 -> 224.0.0.5)

    2024-09-02 10:07:18 OSPF: -----------------------------------------------------

    2024-09-02 10:07:18 OSPF: Header

    2024-09-02 10:07:18 OSPF:   Version 2

    2024-09-02 10:07:18 OSPF:   Type 4 (Link State Update)

    2024-09-02 10:07:18 OSPF:   Packet Len 72

    2024-09-02 10:07:18 OSPF:   Router ID 1.1.1.1

    2024-09-02 10:07:18 OSPF:   Area ID 0.0.0.0

    2024-09-02 10:07:18 OSPF:   Checksum 0x0

    2024-09-02 10:07:18 OSPF:   AuType 2

    2024-09-02 10:07:18 OSPF:   Cryptographic Authentication

    2024-09-02 10:07:18 OSPF:   Key ID 1

    2024-09-02 10:07:18 OSPF:   Auth Data Len 16

    2024-09-02 10:07:18 OSPF:   Sequence number 671

    2024-09-02 10:07:18 OSPF: Link State Update

    2024-09-02 10:07:18 OSPF:   # LSAs 1

    2024-09-02 10:07:18 OSPF:   LSA Header

    2024-09-02 10:07:18 OSPF:     LS age 1

    2024-09-02 10:07:18 OSPF:     Options 0x2

    2024-09-02 10:07:18 OSPF:     LS type 9 (Link-Local Opaque-LSA)

    2024-09-02 10:07:18 OSPF:     Link State ID 3.0.0.0

    2024-09-02 10:07:18 OSPF:     Advertising Router 1.1.1.1

    2024-09-02 10:07:18 OSPF:     LS sequence number 0x80000001

    2024-09-02 10:07:18 OSPF:     LS checksum 0xf90b

    2024-09-02 10:07:18 OSPF:     length 44

    2024-09-02 10:07:18 OSPF:   Grace-LSA

    2024-09-02 10:07:18 OSPF:     Grace Period: 30

    2024-09-02 10:07:18 OSPF:     Graceful Restart Reason: Switch to redundant control processor

    2024-09-02 10:07:18 OSPF:     IP interface Address: 10.199.1.17

     

     

  4. On the OSPF neighbor, it is possible to see that neighbor FortiGate is in a graceful restart period and the dead time of 30 sec configured shows instead of the original dead-interval of 4 sec for this neighbor.

    Neighbor # get router info ospf neighbor

    OSPF process 1, VRF 1:

    Neighbor ID     Pri   State           Dead Time   Address         Interface

    1.1.1.1           1   Full/Backup     00:00:30*   10.199.1.17  port4

     

     

  5. During this time, the OSPF neighbor and the new Primary FortiGate continue forwarding the packets using the FIB-learned routes through HA. This continues until the route-ttl expires.

     

     

  6. After OSPF neighborship comes up, newly learned routes will be installed in the routing-table with a lower priority

     

    FGT2 # get router info kernel | grep 163

    tab=254 vf=0 scope=0 type=1 proto=11 prio=65537 0.0.0.0/0.0.0.0/0->10.163.0.9/32 pref=0.0.0.0 gwy=10.199.1.18 dev=41(port4)

    tab=254 vf=0 scope=0 type=1 proto=20 prio=2164326401 0.0.0.0/0.0.0.0/0->10.163.0.9/32 pref=0.0.0.0 gwy=10.199.1.18 dev=41(port4)

     

     

  7. Both the routes i.e. learned via new OSPF neighborship and FIB routes synced via HA, will stay in the routing-table.

    After route-ttl expires for HA FIB routes, it will be removed from the FIB.

     

    FGT2 # get router info kernel | grep 163

    tab=254 vf=0 scope=0 type=1 proto=11 prio=65537 0.0.0.0/0.0.0.0/0->10.163.0.9/32 pref=0.0.0.0 gwy=10.199.1.18 dev=41(port4)

     

     

  8. Always configure restart-period less than the route-ttl.

    If having 'set restart-period' under 'config router ospf' configured with a higher value than 'set route-ttl' under 'config system ha', interruption of traffic will be expected as FIB routes would be removed before the graceful-restart period finishes to learn new routes.

     

Related articles: 

Technical Tip: Configuring FortiGate HA and OSPF graceful-restart to avoid traffic interruption duri...
Technical Tip: Controlling how HA synchronizes routing table updates