Technical Tip: BGP is flapping

Description

This article provides a scenario where there is a BGP setup between 2 devices. One or both FortiGates BGP is flapping up and down.

Scope

FortiGate.

Solution

A common cause of this is ISP connectivity or packet loss.

Step 1. Check connectivity by pinging the neighbor. The interface with IP address that is being used to form BGP relationships must be reachable. This can be either a physical or logical interface. 

Step 2. Check the current BGP neighbor adjacency states. 

get router info bgp summary

Step 3. Sniff the packets and check the flow and event log.

diagnose sniffer packet <connected port or any> “host x.x.x.x and port 179” 6 0 l 

Where x.x.x.x is the remote neighbor address.

Debug flow:

diagnose debug flow filter dport 179
diagnose debug flow filter addr <neighbor ip>
diagnose debug flow show iprope enable
diagnose debug en
diagnose debug flow trace start

Note:

Starting from v7.2.0+, it is possible to collect BGP debugs for a specific neighbor by using the filter command 'diagnose ip router bgp set-filter neighbor <neighbor address>'. For more details, see Technical Tip: Capture BGP debugs for a specific neighbor.

Check the event log to see if the port is going up and going down.

If the issue is not solved by verifying, connectivity is working with no loss.

Run BGP debug:

diagnose ip router bgp all enable
diagnose ip router bgp level info 
diagnose debug en
diagnose debug console timestamp enable

Here is an example of BGP debug showing the symptoms:

BGP: <neighbor ip>-Outgoing [NETWORK] FD=26, Sock Status: 110-Connection timed out
BGP: <neighbor ip>-Outgoing [FSM] State: Connect Event: 18

In this example, FortiGate was not receiving a reply from a neighbor to establish a BGP session, so it will try again until it receives the neighbor's BGP protocol packet.

MTU should also be consistent in the path. Though MTU is not a requirement for BGP neighborship to come up, when the NLRI is exchanged in the BGP update packets, it will try to send a packet as large as the MTU of the outgoing interface. In the path, if fragmentation is not allowed and a lower MTU is present in the path, these packets will get dropped. Consequently, the BGP neighborship will flap every 180 seconds or after the time configured in the hold-down timer.

Notes:

• High CPU usage can also disrupt BGP processing, as a device processing a huge amount of traffic might fail to process data on time.

• BGP flapping could also be caused by an IP conflict caused by other devices sharing the same IP address as the BGP neighbor. This is observed during ARP response, in which there is more than one device replying with an ARP reply.

Related articles:

Technical Tip: BGP Neighbor Adjacency States

Technical Tip: FortiOS BGP Resource List