Technical Tip: How to troubleshoot OSPF neighborship in various states

[gbamania]

Description	This article describes how to troubleshoot OSPF neighborship between firewalls/routers.
Scope	FortiGate.
Solution	Topology: 10.10.10.1 --> Router id: 1.1.1.1. 10.10.10.2 --> Router id: 2.2.2.2. Init to 2-way state: In the 'Init' state, the firewall has sent the OSPF 'hello' packet but has not received the 'hello' packet from the neighbor firewall. Whenever a router/firewall sees its router ID in a neighbor hello packet, it confirms bidirectional communication and transitions the neighbor state to two-way. In the snippet above, when the neighboring firewall/router receives the OSPF hello from 10.10.10.1, it enters the init state. It will stay in the init state until it receives the following type of hello packet from 10.10.10.1: In the above snippet, an active neighbor router-id is added and sent over a multicast address. Once its neighbor 2.2.2.2 receives this hello packet, it will change the state from 'init' to a 2-way state. To add the active neighbor in the 'hello' packet, the following parameters are checked: Same Area. Same Layer3 broadcast network. Subnet mask. Hello and dead time intervals. Authentication. Matching stub flags. 2-way to ExStart/Exhange/ Loading state: The 2-way state means a bi-directional communication has been established between the routers. At this point, the routers perform DR and BDR elections. Note: DR and BDR election happens for each segment. Once DR and BDR are elected, both routers consider them as 'Master' and set the flag to 'Y'. MTU values are also exchanged between them at this stage. Keep a note of the MTU. Once the master/slave relationship is established. The router with the highest router ID becomes the master and the next highest is selected as slave device. The DBD (Database Descriptor) packets are exchanged in the exchange state. The DBD packets are link state advertisements and describe the entire Link state database contest.  A router attempts to form a full adjacency with a neighbor if one of the two routers is the DR or BDR. OSPF routers become fully adjacent to routers with which they have completed the database synchronization process. This is the process by which OSPF routers exchange link-state information to populate their databases with the same information. Again, this database synchronization process is only executed between two routers if one of the two routers is the DR or BDR. MTU needs to be matched, otherwise, the neighbor-ship is stuck in the Exstart state. Note: Sometimes, MTU does match on both sides but still, the neighbor-ship is stuck in the Exstart state because DBD/LSU packets are expiring and they are unable to reach the other side. This could be caused by an L2 switch or L2-WAN device (MPLS) which is placed between OSPF routers and does not forward packets at this MTU. FortiGate can only send and receive 5 Database Descriptors (DD) packets at a given point of time in a queue for all interfaces trying to form a neighbor-ship. If multiple OSPF interfaces are negotiating at the same time, it will take some time to form the neighbor-ship between routers. Note: If OSPF interfaces are stuck in ExStart/Exchange state for some reason and the OSPF process queue is 5/5, then other OSPF interfaces will not be able to form neighbor-ship to 'Full' as well. Some user commands: diag sniffer packet any “proto 89” 6 0 l diag sniffer packet any “ host 224.0.0.5” 6 0 l get router info ospf neighbor get router info ospf interface In case of configuration with more OSPF peers, to filter the sniffer for specific peers, commands below can be used : diag sniffer packet any "(host 10.10.10.1  and proto 89) or (proto 89 and host 10.10.10.2)" 6 0 l This sniffer will capture all the traffic from 10.10.10.1 and protocol number 89 or traffic from 10.10.10.2 and protocol number 89. For OSPF always the source is unicast address, and the destination is multicast. The OSPF debug can show error messages as well if the adjacency is not forming.  diagnose debug disable diagnose debug reset diagnose ip router ospf all enable / disable diagnose ip router ospf level info diagnose debug console timestamp enable  diagnose debug enable The sample debug output below shows that the OSPF Hello received on the 'ospftest' interface did not contain the correct OSPF area ID. OSPF: RECV[Hello]: From 10.90.32.1 via ospftest:10.90.32.2: Invalid Area ID 0.0.0.0  OSPF: IFSM[ospftest:10.90.32.2]: Hello timer expire  OSPF: SEND[Hello]: To 224.0.0.5 via ospftest:10.90.32.2, length 44  OSPF: -----------------------------------------------------  OSPF: Header  OSPF:   Version 2  OSPF:   Type 1 (Hello)  OSPF:   Packet Len 44  OSPF:   Router ID 10.90.32.2  OSPF:   Area ID 0.0.0.1  OSPF:   Checksum 0xd142  OSPF:   AuType 0  OSPF: Hello  OSPF:   NetworkMask 255.255.255.255  OSPF:   HelloInterval 10  OSPF:   Options 0x2 (*|-|-|-|-|-|E|-)  OSPF:   RtrPriority 1  OSPF:   RtrDeadInterval 40  OSPF:   DRouter 0.0.0.0  OSPF:   BDRouter 0.0.0.0  OSPF:   # Neighbors 0  OSPF: -----------------------------------------------------