OSPF area problem, requesting debug guidance

[journeyman]

We have two Fortigates of interest FGT1 and FGT2 participating in both area 0 and area 12.

FGT1 receives a route from FGT3 via FGT2 via area 0 (as expected).

But FGT2 receives a route from FGT1 via area 12 (not expected).

This creates asymmetric routing. Any guidance on debugging would be appreciated.

Area 0 is a managed network, we do not control the Cisco routers. There is nothing in the wan1 cloud other than interconnects.

FGT1 at Site 1 originates a VM subnet /30. FGT1 participates in both area 0 and area 12.

FGT2 at Site 2 also participates in both area 0 and area 12.

FGT2 at Site 2 routing table shows the /30 via area 12. My expectation is the route would be learned via area 0.

FGT3 originates a /24 route that appears at FGT1 via area 0 as expected.

FGT4 is only on area 0 and receives the /30 route via area 0 as expected.

The core issue here is that we have asymmetric routing at FGT2.

I understand that intra-area OSPF traffic is preferred over inter-area traffic. However, the /30 route originates at FGT1 and as far as I can see is not in an area (yet?). LANs connected at FGT1 are redistributed with metric = 600.

FGT1 and FGT2 are both ABR and ASBR
FGT3 is neither ABR nor ASBR
FGT4 is ASBR but not ABR
ASBR is not relevant to this problem, mentioned for completeness.

How can I diagnose the issue to see the reason FGT2 does not prefer the route via area 0? I have had a go using get routing info ospf commands but not found what I need.

Many thanks in advance

simplified network

[Carson_Daniels]

Verify OSPF Database

Check the LSDB (Link-State Database) on FGT2 to see how the /30 route is advertised in each area:

get router info ospf database

Look for the Type-1 LSAs (Router LSAs) and Type-3 LSAs (Summary LSAs).

Confirm which area is advertising the /30 subnet and with what metric.

Goal: Determine whether FGT2 sees the /30 route from area 0, area 12, or both.

2. Check OSPF Route Preferences

OSPF prefers intra-area over inter-area. But summary LSAs from ABRs can override preference if the metric is lower.

get router info ospf route

Look at metric and source area of the /30 route.

Check if the area 0 route has a higher cost than the area 12 route.

3. Compare Metrics

FGT1 redistributes the /30 route with metric = 600.

If the route via area 12 ends up with a lower metric (less than 600), FGT2 will prefer it.

Use:

diagnose debug enable
diagnose debug ospf log

Then trigger a route advertisement or check the logs to see LSA metric calculation.

4. Confirm ABR Behavior

Since FGT1 and FGT2 are ABRs, check if the /30 route is being advertised into area 12 by FGT1. Sometimes an ABR will inject routes into a non-originating area with a lower metric than intra-area routes.

Check ABR summary LSA injection:

get router info ospf abr

Look for summary LSAs of the /30 subnet.

5. Optional: OSPF Route Filtering

If you want FGT2 to prefer area 0:

Consider OSPF route filtering on FGT1 to prevent advertising the /30 into area 12.

Or adjust metric of inter-area injection to ensure intra-area is always preferred.

[Yurisk]

Statically analyzing OSPF topologies bad for the health :), just run real time debug on the FGT2 and see what it learns from whom and why. If it is not a production environment you may need to clear OSPF routing process to speed up the process (this will delete all OSPF routes).

diagnose ip router ospf level info

diagnose ip router ospf all

dia deb enable

Some other options for dia ip router ospf:

FGT2 # diagnose ip router ospf
all       Enable all debugging. [Take 0-1 arg(s)]
lsa       OSPF Link State Advertisement. [Take 0-2 arg(s)]
nfsm      OSPF Neighbor State Machine. [Take 0-2 arg(s)]
packet    OSPF Packets. [Take 0-2 arg(s)]
events    OSPF events. [Take 0-2 arg(s)]
ifsm      OSPF Interface State Machine. [Take 0-2 arg(s)]
nfsm      OSPF Neighbor State Machine. [Take 0-2 arg(s)]
nsm       OSPF NSM interface. [Take 0-2 arg(s)]
route     OSPF route information. [Take 0-2 arg(s)]
show      Show status of OSPF debugging.
level     Debug level.

[journeyman]

Haha 'bad for the health' - I agree. Thank you for the replies so far.

Investigations continue. This is a production environment so I need to be careful, but I can change the /30 route any time. The example is simplified, there are a more areas and many more fortigates running as routers. Each "Cisco" is actually a pair of routers for diversity purposes. I did come across a post that made me wish we had implemented iBGP but anyway.

When I ran ospf debug I ended up with ~30MB of logs very quickly and found it difficult to analyse as the logs are not timestamped. Next I ran packet captures on relevant interfaces and converted to pcap. This was much easier to analyse.

Learnings:

- FGT1 advertises the change of route into both area 0 and 12

- Cisco routers adjacent to FGT1 send LS Acknowledge to 224.0.0.6 ie to the DR. I would have expected one of the Ciscos to be Designated Router, but they have priority 1 and so the FGTs become DR by virtue of having a higher IP address on the subnet. lol.

- FGT2 learns of the route change from FGT1 via area 12 but does not learn the change via the Ciscos on area 0. I find this odd.

- I have read-only access to the Cisco configs, their configuration is very vanilla.

- From doco, fortigate connected routes are redistributed as External Type 2 by default.

- External Type 2 uses only the external metric and ignores the OSPF path cost to determine route preference.

- Connected routes can be redistributed as External Type 1, which sums ospf path cost and external metric.

It seems odd to me that connected subnets are redistributed as E2 "far from the AS". It's connected, not far. I am pondering the pros and cons between redistributed connected routes vs a native OSPF routes (which would then be assigned an area).

I am not sure if the FGT being DR is of any relevance to the problem. I would have to engage managed services to promote the Cisco priority, unless I demote the FGTs to 0, probably unwise.

I still don't know why the Ciscos adjacent to FGT2 do not advertise the route. Anyone?

I plan to trial changing the redistribute connected to external type 1.

I am also considering adding the routes to OSPF instead (making them internal), rather than redistributing them into OSPF.

Your comments and suggestions are most welcome.