Hub-and-Spoke (ADVPN and OSPF) Network Scaling

[Sadhi_Jayz]

Hello Fortinet community,

We recently deployed an ADVPN‑based hub‑and‑spoke topology using FortiGate firewalls:

• Hub: FG‑601F (FortiOS 7.4.8M)
• Spokes: FG‑40F (low‑user sites - FortiOS 7.4.6,7.4.7,7.4.8)
• FG‑100F (mid‑user sites - FortiOS 7.4.6,7.4.7,7.4.8)

• Scale: ~450 total spokes
• Phase 1: ~300 spokes deployed
• Phase 2: remaining ~150 spokes deployed

At each spoke site, we have 2 or 3 ISPs, each establishing separate IPsec tunnels to the hub (via ADVPN). OSPF is used for dynamic routing across a single OSPF area.

After Phase 1, everything worked cleanly.

After Phase 2, roughly 70–90 spokes intermittently lost access to resources behind the hub, despite their ADVPN tunnels remaining UP (Including phase 1 devices).

Based on our investigation so far, we suspect an OSPF routing or neighbor issue at the hub, possibly due to the high number of neighbors (since each spoke generates multiple neighbor adjacencies to the hub).

My Key Questions:

1. Has anyone successfully deployed ADVPN + OSPF with ~450 spokes ? Any experience with scalability at this level?

2. Can an 601F reliably support OSPF neighbor count in the ~1,000‑neighbor range (e.g. each spoke having 2–3 tunnels/links)? Are there known limitations or performance impacts? (Note: We have not observed any CPU spikes or high memory utilization on the devices. Additionally, deep packet inspection is not enabled on either the hub or spoke FortiGate units.)

3. What are potential causes for only some spokes (70–90) losing reachability post-deployment, despite tunnel interfaces staying active?

Any insights, best practices, or troubleshooting tips are greatly appreciated!

Thank you in advance.

[Jean-Philippe_P]

Hello Sadhi_Jayz, 

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible. 

Thanks, 

[Jean-Philippe_P]

Hello,

We are still looking for an answer to your question.

We will come back to you ASAP.

Thanks,

[filiaks1]

Maybe see (104) Ultimate ADVPN Setup Guide: BGP & OSPF Hub and Spoke - YouTube ans try summrizing the routes and using ospf different zones.  As you have ISP probably the spokes can't get from the hub a default route when using zones if traffic needs to go through the ISP that is for internet and not through the ADVPN.

[aguerriero]

couple ways you could do this. 

Convert each spoke to an Area. totally stub or nssa. This protects the spokes from receiving all spoke routes, they only receive a default or summaries if you that is what you are sending. The hub is the only place you will need all of the spoke routes. 

Use broadcast OSPF but set the stubs to priority zero and then hubs 1 or 2 with a priority greater than 0 so they will be the only valid DR or BDR. This keeps your hub/s as DR/BDR instead of accidentally allowing a spoke to assume that function.

Use NBMA(P2MP) ospf at the hub and P2P at the spokes. The negative here is that you need to adjust hello/dead timers and manually specify neighbors. And with 400+ spokes it gets configuration intensive.

Since you are using ADVPN you are probably allowing direct spoke to spoke communication through shortcut switching, similar to cisco NHRP phase 3 DMVPN. So everything should work similar to NHRP on a cisco. You get direct spoke to spoke redirects without sharing full tables or adjacency with spokes but the hub servers as the shortcut redirect and has all routing information.

A possible reason for loss of reachability is that when a neighbor goes up or down in a network all routers in the same area have to recompute their LSDB. By segregating to areas into stubs with summary routes you keep adjacency changes from one spoke from affecting the LSDB recalculation on other spokes.