BGP Flapping / Carrier issue

[RJ1]

I am running a SDWAN infrastructure with one Hub and several spoke (15 spokes). All sites are equipped with FGT  clusters running 7.2.x. we have ADVPN configured with iBGP running over it for route advertisement .On 4-5 spoke sites  I have a strange behavior that is every 2-3 days   have this error in router events logs as below:

BGP: %BGP-5-ADJCHANGE: VRF 0 neighbor 192.168.88.254 Down Hold Timer Expired

BGP: %BGP-3-NOTIFICATION: sending to 192.168.88.254 4/0 (Hold Timer Expired/Unspecified Error Subcode) 0 data-bytes []

BGP: %BGP-5-ADJCHANGE: VRF 0 neighbor 192.168.88.254 Down BGP Notification FSM-ERR

BGP: %BGP-5-ADJCHANGE: VRF 0 neighbor192.168.88.254 Up

The problem is that during this event all connections from spoke to hub are shut during roughly 60 or 120 sec

[Toshi_Esumi]

I simply googled with one of messages and found this. Nothing to do with ADVPN though. But it was inconclusive.
https://community.fortinet.com/t5/Support-Forum/BGP-instability-on-IPSEC-tunnel/m-p/65094

Toshi

[RJ1]

Yes I have gone thru this but as you said it was inconclusive

[Toshi_Esumi]

Because the OP didn't update after my comment.
I also found a KB below in another google search.
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-BGP-Hold-Timer-Expired-Unspecified-E...

Toshi

[Toshi_Esumi]

But ultimately you need to run a packet capture on TCP 179 on both remote and local sides to capture the moment when that happens. That would tell exactly what was sent but what was not received on the other side in the BGP communication.

Toshi

[RJ1]

It's difficult to perform live packet capture as the incident occurs anytime of the day and gets resolved within 2-3 minutes.