BGP Established but can't ping

[HS08]

My fortigate hub have BGP connection to the spoke, the spoke ip is 10.10.112.11 and BGP was established.

But why i can't ping to that spoke bgp peer ip? So traffic from the hub can't reach the spoke using this tunnel interface

If i ping 10.10.112.1 from spoke to the hub the result is reply.

[HS08]

Did you see ping replies from the tunnel when you pinged from the spoke toward the hub? Yes

Because ADVPN is supposed to need only one manual IPsec configured between one hub and one spoke. Then ADVPN itself sets up spokes to spokes automatically.

This because hub and spoke have 2 internet connection so if we can build in full mesh it's better for redundancy. 

You have two iBGP peerings between this pairing:the hub and this spoke, right? Yes

Result of get router info bgp sum

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.10.111.1 4 65103 130847 130861 23 0 0 00:58:20 15
10.10.112.1 4 65103 94831 94814 24 0 0 00:00:40 15
10.103.113.1 4 65103 777607 765351 23 0 0 00:58:18 15
10.201.0.4 4 65515 800150 793575 22 0 0 00:04:37 1
10.201.0.5 4 65515 857558 851481 21 0 0 00:06:24 1

we can ignore neighbor 10.103.113.1 due this for MPLS, 10.201.0.4 and 10.201.0.5 is to azure.

[Toshi_Esumi]

I'm not saying having 4 tunnels is not good. But that's not what ADVPN is designed for because it's designed to make admin's work minimum to have only one manual IPsec: the hub and each spoke, then the rest:spokes to spokes, happens automatically in large scale network like more than a dozen of locations are involved. It just doesn't fit in your design. It's called "mesh" but not YOUR "mesh" between two particular FGTs.

BGP looks good on the spoke. Both peerings are receiving the same 15 routes from the hub. Only my concern is the peering with the second HUB neighbor:10.10.112.1 has established 40 seconds ago when you get this. Is it staying up? You just need to keep running the same command time to time to see the duration keeps incrementing without restarting from 00:00:00.

Then check the same at the hub to see the BGP states with this location's neighbors. Are they staying up and receiving the same number of routes?

Toshi

[HS08]

Hi,

The reason why we see only 40 second is I shutdown the tunnel interface if i'm not troubleshoot this. Let this tunnel up will make connection from hub subnet to the spoke subnet will be not established. So i shutdown to let bgp send the traffic using another path.

[Toshi_Esumi]

Then, if you take 3 tunnels down out of those 4, I bet everything would work fine as you intended.

Toshi

[HS08]

Trying only let tunnel 10.10.112.6 up and let the 3 tunnels down, but still the traffic can't passing via this tunnel. Ping also is not working.

[Toshi_Esumi]

Check BGP state/routes on the hub.

[HS08]

The BGP was established and routing to teh spoke from hub via 10.10.112.6

B 10.4.0.0/16 [200/0] via 10.10.112.6 (recursive is directly connected, DC-SEG-BALI), 00:01:30, [1/0]
[200/0] via 10.103.113.6 (recursive is directly connected, DC-SEG-MPLS), 00:01:30, [1/0]

[Toshi_Esumi]

So the same route is coming over MPLS with iBGP as well, as a parallel route to one of IPsec paths.

I'll give up. It's been a week not much traction to figure out the entire picture and find any possible root causes for multiple symptoms via limited information through multiple threads.
It's time for you to call in TAC and get your environment to be looked at via a remote session.

Toshi

[dingjerry_FTNT]

Hi @HS08 ,

Please then run the debug flow commands on the Hub:

diag debug flow show iprope enable

diag debug flow filter clear

diag debug flow filter proto 1

diag debug flow filter addr 10.10.112.11

diag debug flow trace start 10

diag debug enable

Then initiate Ping, please do not run continuous Ping.

[HS08]

hi @dingjerry_FTNT 

Here i enable the debug on the hub, then i open a new console and ping to 10.10.112.11 but there is no message log.