Skip to main content
Julien87
Contributor II
September 28, 2022
Solved

AdVPN - ibgp multipath

  • September 28, 2022
  • 1 reply
  • 5772 views

Hello Community,

 

I continue my research on advpn and bgp. I would like to enable ibgp multipath so I can add SDWAN rules and split traffic based on link health for branches.

Before the multipath modification, the routes were well learned by my branches, when I activate either ISP on my HUB.

But after applying multipath configuration in HUB and Branches. Only hub routes are visible in route table database for my branches. The route are received by the bgp protocol from neighbor.

 

I can not use the new feature self-healing because i have 2 branches with 6.4 version.

 

I have the same topology as multipath link, but with two tunnel for my branches.

Source multipath : Multipath doc fortinet 

 

Thanks for your help, you can found below config and info bgp.

 

Julien

>>>Config HUB bgp

config router bgp
set as 65505
set router-id 1.1.1.1
set ibgp-multipath enable
set additional-path enable

set additional-path-select 2
config neighbor-group
edit "advpn"

set capability-default-originate enable
set link-down-failover enable
set remote-as 65505
set additional-path both

set adv-additional-path 2
set route-reflector-client enable
next
end
config neighbor-range
edit 2
set prefix 10.10.0.0 255.255.0.0
set neighbor-group "advpn"
next
end

 

>>> Branche BGP config

config router bgp
set as 65505
set router-id 1.1.1.2
set ibgp-multipath enable
set additional-path enable

set additional-path-select 2
config neighbor
edit "10.10.100.254"
set advertisement-interval 1
set link-down-failover enable
set soft-reconfiguration enable
set remote-as 65505
set additional-path both
next
edit "10.10.101.254"
set advertisement-interval 1
set link-down-failover enable
set soft-reconfiguration enable
set remote-as 65505
set additional-path both
next
end

 

 

 

>>> routing table for the hub

FortiGate-HUB # get router info routing-table bgp
Routing table for VRF=0
B 10.19.3.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 15:50:35
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 15:50:35
B 10.19.30.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 15:50:35
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 15:50:35
B 10.19.103.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 15:50:35
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 15:50:35
B 10.19.130.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 15:50:35
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 15:50:35
B 10.23.1.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 15:50:34
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 15:50:34
B 10.23.101.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 15:50:34
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 15:50:34

 

 

>> routing table for spoke

FortiGate-SPOKE# get router info routing-table bgp
Routing table for VRF=0
B 10.19.1.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel x.x.x.x), 15:51:22
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel x.x.x.x), 15:51:22
B 10.19.101.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel x.x.x.x), 15:51:22
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel x.x.x.x), 15:51:22

 

>>> Route learn from neighbor for branches

FortiGate-SPOKE# get router info bgp neighbors 10.10.100.254 received-routes
VRF 0 BGP table version is 11, local router ID is 1.1.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*>i10.19.1.0/24     10.10.100.254                 100      0        0 i <1/->
*>i10.19.3.0/24     10.10.101.2                   100      0        0 i <2/->
*>i10.19.3.0/24     10.10.100.2                   100      0        0 i <1/->
*>i10.19.30.0/24    10.10.101.25                  100      0        0 i <2/->
*>i10.19.30.0/24    10.10.100.25                  100      0        0 i <1/->
*>i10.19.101.0/24   10.10.100.254                 100      0        0 i <1/->
*>i10.19.103.0/24   10.10.101.2                   100      0        0 i <2/->
*>i10.19.103.0/24   10.10.100.2                   100      0        0 i <1/->
*>i10.19.130.0/24   10.10.101.25                  100      0        0 i <2/->
*>i10.19.130.0/24   10.10.100.25                  100      0        0 i <1/->
*>i10.23.1.0/24     10.10.101.21                  100      0        0 i <2/->
*>i10.23.101.0/24   10.10.101.21                  100      0        0 i <2/->

 

>>> Neighbors info in branches

FortiGate-SPOKE # get router info bgp neighbors 10.10.100.254
VRF 0 neighbor table:
BGP neighbor is 10.10.100.254, remote AS 65505, local AS 65505, internal link
BGP version 4, remote router ID x.x.x.x
BGP state = Established, up for 15:53:20
Last read 00:00:20, hold time is 180, keepalive interval is 60 seconds
Configured hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Address family IPv6 Unicast: advertised and received
Received 1292 messages, 3 notifications, 0 in queue
Sent 1240 messages, 1 notifications, 0 in queue
Route refresh request: received 0, sent 1
NLRI treated as withdraw: 0
Minimum time between advertisement runs is 1 seconds

For address family: IPv4 Unicast
BGP table version 11, neighbor version 10
Index 1, Offset 0, Mask 0x2
Additional Path:
Send-mode: advertised, received
Receive-mode: advertised, received
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor (both)
2 accepted prefixes, 2 prefixes in rib
2 announced prefixes

For address family: IPv6 Unicast
BGP table version 1, neighbor version 1
Index 1, Offset 0, Mask 0x2
Community attribute sent to this neighbor (both)
0 accepted prefixes, 0 prefixes in rib
0 announced prefixes

Connections established 6; dropped 5
Local host: 10.10.100.21, Local port: 21863
Foreign host: 10.10.100.254, Foreign port: 179
Egress interface: 15
Nexthop: 10.10.100.21
Nexthop interface: spk2-ph1
Nexthop global: ::j
Nexthop local: ::
BGP connection: non shared network
Last Reset: 15:53:27, due to BGP Notification received
Notification Error Message: (CeaseUnspecified Error Subcode)

Best answer by akristof

Hi,

First thing I would do is split neighbor-group for each ADVPN tunnels.

Potentially it will work like this, but it would be cleaner.

Second, can you run debug on spoke:

diag ip router bgp level info

diag ip router bgp nsm en

diag ip router bgp all en

diag debug console time en

diag debug en

 

When debug will be enabled, hard-clear one neighbor (on spoke) and wait until it will negotiate and share the full output with me (attach as file).

Then disable debug:

diag debug reset

diag debug disable

1 reply

akristof
Staff
Staff
September 28, 2022

Hi,

Can you share with me following output from spoke that has the problem as hostnames are confusing as they are same for HUB and spokes:

 

get router info routing-table all

get router info routing-table data

get router info bgp network

 

 

Julien87
Julien87Author
Contributor II
September 28, 2022

Hi Adrian

 

i have change hostname in my first post with HUB or SPOKE.

 

>>> HUB

FortiGate-HUB # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default

Routing table for VRF=0
S* 0.0.0.0/0 [1/0] via ISP1, port1
[1/0] via ISP2, port2
S 10.10.100.2/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.2
S 10.10.100.21/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.21
S 10.10.100.25/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.25
C 10.10.100.254/32 is directly connected, hubwan2-ph1
S 10.10.101.2/32 [15/0] via hub-ph1-s tunnel 10.10.101.2
S 10.10.101.21/32 [15/0] via hub-ph1-s tunnel 10.10.101.21
S 10.10.101.25/32 [15/0] via hub-ph1-s tunnel 10.10.101.25
C 10.10.101.254/32 is directly connected, hub-ph1-s
C 10.19.1.0/24 is directly connected, port3
B 10.19.3.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 17:55:14
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 17:55:14
B 10.19.30.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 17:55:14
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 17:55:14
C 10.19.101.0/24 is directly connected, port4
B 10.19.103.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 17:55:14
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 17:55:14
B 10.19.130.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 17:55:14
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 17:55:14
B 10.23.1.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 17:55:13
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 17:55:13
B 10.23.101.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 17:55:13
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 17:55:13
C ISP1/26 is directly connected, port1
C ISP2/26 is directly connected, port2

 

 

akristof
Staff
Staff
September 28, 2022

Hi,

Thanks. Can you now share with me routing-table from spoke? I want to see what is in the routing-table. I see that the routes are received from BGP, so now I need to see routing-table