Hello Community,
I continue my research on advpn and bgp. I would like to enable ibgp multipath so I can add SDWAN rules and split traffic based on link health for branches.
Before the multipath modification, the routes were well learned by my branches, when I activate either ISP on my HUB.
But after applying multipath configuration in HUB and Branches. Only hub routes are visible in route table database for my branches. The route are received by the bgp protocol from neighbor.
I can not use the new feature self-healing because i have 2 branches with 6.4 version.
I have the same topology as multipath link, but with two tunnel for my branches.
Source multipath : Multipath doc fortinet
Thanks for your help, you can found below config and info bgp.
Julien
>>>Config HUB bgp
config router bgp
set as 65505
set router-id 1.1.1.1
set ibgp-multipath enable
set additional-path enable
set additional-path-select 2
config neighbor-group
edit "advpn"
set capability-default-originate enable
set link-down-failover enable
set remote-as 65505
set additional-path both
set adv-additional-path 2
set route-reflector-client enable
next
end
config neighbor-range
edit 2
set prefix 10.10.0.0 255.255.0.0
set neighbor-group "advpn"
next
end
>>> Branche BGP config
config router bgp
set as 65505
set router-id 1.1.1.2
set ibgp-multipath enable
set additional-path enable
set additional-path-select 2
config neighbor
edit "10.10.100.254"
set advertisement-interval 1
set link-down-failover enable
set soft-reconfiguration enable
set remote-as 65505
set additional-path both
next
edit "10.10.101.254"
set advertisement-interval 1
set link-down-failover enable
set soft-reconfiguration enable
set remote-as 65505
set additional-path both
next
end
>>> routing table for the hub
FortiGate-HUB # get router info routing-table bgp
Routing table for VRF=0
B 10.19.3.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 15:50:35
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 15:50:35
B 10.19.30.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 15:50:35
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 15:50:35
B 10.19.103.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 15:50:35
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 15:50:35
B 10.19.130.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 15:50:35
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 15:50:35
B 10.23.1.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 15:50:34
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 15:50:34
B 10.23.101.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 15:50:34
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 15:50:34
>> routing table for spoke
FortiGate-SPOKE# get router info routing-table bgp
Routing table for VRF=0
B 10.19.1.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel x.x.x.x), 15:51:22
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel x.x.x.x), 15:51:22
B 10.19.101.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel x.x.x.x), 15:51:22
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel x.x.x.x), 15:51:22
>>> Route learn from neighbor for branches
FortiGate-SPOKE# get router info bgp neighbors 10.10.100.254 received-routes
VRF 0 BGP table version is 11, local router ID is 1.1.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight RouteTag Path
*>i10.19.1.0/24 10.10.100.254 100 0 0 i <1/->
*>i10.19.3.0/24 10.10.101.2 100 0 0 i <2/->
*>i10.19.3.0/24 10.10.100.2 100 0 0 i <1/->
*>i10.19.30.0/24 10.10.101.25 100 0 0 i <2/->
*>i10.19.30.0/24 10.10.100.25 100 0 0 i <1/->
*>i10.19.101.0/24 10.10.100.254 100 0 0 i <1/->
*>i10.19.103.0/24 10.10.101.2 100 0 0 i <2/->
*>i10.19.103.0/24 10.10.100.2 100 0 0 i <1/->
*>i10.19.130.0/24 10.10.101.25 100 0 0 i <2/->
*>i10.19.130.0/24 10.10.100.25 100 0 0 i <1/->
*>i10.23.1.0/24 10.10.101.21 100 0 0 i <2/->
*>i10.23.101.0/24 10.10.101.21 100 0 0 i <2/->
>>> Neighbors info in branches
FortiGate-SPOKE # get router info bgp neighbors 10.10.100.254
VRF 0 neighbor table:
BGP neighbor is 10.10.100.254, remote AS 65505, local AS 65505, internal link
BGP version 4, remote router ID x.x.x.x
BGP state = Established, up for 15:53:20
Last read 00:00:20, hold time is 180, keepalive interval is 60 seconds
Configured hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received (old and new)
Address family IPv4 Unicast: advertised and received
Address family IPv6 Unicast: advertised and received
Received 1292 messages, 3 notifications, 0 in queue
Sent 1240 messages, 1 notifications, 0 in queue
Route refresh request: received 0, sent 1
NLRI treated as withdraw: 0
Minimum time between advertisement runs is 1 seconds
For address family: IPv4 Unicast
BGP table version 11, neighbor version 10
Index 1, Offset 0, Mask 0x2
Additional Path:
Send-mode: advertised, received
Receive-mode: advertised, received
Inbound soft reconfiguration allowed
Community attribute sent to this neighbor (both)
2 accepted prefixes, 2 prefixes in rib
2 announced prefixes
For address family: IPv6 Unicast
BGP table version 1, neighbor version 1
Index 1, Offset 0, Mask 0x2
Community attribute sent to this neighbor (both)
0 accepted prefixes, 0 prefixes in rib
0 announced prefixes
Connections established 6; dropped 5
Local host: 10.10.100.21, Local port: 21863
Foreign host: 10.10.100.254, Foreign port: 179
Egress interface: 15
Nexthop: 10.10.100.21
Nexthop interface: spk2-ph1
Nexthop global: ::j
Nexthop local: ::
BGP connection: non shared network
Last Reset: 15:53:27, due to BGP Notification received
Notification Error Message: (CeaseUnspecified Error Subcode)
Solved! Go to Solution.
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Hi,
First thing I would do is split neighbor-group for each ADVPN tunnels.
Potentially it will work like this, but it would be cleaner.
Second, can you run debug on spoke:
diag ip router bgp level info
diag ip router bgp nsm en
diag ip router bgp all en
diag debug console time en
diag debug en
When debug will be enabled, hard-clear one neighbor (on spoke) and wait until it will negotiate and share the full output with me (attach as file).
Then disable debug:
diag debug reset
diag debug disable
Hi,
Can you share with me following output from spoke that has the problem as hostnames are confusing as they are same for HUB and spokes:
get router info routing-table all
get router info routing-table data
get router info bgp network
Created on 09-28-2022 02:20 AM Edited on 09-28-2022 02:56 AM
Hi Adrian
i have change hostname in my first post with HUB or SPOKE.
>>> HUB
FortiGate-HUB # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
Routing table for VRF=0
S* 0.0.0.0/0 [1/0] via ISP1, port1
[1/0] via ISP2, port2
S 10.10.100.2/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.2
S 10.10.100.21/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.21
S 10.10.100.25/32 [15/0] via hubwan2-ph1 tunnel 10.10.100.25
C 10.10.100.254/32 is directly connected, hubwan2-ph1
S 10.10.101.2/32 [15/0] via hub-ph1-s tunnel 10.10.101.2
S 10.10.101.21/32 [15/0] via hub-ph1-s tunnel 10.10.101.21
S 10.10.101.25/32 [15/0] via hub-ph1-s tunnel 10.10.101.25
C 10.10.101.254/32 is directly connected, hub-ph1-s
C 10.19.1.0/24 is directly connected, port3
B 10.19.3.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 17:55:14
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 17:55:14
B 10.19.30.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 17:55:14
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 17:55:14
C 10.19.101.0/24 is directly connected, port4
B 10.19.103.0/24 [200/0] via 10.10.100.2 (recursive via hubwan2-ph1 tunnel 10.10.100.2), 17:55:14
[200/0] via 10.10.101.2 (recursive via hub-ph1-s tunnel 10.10.101.2), 17:55:14
B 10.19.130.0/24 [200/0] via 10.10.100.25 (recursive via hubwan2-ph1 tunnel 10.10.100.25), 17:55:14
[200/0] via 10.10.101.25 (recursive via hub-ph1-s tunnel 10.10.101.25), 17:55:14
B 10.23.1.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 17:55:13
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 17:55:13
B 10.23.101.0/24 [200/0] via 10.10.100.21 (recursive via hubwan2-ph1 tunnel 10.10.100.21), 17:55:13
[200/0] via 10.10.101.21 (recursive via hub-ph1-s tunnel 10.10.101.21), 17:55:13
C ISP1/26 is directly connected, port1
C ISP2/26 is directly connected, port2
Hi,
Thanks. Can you now share with me routing-table from spoke? I want to see what is in the routing-table. I see that the routes are received from BGP, so now I need to see routing-table
Created on 09-28-2022 02:59 AM Edited on 09-28-2022 03:00 AM
Hi,
sorry, my previous message are not send correctly.
I have the same issue with 7.0.6 version.
FortiGate-SPOKE # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
Routing table for VRF=0
S* 0.0.0.0/0 [1/0] via spk2-ph1 tunnel ISP1
[1/0] via spk2-ph1s tunnel ISP2
S 10.10.100.0/24 [5/0] via spk2-ph1 tunnel ISP1
C 10.10.100.21/32 is directly connected, spk2-ph1
S 10.10.100.254/32 [15/0] via spk2-ph1 tunnel ISP1
S 10.10.101.0/24 [5/0] via spk2-ph1s tunnel ISP2
C 10.10.101.21/32 is directly connected, spk2-ph1s
S 10.10.101.254/32 [15/0] via spk2-ph1s tunnel ISP2
B 10.19.1.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel 147.78.174.134), 18:00:41
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel 147.78.174.225), 18:00:41
B 10.19.101.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel 147.78.174.134), 18:00:41
[200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel 147.78.174.225), 18:00:41
C 10.23.1.0/24 is directly connected, port2
C 10.23.101.0/24 is directly connected, port3
S 130.93.6.131/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
S 130.93.85.62/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
C 130.93.98.0/24 is directly connected, port1
S 131.93.6.131/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
S 131.93.85.62/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
C 131.93.98.0/24 is directly connected, port4
S 147.78.174.134/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
S 147.78.174.225/32 [1/0] via ISP1SPK, port1
[1/0] via ISP2SPK, port4
FortiGate-SPOKE# get router info routing-table data
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Routing table for VRF=0
S *> 0.0.0.0/0 [1/0] via spk2-ph1 tunnel ISP1
*> [1/0] via spk2-ph1s tunnel ISP2
S *> 10.10.100.0/24 [5/0] via spk2-ph1 tunnel ISP1
C *> 10.10.100.21/32 is directly connected, spk2-ph1
S *> 10.10.100.254/32 [15/0] via spk2-ph1 tunnel ISP1
S *> 10.10.101.0/24 [5/0] via spk2-ph1s tunnel ISP2
C *> 10.10.101.21/32 is directly connected, spk2-ph1s
S *> 10.10.101.254/32 [15/0] via spk2-ph1s tunnel ISP2
B *> 10.19.1.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel ISP1), 18:00:54
*> [200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel ISP2), 18:00:54
B *> 10.19.101.0/24 [200/0] via 10.10.100.254 (recursive via spk2-ph1 tunnel ISP1), 18:00:54
*> [200/0] via 10.10.101.254 (recursive via spk2-ph1s tunnel ISP2), 18:00:54
C *> 10.23.1.0/24 is directly connected, port2
C *> 10.23.101.0/24 is directly connected, port3
S *> ISP1SPK/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
S *> ISP2SPK/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
C *> ISPNETWORK/24 is directly connected, port1
S *> ISP2SPK/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
S *> ISP2SPK/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
C *> ISP2SPK/24 is directly connected, port4
S *> ISP1/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
S *> ISP2/32 [1/0] via ISP1SPK, port1
*> [1/0] via ISP2SPK, port4
FortiGate-SPOKE# get router info bgp network
VRF 0 BGP table version is 11, local router ID is 1.1.1.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight RouteTag Path
*>i10.19.1.0/24 10.10.100.254 0 100 0 0 i <1/1>
*>i 10.10.101.254 0 100 0 0 i <1/2>
*>i10.19.101.0/24 10.10.100.254 0 100 0 0 i <1/1>
*>i 10.10.101.254 0 100 0 0 i <1/2>
*> 10.23.1.0/24 0.0.0.0 100 32768 0 i <-/1>
*> 10.23.101.0/24 0.0.0.0 100 32768 0 i <-/1>
Total number of prefixes 4
Hi,
First thing I would do is split neighbor-group for each ADVPN tunnels.
Potentially it will work like this, but it would be cleaner.
Second, can you run debug on spoke:
diag ip router bgp level info
diag ip router bgp nsm en
diag ip router bgp all en
diag debug console time en
diag debug en
When debug will be enabled, hard-clear one neighbor (on spoke) and wait until it will negotiate and share the full output with me (attach as file).
Then disable debug:
diag debug reset
diag debug disable
Hi Adrian,
Thanks for your help. I have a problem with my copy/paste... i have not change router-id....
i'm confuse for this easy error.
Julien
Just last point, you can confirm this config upgrade for separate group ?
config neighbor-group
edit "advpn"
set capability-default-originate enable
set link-down-failover enable
set remote-as 65505
set additional-path both
set route-reflector-client enable
next
edit "advpn1"
set capability-default-originate enable
set link-down-failover enable
set remote-as 65505
set additional-path both
set route-reflector-client enable
next
end
config neighbor-range
edit 2
set prefix 10.10.101.0 255.255.255.0
set neighbor-group "advpn1"
next
edit 1
set prefix 10.10.100.0 255.255.255.0
set neighbor-group "advpn"
next
Hello,
Happy to help. Yes, exactly. To me this is preferred design as you can do some BGP "stuff" for each advpn group separately.
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1696 | |
1091 | |
752 | |
446 | |
228 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.