|
The spoke has two WAN interfaces (and two IPsec tunnels). When the BGP is negotiated on the primary IPsec (on wan1) and disable the wan1 interfaces. After the DPD timeout, the tunnel goes down. The routing table is properly updated, but the BGP is still sending messages out over the primary IPsec tunnel (the one that is down at that moment).
The routing table when WAN1 is down (on the spoke):
Routing table for VRF=0
S > 0.0.0.0/0 [10/0] via y.y.y.y, wan1 inactive, [1/0]
*> [10/0] via 192.168.8.1, wan2, [5/0]
S 10.0.0.0/8 [255/0] is a summary, Null, [1/0]
S 172.16.0.0/12 [255/0] is a summary, Null, [1/0]
S *> 172.18.0.0/24 [5/0] via HUB_AWS_12 tunnel 10.0.0.1, [1/0]
> [5/0] via HUB_AWS_11 tunnel x.x.x.x inactive, [1/0]
C *> 172.18.0.0/32 is directly connected, LOOPBACK
S *> 172.18.0.254/32 [15/0] via HUB_AWS_12 tunnel 10.0.0.1, [1/0]
C *> 172.20.0.0/23 is directly connected, internal1
S 192.168.0.0/16 [255/0] is a summary, Null, [1/0]
C *> 192.168.8.0/24 is directly connected, wan2
The sniffer that shows the BGP packets on the spoke:
FortiGate # diagnose sniffer packet any 'host 172.18.0.0 and tcp port 179' 4a
interfaces=[any]
filters=[host 172.18.0.0 and tcp port 179]
1.465596 HUB_AWS_11 out 172.18.0.0.24559 -> 172.18.0.254.179: syn 1670287936
1.465744 HUB_AWS_11 out 172.18.0.0.21293 -> 172.18.0.254.179: syn 3892762205
2.465591 HUB_AWS_11 out 172.18.0.0.21293 -> 172.18.0.254.179: syn 3892762205
2.465744 HUB_AWS_11 out 172.18.0.0.22525 -> 172.18.0.254.179: syn 1012724640
To stop the sniffer, press Ctrl + C.
- HUB_AWS_11 is the primary tunnel on WAN1
- HUB_AWS_12 is the secondary tunnel on WAN2
As per the sniffer, it is observed that even when the WAN1 interface is down, BGP packets are being sent over the primary tunnel, which leads to flapping of BGP.
Upon checking the kernel routes, it is observed that a route with the same source/destination is present, which is being used for health check:
tab=254 vf=0 scope=0 type=1 proto=17 prio=0 172.18.0.2/255.255.255.255/0->172.18.0.254/32 pref=0.0.0.0 gwy=x.x.x.x dev=27(HUB_AWS_11)
Solution:
BGP packets will follow the same route and be sent to HUB_AWS_11, as they match the destination and source. Hence, it is important to make sure the health check is using a different source or destination, so it will not affect BGP packets.
|
