Description
This article describes how to fix the random load-balancing traffic issue on redundant active IPSEC tunnels between FortiGate and AWS VPG.
Scope
- Redundant IPSEC tunnels created between FortiGate and AWS environment. Follow the article below that explains in detail to deploy such a setup. Make sure to use dynamic routing (BGP): Technical Tip: How to configure redundant IPSEC tu... - Fortinet Community
- The LAN network of the FortiGate is 172.16.1.0/24 and VPC subnet is 10.0.0.0/16.
- Both the Tunnel-1 and Tunnel-2 are up at the same time.
- When a packet (for example ICMP ECHO Request) is sent out via Tunnel-1 from FortiGate to an Ec2 instance, AWS uses its own random algorithm and could send the reply traffic (ICMP ECHO reply) back through Tunnel-2. In this scenario, the FortiGate will drop the traffic due to asymmetric routing.
Solution
Log in to the AWS management console and Navigate to VPC -> Virtual private network (VPN) -> Site-to-Site connections. Select the VPN tunnel created and download the configuration. It will show the BGP configuration and set it up according to the instructions.
After configuring the BGP, there should be 2 neighbors to the AWS VPG. This can be verified by ‘get router info bgp summary'.
If looking at the advertised routes to the AWS environment, it will show that the neighbor 169.254.21.89 (Tunnel-1) is advertising the route 172.16.1.0/24 with 0 metric and default AS-PATH. Also, the BGP neighbor 169.254.193.201(Tunnel-2) is advertising same the route 172.16.1.0/24 with 0 metric and default AS-PATH.
To determine the best path to 172.16.1.0/24 from AWS VPC, the AWS VPC will rely on BGP attributes to determine the best route (Tunnel-1 or Tunnel-2). It will check BGP attributes AS PATH, the lower AS path is preferred by AWS. If As PATH is the same, then it will check the MED (Metric) value.
Similarly, the FortiGate is learning 10.0.0.0/16 via Tunnel-1 and Tunnel-2.
But, AWS will automatically advertise the route through the primary tunnel with a lower Metric value of 100 and a higher Metric Value of 200 for the secondary tunnel so that the priority route is always through Tunnel-1.
The FortiGate is required to advertise the routes with altered BGP attributes (AS-PATH or Metric) for AWS to learn the best route through the primary tunnel.
Solution 1: AS-PATH prepend.
- Follow the article below to perform the BGP AS-PATH pretending:Technical Tip: How to configure BGP AS prepending
- In the above case, the following configuration needs to be added:
config router prefix-list
edit "to_AWS"
config rule
edit 1
set prefix 172.16.1.0 255.255.255.0
unset ge
unset le
next
end
next
end
config router route-map
edit "to_AWS_Tunnel_2"
config rule
edit 1
set match-ip-address "to_AWS" <- ADDED prefix to route map.
set set-aspath "65002 65002 65002"
next
end
next
end
Note:
This 'aspath' 65002 is just a random AS, The idea is to manipulate the aspath to exhibit a higher path so that the preference given to this tunnel is lower.
config router bgp
set as 65000
set router-id 184.65.75.178
set ebgp-multipath enable
set ibgp-multipath enable
config neighbor
edit "169.254.21.89"
set remote-as 64512
next
edit "169.254.193.201"
set remote-as 64512
set route-map-out "to_AWS_Tunnel_2" <- ADDED route-map out to backup tunnel.
next
end
- After making the changes, the 172.16.1.0/24 route is advertised with default AS-PATH through tunnel-1, and the same route is advertised with an AS-PATH of '65002 65002 65002' to AWS through Tunnel-2. Since a lower AS-PATH is preferred, AWS would prioritize the route through tunnel-1 and would not do the random load balance for reply traffic from AWS VPC.
Note:
When choosing the prepended AS number, make sure the AS number is not the same as the remote BGP peer.
If the prepended AS number is the same, the BGP will reject the route, BGP uses AS number attribute to detect and prevent loops.
Solution 2: Metric Alteration.
- Follow the article below to perform the BGP Metric Alteration: Technical Tip: Influencing BGP routes using Metric
- In the above case, the following configuration needs to be added:
config router prefix-list
edit "to_AWS"
config rule
edit 1
set prefix 172.16.1.0 255.255.255.0
unset ge
unset le
next
end
next
end
config router route-map
edit "to_AWS_Tunnel_2"
config rule
edit 1
set match-ip-address "to_AWS" <- ADDED prefix to route map.
set set-metric 200
next
end
next
end
config router bgp
set as 65000
set router-id 184.65.75.178
set ebgp-multipath enable
set ibgp-multipath enable
config neighbor
edit "169.254.21.89"
set remote-as 64512
next
edit "169.254.193.201"
set remote-as 64512
set route-map-out "to_AWS_Tunnel_2 <- ADDED route-map out the to backup tunnel.
next
end
end
- After making the changes, the 172.16.1.0/24 route is advertised with a default Metric of 0 through tunnel-1 and the same route is advertised with a Metric of 200 to AWS through Tunnel-2. Since a lower METRIC is preferred, AWS would prioritize the route through tunnel-1 and would not do the random load balance from reply traffic from AWS VPG.
