Technical Tip: How to fix the random load-balancing traffic issue on redundant active IPSEC tunnels between FortiGate and AWS VPG

JaskiratM · ‎06-08-2023

Description

This article describes how to fix the random load-balancing traffic issue on redundant active IPSEC tunnels between FortiGate and AWS VPG.

Scope

- Redundant IPSEC tunnels created between FortiGate and AWS environment. Follow the article below that explains in detail to deploy such a setup. Make sure to use dynamic routing (BGP):

Technical Tip: How to configure redundant IPSEC tu... - Fortinet Community

- The LAN network of the FortiGate is 172.16.1.0/24 and VPC subnet is 10.0.0.0/16.

- Both the Tunnel-1 and Tunnel-2 are up at the same time.

- When a packet (for example ICMP ECHO Request) is sent out via Tunnel-1 from FortiGate to an Ec2 instance, AWS uses its own random algorithm and could send the reply traffic (ICMP ECHO reply) back through Tunnel-2. In this scenario, the FortiGate will drop the traffic due to asymmetric routing.

Solution

1) Log in to the AWS management console and Navigate to VPC -> Virtual private network (VPN) -> Site-to-Site connections. Select the VPN tunnel created and download the configuration. It will show the BGP configuration and set it up according to the instructions.

2) After configuring the BGP, there should be 2 neighbors to the AWS VPG. This can be verified by ‘get router info bgp summary'.

If looking at the advertised routes to the AWS environment, it will show that the neighbor 169.254.21.89 (Tunnel-1) is advertising the route 172.16.1.0/24 with 0 metric and default AS-PATH. Also, the BGP neighbor 169.254.193.201(Tunnel-2) is advertising same the route 172.16.1.0/24 with 0 metric and default AS-PATH.

So, in order to determine the best path to 172.16.1.0/24 from AWS VPC, the AWS VPG will rely on BGP attributes to determine the best route (Tunnel-1 or Tunnel-2). It will check BGP attributes AS PATH, the lower AS path is preferred by AWS. If As PATH is the same then it will check MED (Metric) value.

3) Similarly, the FortiGate is learning 10.0.0.0/16 via Tunnel-1 and Tunnel-2.

But, AWS will automatically advertise the route through the primary tunnel with a lower Metric value of 100 and a higher Metric Value of 200 for the secondary tunnel so that the priority route is always through Tunnel-1.

4) The FortiGate is required to advertise the routes with altered BGP attributes (AS-PATH or Metric) for AWS to learn the best route through the primary tunnel.

Solution 1: AS-PATH prepend.

- Follow the article below to perform the BGP AS-PATH prepending:
https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-configure-BGP-AS-prepending/ta-p/19...

- In the above case, the following configuration needs to be added:

config router prefix-list

edit "to_AWS"

config rule

edit 1

set prefix 172.16.1.0 255.255.255.0

unset ge

unset le

config router route-map

edit "to_AWS_Tunnel_2"

config rule

edit 1

set match-ip-address "to_AWS" <- ADDED prefix to route map.
set set-aspath "65002 65002 65002"

config router bgp

set as 65000

set router-id 184.65.75.178

set ebgp-multipath enable

set ibgp-multipath enable

config neighbor

edit "169.254.21.89"

set remote-as 64512

set route-map-out "to_AWS_Tunnel_2" <- ADDED route-map out to backup tunnel.

- After making the changes, the 172.16.1.0/24 route is advertised with default AS-PATH through tunnel-1 and the same route is advertised with an AS-PATH of '65002 65002 65002' to AWS through Tunnel-2. Since a lower AS-PATH is preferred, AWS would prioritize the route through tunnel-1 and will not do the random load balance for reply traffic from AWS VPC.

Note: when choosing the prepended AS number, make sure the AS number is not the same as the remote BGP peer.

If the prepended AS number is the same, the BGP will reject the route, BGP uses AS number attribute to detect and prevent loops.

Solution 2: Metric Alteration.

- Follow the article below to perform the BGP Metric Alteration:

https://community.fortinet.com/t5/FortiGate/Technical-Note-Influencing-BGP-routes-using-Metric/ta-p/...

- In the above case, the following configuration needs to be added:

config router prefix-list

edit "to_AWS"

config rule

edit 1

set prefix 172.16.1.0 255.255.255.0

unset ge

unset le

config router route-map

edit "to_AWS_Tunnel_2"

config rule

edit 1

set match-ip-address "to_AWS" <- ADDED prefix to route map.

set set-metric 200

set as 65000

set router-id 184.65.75.178

set ebgp-multipath enable

set ibgp-multipath enable

config neighbor

edit "169.254.21.89"

set remote-as 64512

set route-map-out "to_AWS_Tunnel_2 <- ADDED route-map out the to backup tunnel.

end

- After making the changes, the 172.16.1.0/24 route is advertised with a default Metric of 0 through tunnel-1 and the same route is advertised with a Metric of 200 to AWS through Tunnel-2. Since lower METRIC is preferred, AWS would prioritize the route through tunnel-1 and will not do the random load balance from reply traffic from AWS VPG.