Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
freaky
New Contributor

Stateful firewall rules with redundant routing

Hi there, we set up a VPN to Amazon EC2 (VPC). This uses 2 VPN connections for redundancy (that is, we connect from our 1 IP to 2 IP' s at Amazon simultaneously). So there are 2 VPN interfaces. The routes are learned through BGP, normally only one is actively listed under route monitor, with BGP monitoring tools you can see both advertise the same route. Anyways, we use stateful firewalling to filter the traffic (in 1 direction only) by creating 2 policies: 1. Internal/Any to VPN1/Any 2. Internal/Any to VPN2/Any Very simple, nothing fancy. But it causes issues (not always, depending on how to routing goes). We had the situation where our traffic left over the VPN2 interface, but came back over the VPN1 interface. The stateful firewall doesn' t get it. Is there anything special I need to enable or something? If I allow all traffic in from VPN1/VPN2 this works oc. But we don' t really want that.
7 REPLIES 7
emnoc
Esteemed Contributor III

I just did some VPC work with a client of mine, how we got around this was to set the admin value for the BGP routes learned over one vpn-instance to be higher. This will force all traffic thru connection #1. We also played around with sending MEDs to amazon who acted accordingly. Hopes that helps.

PCNSE 

NSE 

StrongSwan  

PCNSE NSE StrongSwan
freaky
New Contributor

No clue what a MED is. How does your solution solve the problem though? The issue wasn' t with the outgoing traffic, it was the returning traffic that took the different path :). Kinda unusual probably, as it seems to prefer the 1st connection. Because I played around with vpn' s up/down outgoing traffic was taking the second connection. I wonder if this can be solved by grouping to 2 interfaces into a zone. But it requires modifying the current configuration as the interfaces can' t have firewall rules etc. if they are to be added to a zone.
freaky
New Contributor

Odd, really thought zones would solve this... :(. Created a zone of the 2 groups. Put the first connection administratively down for a sec or 2 (the route over the second connection then becomes primary) and re-enabled it (the route over connection 1 exists again, but connection 2 route remains preferred). This causes the fortigate to route out over connection 2. Amazon however prefers connection 1 and thus routes back over that. Tried to connect over SSH... epic fail. Put connection 2 down for a couple of secs (route over connection 1 preferred again), all is fine. (All is also fine if connection 1 remains turned down, so my firewall rules are ok).
emnoc
Esteemed Contributor III

How does your solution solve the problem though? The issue wasn' t with the outgoing traffic, it was the returning traffic that took the different path :). Kind of unusual probably, as it seems to prefer the 1st connection. Because I played around with vpn' s up/down outgoing traffic was taking the second connection.
If you understand BGP than it would make sense. If I was you, I would Google BGP path selection and MED " Multi Exit Discriminator" Also the AWS forum speaks about this in a few posting, but to sum it up & with trying to do with metric, in a nutshell, you " force" AWS to use one side of the VPN connection for returned traffic. This would eliminate any asymmetrical routing issues at the FW and stateful-inspection e.g custgw-vpn1-to-aws set metric lower custgw-vpn2-to-aws set metric higher This would enforce the vpn1 connection as the primary path into your AS. MED only works locally between the peering AS when you have multiple uplinks. the below document is straight from AWS on some the VPN topology and general outline http://docs.amazonwebservices.com/AmazonVPC/latest/NetworkAdminGuide/Introduction.html and this https://forums.aws.amazon.com/thread.jspa?messageID=170042頺 If you used MED, asymmetrical routing is eliminated at the expense of ALL traffic is sent via one uplink ( vpn-instance ) and not load balance across both vpn-instance. We did something similar with a big client of mine and 2 BGP routers, by weighting path selection and enforce traffic via one vpn-instance. Ideally, you would want to keep the firewall and bgp-speakers separate and 2 unique process. ( see drawing for a suggested topo ) YMMV but you have a lot of options available. I found you gain more flexibility with separation of the VPN and firewall roles into 2 distinctive layers.

PCNSE 

NSE 

StrongSwan  

PCNSE NSE StrongSwan
freaky
New Contributor

Thanks for the re' :). I' m not quite set on solving the asymmetrical routing tho' , it supposed to work that way. Was hoping that when I added the 2 interfaces to a zone, the session table would register the zone as interface, not the interface itself and thus solve the issue. But apparently it doesn' t work that way. This mainly because I also want to investigate using our second internet line for this as well, creating 4 tunnels (both wan1 and wan2 would have 2 vpn tunnels each then). Asymmetrical routing can also increase the bandwidth then.
emnoc
Esteemed Contributor III

Have you looked at placing an alternative " edge-router" between AWS/internet and FW? Most of the orgs that I' ve worked and consult on, prefer this approach. fwiw: Zones don' t necessary place all interfaces into a " single virt-interface" but rather make it easily for managing firewall policies for similar interfaces. It per se has nothing todo with the L3 routing , but more of a fwpolicies grouping and management & grouping of multiple interfaces. If you really need asymmetrical routing then you always have the ; config system settings set asymroute enable end but this breaks statefull inspection & opens you up for spoof' d traffic.

PCNSE 

NSE 

StrongSwan  

PCNSE NSE StrongSwan
james_mackinnon
New Contributor

Hi There, I am now trying to set up a S2S VPN to Amazon with limited success. We have also been sent a partial config by Amazon that does not really make sense. Would you be kind enough to share what the VPN config should look like? AWS forums have people setting up loopback interfaces and a number of people just give up. I also suspect that what Amazon have supplied may be wrong or from a very old FortiOS.
Kind Regards, James
Kind Regards, James
Labels
Top Kudoed Authors