Dial-up IPsec tunnels with same source subnet - unexpected server routing

[themanyandonlyglenn]

I have a FortiGate configured with two tunnels on two Ethernet ports with the intention to do load balancing or traffic steering on them. They go through a router to converge onto one port/IP at another FortiGate (a.k.a. server). The server is configured with one dynamic tunnel, and I left the dst-subnet in phase2 empty.

I can connect to the server fine, and the server "spawns" logical tunnel interfaces _0 and _1 for each dial-up. However when I ping from client to server or initiate any TCP connections, the responses all come back on _1 even if the origin is from tunnel _0. When I look at the FortiView Sessions list in GUI it just shows the session from the parent tunnel name. The Debug Flow shows the origin point of the session was from the right tunnel but it chooses to output on the other tunnel. It is as if it is using the routing table to pick the interface, not the origin tunnel, and I don't know what makes it pick _1 all the time either.

Is this a misconfiguration on my part or FG is not designed to handle dial-ups from the same source? I had to "set allow-overlap allow" to even allow both tunnels to be up or FG server deletes the old tunnel in favor of new one.

[mpatel]

Okay so if you will have 2 separate ISP on client side each with its own Public IP you will not need any local ID and you will not see the behavior you are seeing in your test. The server FGT will see the tunnels coming from different Public IP addresses. Can you confirm if this is the case.

If no, then check your config you are using same local id in both tunnels versus the document i send you wherein they used dialup1 and dialup2 as different localIDs.

The routing entries you provided are from the Server FGT and not the client FGT. Kindly provide the routing table entries on client FGT as that's where you decide which tunnel to use to send traffic over to remote side.

[themanyandonlyglenn]

The routing table I sent you is on client. I did not give you anything from server side (the 60F box). The tunnels connect to server from the public IPs of the client WAN ports. Everything in the server side debug flow appears to show it recognizes these as distinct paths/tunnels. It is just making the strange decision to output a response packet on the wrong tunnel.

[themanyandonlyglenn]

This is the server side routing table

[mpatel]

I see the confusion now. The server FGT is the 60F hosting 192.168.4.0/24 subnet.

The client is hosting 192.168.6.0/24 subnet. The src-subnet and dst-subnet in the IPSEC tunnels on your Client FGT are swapped.

The src-subnet on both tunnels needs to be 192.168.6.0/24 and dst-subnet needs to be 192.168.4.0/24.

[mpatel]

Also dont forget to swap the subnets on the Server FGT.

The Ipsec tunnel on the Server needs to have src-subnet 192.168.4.0/24 and dst-subnet as 192.168.4.0/24.

[themanyandonlyglenn]

Sorry to confuse you that was my bad when I hand-typed the config while reading it off a screen, I couldn't electronically copy it easily. I fixed the post, my FG config was correct.

This is the server side phase2:

edit "con1"

set phase1name con1

set proposal null-sha256

set route-overlap allow

set src-subnet 192.168.4.0 255.255.255.0

next

[mpatel]

Great. Lets see what you get when you modify the client FGT IPsec config.

[themanyandonlyglenn]

I do not need to change the config, it was a typo what I put here.

[mpatel]

If your config is as I have mentioned then this behavior is due to default ECMP (source-ip based) and is expected.

https://docs.fortinet.com/document/fortigate/6.2.16/cookbook/25967/equal-cost-multi-path.

https://community.fortinet.com/tpykb84852/attachments/tpykb84852/TKB20/3662/1/ECMP%20and%20Asymmetri...

You will need to prioritize the routes using same distance but different priorities to ensure FGT routes properly. 

[Toshi_Esumi]

@mpatelso you/these docs are saying if ECPM routes for a detination subnet is set [1) same cost+same priority, or 2)same cost + different priorities] is set, the FGT allows originating direction packets go out one interface and returning direction packets come back in the second interface is NOT treated as asymmetric routing and doesn't drop the returned packets.

I've learned something new today.

Toshi