New Member

Solved

Dial-up IPsec tunnels with same source subnet - unexpected server routing

Forum|Forum|2 years ago
April 16, 2024
9 replies
9430 views

I have a FortiGate configured with two tunnels on two Ethernet ports with the intention to do load balancing or traffic steering on them. They go through a router to converge onto one port/IP at another FortiGate (a.k.a. server). The server is configured with one dynamic tunnel, and I left the dst-subnet in phase2 empty.

I can connect to the server fine, and the server "spawns" logical tunnel interfaces _0 and _1 for each dial-up. However when I ping from client to server or initiate any TCP connections, the responses all come back on _1 even if the origin is from tunnel _0. When I look at the FortiView Sessions list in GUI it just shows the session from the parent tunnel name. The Debug Flow shows the origin point of the session was from the right tunnel but it chooses to output on the other tunnel. It is as if it is using the routing table to pick the interface, not the origin tunnel, and I don't know what makes it pick _1 all the time either.

Is this a misconfiguration on my part or FG is not designed to handle dial-ups from the same source? I had to "set allow-overlap allow" to even allow both tunnels to be up or FG server deletes the old tunnel in favor of new one.

FortiGate

Best answer by mpatel

Intrigued!!

Can you configure "set netdevice enable" on the server end FGT IPSEC tunnel Con1. This will cause the fortigate to create a virtual interface for all incoming IPSEC connections. Keep ecmp mode as source-dst-ip-based and test.

mpatel

Staff

Hi,

The two tunnels you have on the client FGT are they both WAN ports with different ISP circuits? (I assume not). The below link explain the behavior you are seeing.

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Use-of-PeerID-and-LocalID-in-IPsec-VPN-between-two/ta-p/196761

You will have to use peer ID in the dial up tunnel config.

https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-use-Peer-IDs-to-select-an-IPSec-dialup/ta-p/192292

Configure 2 tunnels using the above KB.

Then add these tunnels into SDWAN and you will be able to steer traffic or perform load balancing.

The allow-overlap config is used when both ends of tunnel have overlapping subnets that need to communicate with each other.

Hope you find this helpful.

Toshi_Esumi

SuperUser

Are you saying it looks like this? To me it's almost impossible unless you enabled asymmetric routing on the server_FGT. Did you confirm it by sniffing at server_FGT or client_FGT? You must have two separate phase1/phase2 config sets on the client_FGT side.

Toshi

themanyandonlyglennAuthor

New Member

Yes that looks about right. Each tunnel from client has the same phase2 subnets.

mpatel

Staff

Kindly attach the IPsec configs for both tunnels on the Client FGT.

Also send the output of the command “get router info routing-table all” and get router info routing-table database” on the client FGT.

Also how are you deciding which tunnel the client will use to reach remote in case the phase 2 selectors are 0.0.0.0/0.0.0.0 on both tunnels.

themanyandonlyglennAuthor

New Member

This is the client side IPsec config

config vpn ipsec phase1-interface

edit "dialup1"

set interface "port9"

set mode aggressive

set peertype any

set net-device disable

set proposal aes128-sha256 etc...

set localid "abc"

set remote-gw 192.168.5.2

set psksecret redacted

set interface "port12"

set mode aggressive

set peertype any

set net-device disable

set proposal aes128-sha256 etc...

set localid "abc"

set remote-gw 192.168.5.2

set psksecret redacted

config vpn ipsec phase2-interface

edit "dialup1"

set phase1name dialup1

set proposal null-sha256

set auto-negotiate enable

set src-subnet 192.168.6.0 255.255.255.0

set dst-subnet 192.168.4.0 255.255.255.0

set phase1name dialup2

set proposal null-sha256

set auto-negotiate enable

set src-subnet 192.168.6.0 255.255.255.0

set dst-subnet 192.168.4.0 255.255.255.0

end

The router table relevant rows are here, I have to hand-type it in so I'm not typing the whole thing:

S 192.168.4.0/24 [2/0] via dialup1 tunnel 192.168.5.2, [1/50]

S [2/0] via dialup2 tunnel 10.0.0.1, [1/50]
S 192.168.5.2/32 [2/0] via 192.168.11.2, port9, [1/50]

[2/0] via 192.168.12.2, port12, [1/50]

C 192.168.6.0/24 is directly connected, port8

C 192.168.11.0/24 is directly connected, port9

C 192.168.12.0/24 is directly connected, port12

I am doing ECMP with the interface weights for dialup1 and dialup2 at 50 each.

mpatel

Staff

Okay so if you will have 2 separate ISP on client side each with its own Public IP you will not need any local ID and you will not see the behavior you are seeing in your test. The server FGT will see the tunnels coming from different Public IP addresses. Can you confirm if this is the case.

If no, then check your config you are using same local id in both tunnels versus the document i send you wherein they used dialup1 and dialup2 as different localIDs.

The routing entries you provided are from the Server FGT and not the client FGT. Kindly provide the routing table entries on client FGT as that's where you decide which tunnel to use to send traffic over to remote side.

themanyandonlyglennAuthor

New Member

This is the server side routing table

mpatel

Staff

I see the confusion now. The server FGT is the 60F hosting 192.168.4.0/24 subnet.

The client is hosting 192.168.6.0/24 subnet. The src-subnet and dst-subnet in the IPSEC tunnels on your Client FGT are swapped.

The src-subnet on both tunnels needs to be 192.168.6.0/24 and dst-subnet needs to be 192.168.4.0/24.

mpatel

Staff

Also dont forget to swap the subnets on the Server FGT.

The Ipsec tunnel on the Server needs to have src-subnet 192.168.4.0/24 and dst-subnet as 192.168.4.0/24.

mpatel

Staff

If your config is as I have mentioned then this behavior is due to default ECMP (source-ip based) and is expected.

https://docs.fortinet.com/document/fortigate/6.2.16/cookbook/25967/equal-cost-multi-path.

https://community.fortinet.com/tpykb84852/attachments/tpykb84852/TKB20/3662/1/ECMP%20and%20Asymmetric%20Return%20Path%20Case%20Study.pdf

You will need to prioritize the routes using same distance but different priorities to ensure FGT routes properly.

Toshi_Esumi

SuperUser

@mpatelso you/these docs are saying if ECPM routes for a detination subnet is set [1) same cost+same priority, or 2)same cost + different priorities] is set, the FGT allows originating direction packets go out one interface and returning direction packets come back in the second interface is NOT treated as asymmetric routing and doesn't drop the returned packets.

I've learned something new today.

Toshi

themanyandonlyglennAuthor

New Member

Also on server I tried every v4-ecmp-mode and it just seems random which path it chooses. I did a TCP connect and the response debug flow was

enter IPSec interface con1,tun_id=192.168.11.1

output to IPSec tunnel con1_1, tun_id=192.168.12.1, vrf 0

This is the part I am struggling to understand, something in its routing logic can determine the tunnel ID of the session correctly (192.168.11.1) but it chooses to output on the other tunnel anyway, again randomly because I have ping traffic going also and "coincidentally" it responds on the expected tunnel. This last test was v4-ecmp-mode source-dst-ip-based but again this doesn't seem to have any effect.

mpatelAnswer

Staff

Intrigued!!

Can you configure "set netdevice enable" on the server end FGT IPSEC tunnel Con1. This will cause the fortigate to create a virtual interface for all incoming IPSEC connections. Keep ecmp mode as source-dst-ip-based and test.

themanyandonlyglennAuthor

New Member

Thanks! So far so good! Now the debug flow says enter interface con1_1 and output con1_1, and enter con1_0 and output con1_0. This fix seems to make sense too.

mpatel

Staff

Awesome!!

I am glad to hear that this resolved your issue.

Hope you are now confident in implementing this in Production.

themanyandonlyglennAuthor

New Member

I have another problem related to this configuration. I added a second src-subnet at client and the server is ignoring it.

This is from diag debug application ike -1:

client side:

IPsec SA selectors #src=2 #dst=1

src 0 4 0:192.168.2.0/255.255.255.0:0

src 1 4 0:192.168.6.0/255.255.255.0:0

dst 0 4 0:192.168.4.0/255.255.255.0:0

Server side log:

IPsec SA selectors #src=1 #dst=1

src 0 7 0:192.168.4.0-192.168.4.255:0

dst 0 7 0:192.168.2.0-192.168.2.255:0

add dynamic IPsec SA selectors 282

added dynamic IPsec SA proxyids new 1 282

add route 192.168.2.0/255.255.255.0 gw 192.168.11.1 oif con1_0(37) metric 15 priority 1

etc...

themanyandonlyglennAuthor

New Member

I should have googled first - I found I need to make two separate phase2-interface for each local subnet in a dialup tunnel as it only advertises one to the server. I don't get why, but oh well.

Toshi_Esumi

SuperUser

It's not that advertise or not advertise. They just need to match on both sides.
But, yes, you have to create separate phase2 config for each src-dst pair.
1) 192.168.6.0/24<->192.168.4.0/24
2) 192.168.2.0/24<->192.168.4.0/24

Toshi

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded