Description
This article describes how to fix an issue where the user is unable to login on SSL VPN with 2 ISPs having different MTUs. For example: one ISP is a PPPOE interface, and the other one is a regular physical interface. FortiClient gets stuck at 40 percent connection, and TLS handshake is not successful.
Scope
FortiGate v7.4.
Solution
Network Topology:
Configuration:
config system interface
edit "port17"
set vdom "root"
set type physical
next
edit "PPPoE_ISP"
set vdom "root"
set mode pppoe
set allowaccess ping https http
set alias "ISP 1"
.....
set preserve-session-route enable
set interface "port17"
set vlanid 35
next
edit "port15"
set vdom "root"
set ip 100.2.1.2 255.255.255.0
set allowaccess ping https
set type physical
set alias "ISP 2"
.....
next
end
config system sdwan
set status enable
config zone
edit "virtual-wan-link"
next
end
config members
edit 1
set interface "port15"
set gateway 100.2.1.1
next
edit 3
set interface "PPPoE_ISP"
next
end
end
config router static
edit 1
set distance 1
set comment "ISPs route"
set sdwan-zone "virtual-wan-link"
next
end
config vpn ssl settings
set banned-cipher SHA1 SHA256 SHA384
set servercert "Fortinet_Factory"
set tunnel-ip-pools "SSLVPN_TUNNEL_ADDR1"
set source-interface "PPPoE_ISP" "port15"
set default-portal "full-access"
config authentication-rule
edit 1
set groups "Local_Group"
set portal "full-access"
next
end
end
# get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
* - candidate default
Routing table for VRF=0
S* 0.0.0.0/0 [1/0] via 100.1.1.1, ppp2, [1/0]
[1/0] via 100.2.1.1, port15, [1/0]
The user will try to login on FortiClient with SSL VPN and it will fail. Fragmented packets are observed and the client sends out 'icmp: ip reassembly time exceeded'. This indicates that the Client did not receive all of the fragmented packets and it discards the initial fragment packets that it received.
Packet Capture:
diagnose sniffer packet any 'host 100.3.1.1' 6 1000 l
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: syn 2723465958
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: syn 1933644467 ack 2723465959
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: ack 1933644468
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: psh 2723465959 ack 1933644468
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: ack 2723466142
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: 1933644468 ack 2723466142 (frag 41603:1472@0+)
ppp2 out 100.1.1.2 -> 100.3.1.1: ip-proto-6 (frag 41603:8@1472)
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: 1933645928 ack 2723466142 (frag 41604:1472@0+)
ppp2 out 100.1.1.2 -> 100.3.1.1: ip-proto-6 (frag 41604:8@1472)
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: psh 1933647388 ack 2723466142
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: psh 1933648564 ack 2723466142
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: ack 1933644468
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: ack 1933644468
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: 1933644468 ack 2723466142 (frag 41607:1472@0+)
ppp2 out 100.1.1.2 -> 100.3.1.1: ip-proto-6 (frag 41607:8@1472)
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: 1933645928 ack 2723466142 (frag 41608:1472@0+)
ppp2 out 100.1.1.2 -> 100.3.1.1: ip-proto-6 (frag 41608:8@1472)
.....
ppp2 out 100.1.1.2.443 -> 100.3.1.1.56422: fin 1933649509 ack 2723466142
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: ack 1933644468
.....
ppp2 in 100.3.1.1 -> 100.1.1.2: icmp: ip reassembly time exceeded
ppp2 in 100.3.1.1.56422 -> 100.1.1.2.443: rst 2723466142 ack 1933644468
ppp2 in 100.3.1.1 -> 100.1.1.2: icmp: ip reassembly time exceeded
ppp2 in 100.3.1.1 -> 100.1.1.2: icmp: ip reassembly time exceeded
ppp2 in 100.3.1.1 -> 100.1.1.2: icmp: ip reassembly time exceeded
Solution:
If different network routes have different size limits for data (MTU), and some routers cannot properly handle split-up data (fragments), it can cause issues with any incoming TCP connections (local in / passive local) to the FortiGate. It is necessary to check on every hop where the fragmented packets are getting dropped. The user can still login on SSLVPN if the fragmented packets are not getting drop.
Workaround 1: Change the SD-WAN priority. This will prevent ECMP.
config system sdwan
config members
edit 1
set interface "port15"
set gateway 100.2.1.1
set priority 3
next
edit 3
set interface "PPPoE_ISP"
next
end
end
get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
* - candidate default
Routing table for VRF=0
S* 0.0.0.0/0 [1/0] via 100.1.1.1, ppp2, [1/0]
[1/0] via 100.2.1.1, port15, [3/0] <-----
Workaround 2: Configure MTU override to be the same with PPPoE. This will prevent packets getting fragmented without changing the configuration on SDWAN.
config system interface
edit "port15"
set mtu-override enable
set mtu 1492
next
end
Related article:
