Technical Tip: Dial-up VPN IPsec over TCP best practices
Description
This article describes recommended best practices for deploying an IPsec dial-up Virtual Private Network (VPN) tunnel over Transmission Control Protocol (TCP) on FortiGate devices. This configuration is typically used in environments where User Datagram Protocol (UDP) traffic is restricted or blocked by intermediate firewalls or carrier-grade NAT.
Scope
FortiGate v7.4.2 and above.
Solution
Use case:
IPsec VPN over TCP enables IPsec tunnels to function reliably in networks where standard protocols such as UDP/500 and IP protocol 50 (ESP) are not permitted. TCP encapsulation ensures that VPN traffic can traverse restrictive networks, including public Wi-Fi, mobile carrier networks, or enterprise firewalls with strict outbound rules.
Recommended best practices:
- Use TCP port 443 for transport if Secure Sockets Layer (SSL) VPN or web services are not running on the same port/interface.
- Enable TCP encapsulation only on the dial-up server FortiGate to maintain the expected traffic flow and avoid dual encapsulation.
- Configure tcp-mss-sender and tcp-mss-receiver to prevent fragmentation, recommended value: 1360.
- Disable ESP detection on intermediate firewalls when TCP is exclusively used for transport.
- Monitor system resources, as TCP encapsulation may introduce additional CPU overhead compared to native ESP/IPsec transport.
Example CLI Configuration:
On the dial-up server FortiGate (listening on TCP port 443):
config system settings show full | grep ike-tcp-port set ike-tcp-port 443 end
config vpn ipsec phase1-interface edit "dialup-tcp" set type dynamic set interface "wan1" set ike-version2 set peertype any set proposal aes256-sha256 set dhgrp 14 set transport tcp set psksecret ENC <encrypted_psk> next end
config firewall policy edit <id> set tcp-mss-sender 1360 set tcp-mss-receiver 1360 next end
On the dial-up client FortiGate:
config system settings show full | grep ike-tcp-port set ike-tcp-port 443 end
config vpn ipsec phase1-interface edit "client-tcp" set type dynamic set interface "wan1" set remote-gw x.x.x.x set ike-version 2 set peertype any set proposal aes256-sha256 set dhgrp 14 set transport tcp set psksecret ENC <encrypted_psk> next end
config firewall policy edit <id> set tcp-mss-sender 1360 set tcp-mss-receiver 1360 next end
Monitoring:
Use the following command to verify the tunnel's encapsulation method and active status:
diagnose vpn tunnel list | grep encap
Additional considerations:
- TCP-encapsulated VPN traffic may be subject to inspection or throttling by deep packet inspection engines or secure web gateways.
- Do not bind both IPsec over TCP and SSL VPN services to the same interface and TCP port (for example, port 443) unless virtual IP (VIP) mapping or port forwarding is configured.
- Ensure intermediate network devices allow outbound TCP connections to the configured VPN port.
- This feature only works with IKE version 2.
- IPsec over TCP is not supported on free FortiClient VPN versions. Other limitations are described in FortiClient standalone and licensed version feature comparison.
Related documents:
Technical Tip: Troubleshooting IPsec VPN tunnel connectivity