Skip to main content
JaskiratM
Staff & Editor
JaskiratM
Staff & Editor
March 9, 2026

Technical Tip: File download fails with IPv6 VXLAN over IPsec VPN tunnel

  • March 9, 2026
  • 0 replies
  • 300 views
Description

 

This article describes a scenario where file downloads fail when using IPv6 VXLAN encapsulation within an IPsec VPN tunnel. In this scenario, small  file downloads may succeed, whereas larger-sized transfers that trigger packet fragmentation will fail.

 

Scope

 

FortiGate, IPv6 VXLAN, IPsec.

 

Solution

 

This issue can occur when IPv6-based VXLAN encapsulation is enabled within an IPsec tunnel on the FortiGate. The following is an example of the configuration that can trigger the issue:

 

config vpn ipsec phase1-interface     edit "vpn-tunnel"         set interface <intf>         set encapsulation vxlan      next end

 

In lab testing, file downloads have been observed to fail when the following conditions are met:

  1. The file transfer size is larger than a single packet can carry.
  2. IPv6 is being used (IPv4 did not exhibit any issues with file downloads).
  3. The size of the packets sent from the host to the FortiGate is larger than the maximum transmission unit (MTU) of the IPsec link, which triggers packet fragmentation.

 

Workaround:

 

To avoid packet fragmentation for TCP-based protocols (like SFTP or HTTPS), clamp the TCP MSS to ensure that the packets cannot exceed the IPsec tunnel MTU. This forces endpoints to use smaller TCP segments, which in-turn results in packets that will fit within the end-to-end MTU without requiring fragmentation.

 

config firewall policy     edit 3         set tcp-mss-sender 1292         set tcp-mss-receiver 1292     next end

 

For considerations regarding changes to MSS behavior and values, refer to the notes Technical Tip: Setting TCP MSS Value.

 

Recommended approach:

 

The encapsulation vxlan approach to VXLAN (i.e. enabling it within the IPsec tunnel directly) is an older method that is no longer recommended for usage in production environments. Instead, the recommendation is to use the native VXLAN interface configuration, which involves creating a standalone VXLAN tunnel interface whose traffic can then be routed out any FortiGate interface (including standard route/interface-based IPsec tunnels):

 

config system vxlan     edit <vxlan-name>         set interface <interface-name>         set local-ip <local-ip>         set vni <vni-id>     next end

 

This modern method of VXLAN does not demonstrate the same issues as the method described earlier in the article and will not negatively impact file downloads.

 

Related documents:

Terms & ConditionsAccessibility statement