Technical Tip: Understanding Maximum MTU on VXLAN Interfaces

[akileshc]

Description	This article describes the behavior and configuration details for setting the MTU on VXLAN interfaces when using a physical or loopback interface as the underlying interface. This document also provides step-by-step guidance on adjusting MTU settings and verifying configurations.
Scope	FortiOS.
Solution	The maximum MTU (Maximum Transmission Unit) of a VXLAN interface is determined by the MTU of the underlying physical or logical interface. This MTU value is reduced by the overhead introduced by the VXLAN encapsulation. To calculate the effective VXLAN MTU, use the following formula: VXLAN MTU = Underlying Interface MTU - VXLAN Encapsulation Overhead VXLAN Encapsulation Overhead: VXLAN encapsulation adds several layers of headers to the packet, which contribute to the overall overhead: Outer Ethernet Header: 14 bytes (This header encapsulates the entire VXLAN packet, providing the outer MAC address details without VLAN tagging; add 4 bytes if VLAN tagging is used). Outer IP Header: 20 bytes for IPv4 or 40 bytes for IPv6. Outer UDP Header: 8 bytes. VXLAN Header: 8 bytes. Total Overhead: IPv4 Encapsulation: 50 bytes (54 bytes with VLAN tagging). IPv6 Encapsulation: 70 bytes (74 bytes with VLAN tagging). Example Calculation: For a physical or logical interface with an MTU of 1500 bytes and VXLAN encapsulation using IPv4: Effective VXLAN MTU = 1500 - 50 = 1450 bytes The default MTU for Ethernet networks is usually set to 1500 bytes. However, to improve efficiency in high-speed networks, the MTU can be changed to Jumbo MTU, which typically ranges from 9000 to 9216 bytes. The MTU can be adjusted on physical interfaces in all versions of FortiOS, likewise, the MTU of the VXLAN interface built over the physical interface can also be changed. However, the logical interface like the loopback interface, the ability to change the MTU is version-dependent. Refer to the KB article Technical Tip: FortiGate MTU on loopback interface for further details. In FortiOS version 7.4.0 and above, the MTU on logical interfaces such as the loopback interface can be modified. So, the MTU of the VXLAN interface built over loopback can also be modified. Below are examples of how to configure the MTU for VXLAN interfaces and MTU override settings. Case 1: No MTU Override Enabled (Default MTU). If the MTU override option is not enabled, the loopback interface will retain its default MTU of 1500 bytes. Here is the configuration: Step 1: Configure the Loopback Interface (Default MTU). config system interface     edit "Lo1"         set vdom "root"         set ip 10.51.0.15 255.255.255.255         set allowaccess ping https ssh http         set type loopback         set role lan         set snmp-index 56     next end Step 2: Configure the VXLAN Interface and define the MTU value. config sys interface     edit vxlan-1         set interface "Lo1"         set vni 1001         set remote-ip "10.10.10.2"         set mtu-override enable         set mtu ? <integer> <-- Maximum transmission unit (68-1450). Default 1450.     next end Step 3: Verify MTU on VXLAN Interface. diagnose netlink interface list | grep vxlan-1 if=vxlan-1 family=00 type=1 index=16 mtu=1450 link=0 master=0 Case 2: MTU Override Enabled (Jumbo MTU). If the MTU override option is enabled, the MTU of the loopback interface can be set to a larger value (for example 10000 bytes). Here is the configuration: Step 1: Configure the Loopback Interface with MTU Override. config system interface     edit "Lo1"         set vdom "root"         set ip 10.51.0.15 255.255.255.255         set allowaccess ping https ssh http         set type loopback         set mtu-override enable         set mtu 10000     next end Step 2: Configure the VXLAN Interface with the New MTU. config sys interface     edit vxlan-1         set mtu-override enable         set mtu ? <integer> <-- Maximum transmission unit (68-9950). Default 9950.         set mtu 9950     next end Step 3: Verify MTU on VXLAN Interface. diagnose netlink interface list | grep vxlan-1 if=vxlan-1 family=00 type=1 index=16 mtu=9950 link=0 master=0 The MTU for a VXLAN interface is always related to the MTU of the underlying interface and VXLAN encapsulation overhead. That will make the configuration of the proper MTU of VXLAN tunnels easier and thus ensure efficient transmission of network traffic without fragmentation.