Technical Tip: Understanding PMTU on GRE tunnel and using fragmentation to recover the tunnel traffic

[nevan]

Description	The article discusses the PMTU (path MTU) of the GRE Tunnel, identifying and using the fragmentation to recover the traffic over the tunnel.
Scope	FortiGate.
Solution	When the GRE tunnel is configured in FortiOS, it may be noticed that the GRE tunnel has an MTU of less than 1500. Once the network traffic travels through the interface, it decides how much data is to be transferred in each packet by referring to the MTU or Maximum Transmission Unit. In general, the MTU value of any interface is expected to be 1500 Bytes where 20 Bytes are reserved for the IP header and another 20 Bytes for the TCP header. This results in 1460 Bytes left for the payload. Not all the devices and their interfaces will have the 1500 MTU in the travel path of the traffic. This might result in the packet getting discarded for becoming larger than its expected packet size set in the interface.   From the figure above it is noticed that the path has less MTU than the originator and receiver of the packet. This minimum MTU in the path is called the Path MTU or PMTU. The PATH MTU can be noticed from FortiOS by following the command in the CLI session 'diagnose ip rtcache list' which is explained in this KB article: Technical Tip: FortiGate GRE tunnel with PMTU (Path MTU) The above figure portrays the Path MTU within the sites of a GRE tunnel. GRE has 24 bytes extra header where 4 Bytes are for the GRE header and 20 Bytes for the GRE IP header. This results in the packet size being larger than regular TCP/IP packets. IP Packet Form: PAYLOAD (1460 BYTES) + TCP HEADER (20 BYTES) + IP HEADER (20 BYTES) = 1500 BYTES GRE Packet Form: PAYLOAD (1460 BYTES) + TCP HEADER (20 BYTES) + IP HEADER (20 BYTES) + [GRE HEADER(4 BYTES) + GRE IP HEADER(20 BYTES)] = 1524 BYTES The PMTU discovers the minimum MTU in the traffic path with the TCP path MTU discovery process. The initiator device sends a packet with a DF (Do not Fragment) bit enabled and within the travel path if it is discarded by any router for exceeding the packet size, that router will return an 'fragmentation needed' ICMP message (ICMP type 3 code 4) to the client. This message contains the information about the MTU and the PMTU is updated according to the message. This KB article explains how to discover the responsible device for changing the MTU in the path and discarding the traffic: Troubleshooting Tip: How to find the device responsible for fragmentation along the network path The best practice for preventing packets from getting discarded for the Path MTU is reducing the MTU and payload size by calculating the packet header. If the MTU is 1500 for the GRE traffic, as per calculation, the payload size can be set as (1500-(40+24)) = 1436 bytes. Once the Maximum Segment Size is set or the MSS value is less than 1436 bytes, fragmentation will take place and using fragmentation can avoid the packet getting discarded for the PMTU. Changing the MSS value in the GRE tunnel interface is explained in this KB article: Technical Tip: Setting TCP MSS value Related articles: Technical Tip: Configuring and verifying a GRE tunnel between two FortiGates (static routing) Technical Tip: Enable path MTU discovery