Skip to main content
A
Anonymous_User
Contributor
August 6, 2008
Question

Fragmentation of ESP packets - truncated-ip

  • August 6, 2008
  • 6 replies
  • 6933 views
Hello everyone, I am experincing a lot of fragmentation on all my VPNs. I discovered this when we set up a new VPN over a new MPLS line and thought it was a problem in the MPLS - but that is fine. It also appears to happen on the VPNs that go over the Internet. I tried setting the tcp-mss and MTU to lower values, but this did not help. Now I heard that it may be possible disallow the fragmentation of packets. Do you know if this is possible or if there is anything else I can do? Here is what I see between my VPN peers (FGT400a to FGT50a/60b): 
9.164151 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.164575 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.173824 10.24.1.1 -> 10.24.10.1:  ip-proto-50 156  9.174828 10.24.10.1 -> 10.24.1.1:  ip-proto-50 364  9.174828 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 364  9.183676 10.24.10.1 -> 10.24.1.1:  ip-proto-50 308  9.183676 truncated-ip - 21 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 308  9.183970 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.184073 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  9.184073 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  9.265988 10.24.1.1 -> 10.24.10.1:  ip-proto-50 84  9.278419 10.24.10.1 -> 10.24.1.1:  ip-proto-50 108  9.278419 truncated-ip - 19 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 108  9.287059 10.24.10.1 -> 10.24.1.1:  ip-proto-50 244  9.287059 truncated-ip - 18 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 244  9.287436 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.295359 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1460  9.295359 truncated-ip - 19 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1460  9.296935 10.24.10.1 -> 10.24.1.1:  ip-proto-50 460  9.296935 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 460  9.297291 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.303517 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.303517 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.309656 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.309656 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.310627 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.312327 10.24.10.1 -> 10.24.1.1:  ip-proto-50 724  9.312327 truncated-ip - 15 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 724  9.318821 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.318821 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.319606 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.325128 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.325128 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.328197 10.24.10.1 -> 10.24.1.1:  ip-proto-50 836  9.328197 truncated-ip - 18 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 836  9.328880 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.334591 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.334591 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.340721 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.340721 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.340923 10.24.10.1 -> 10.24.1.1:  ip-proto-50 172  9.340923 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 172  9.341633 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.347794 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.347794 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.348879 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.351979 10.24.10.1 -> 10.24.1.1:  ip-proto-50 988  9.351979 truncated-ip - 18 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 988  9.358363 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.358363 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.359311 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.364560 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.364560 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  9.369774 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1268  9.369774 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1268  9.370619 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.374457 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1140  9.374457 truncated-ip - 14 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1140  9.377278 10.24.10.1 -> 10.24.1.1:  ip-proto-50 724  9.377278 truncated-ip - 21 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 724  9.377864 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  9.384344 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1420  9.384344 truncated-ip - 17 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1420  9.481649 10.24.10.1 -> 10.24.1.1:  ip-proto-50 116  9.481649 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 116  9.482494 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  11.638381 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  11.638381 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  11.638849 10.24.1.1 -> 10.24.10.1:  ip-proto-50 92  15.635853 10.24.1.1 -> 10.24.10.1:  ip-proto-50 92  15.669933 10.24.10.1 -> 10.24.1.1:  ip-proto-50 100  15.669933 truncated-ip - 14 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 100  15.731477 10.24.1.1 -> 10.24.10.1:  ip-proto-50 132  16.044901 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  16.044901 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  16.108037 10.24.1.1 -> 10.24.10.1:  ip-proto-50 148  16.154262 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  16.154262 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  18.405066 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  18.405066 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 92  18.405469 10.24.1.1 -> 10.24.10.1:  ip-proto-50 92  22.615196 10.24.10.1 -> 10.24.1.1:  ip-proto-50 84  22.615196 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 84  22.615595 10.24.1.1 -> 10.24.10.1:  ip-proto-50 84  22.634223 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  22.634223 truncated-ip - 20 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 76  22.635951 10.24.10.1 -> 10.24.1.1:  ip-proto-50 356  22.635951 truncated-ip - 21 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 356  22.642641 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  22.642641 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  22.643239 10.24.1.1 -> 10.24.10.1:  ip-proto-50 76  22.668604 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  22.668604 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  22.674708 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468  22.674708 truncated-ip - 16 bytes missing! 10.24.10.1 -> 10.24.1.1:  ip-proto-50 1468
Thanks for reading! stephan

    6 replies

    rwpatterson
    rwpatterson
    New Member
    August 6, 2008
    Run a ping across the connection with the ' -f' flag (do not truncate) and the ' -l' flag (packet length in bytes, that' s a lower case " L" ). Find out what the largest packet that can be sent is, and adjust your tcp-mss accordingly. I found mine to be 142x bytes before fragmenting, so I set my tcp-mss to 1400 across all tunnels. This should be done on both sides of the IPSec VPN tunnel. This has sped up the remote sites greatly since they no longer have to break down and reassemble each packet across the WAN. Use the below command: 
    > ping -f -l 1400 192.168.x.x
    Good luck
    A
    Anonymous_UserAuthor
    Contributor
    August 6, 2008
    Hi Bob, thanks for your help. I understand what you mean, but I' m not sure if that can help me. I tried pinging with different lenghts, now I tried -f as well. Does not seem to have an effect (with -l 1400, 1200, 1000): 34.436494 10.24.1.1 -> 10.24.2.1: ip-proto-50 1460 34.456613 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 34.456613 truncated-ip - 16 bytes missing! 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 35.437245 10.24.1.1 -> 10.24.2.1: ip-proto-50 1460 35.457505 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 35.457505 truncated-ip - 16 bytes missing! 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 36.437759 10.24.1.1 -> 10.24.2.1: ip-proto-50 1460 36.458004 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 36.458004 truncated-ip - 16 bytes missing! 10.24.2.1 -> 10.24.1.1: ip-proto-50 1460 41.446148 10.24.2.1.500 -> 10.24.1.1.500: udp 92 41.446426 10.24.1.1.500 -> 10.24.2.1.500: udp 92 43.476714 10.24.1.1 -> 10.24.2.1: ip-proto-50 1260 43.494879 10.24.2.1 -> 10.24.1.1: ip-proto-50 1260 43.494879 truncated-ip - 16 bytes missing! 10.24.2.1 -> 10.24.1.1: ip-proto-50 1260 50.483658 10.24.2.1.500 -> 10.24.1.1.500: udp 92 50.483927 10.24.1.1.500 -> 10.24.2.1.500: udp 92 51.106043 10.24.1.1 -> 10.24.2.1: ip-proto-50 1060 51.122140 10.24.2.1 -> 10.24.1.1: ip-proto-50 1060 51.122140 truncated-ip - 16 bytes missing! 10.24.2.1 -> 10.24.1.1: ip-proto-50 1060 When I ping from Fortigate to Fortigate (non-VPN), it goes unfragmented when I set the data-size to 1472, which is correct: 1472 = payload + 20 = IP header + 8 = ICMP Header = 1500 So I know the MTU size is correct. When I use TCP (HTTPS) through the WAN link I do not see any fragmentation messages. Seems like only ESP is affected. Any thoughts? Thanks stephan
    rwpatterson
    rwpatterson
    New Member
    August 6, 2008
    What versions of firmware are you running?
    A
    Anonymous_UserAuthor
    Contributor
    August 6, 2008
    Fortigate 400A: 3.00-b0662(MR6 Patch 1) Fortigate-60B No1: 3.00-b0662(MR6 Patch 1) Fortigate-60B No2: 3.00-b0726(MR7) Thought it might be a problem with the 400a, but it happens between the two 60Bs as well :(
    rwpatterson
    rwpatterson
    New Member
    August 6, 2008
    From the CLI, run the following command:
    diagnose hardware deviceinfo nic interface_name
    If you have errors, run the command additional times and see if the count grows. You may have an autonegotiation issue as I did, or a duplex mismatch.
    A
    Anonymous_UserAuthor
    Contributor
    August 7, 2008
    Hi Bob, I get this on the Fortigate 400: 
      FG400A-2 # diagnose hardware deviceinfo nic port4  Description               Intel(R) PRO/100 M Desktop Adapter  Driver_Name               e100  Driver_Version            2.1.29  PCI_Vendor                0x8086  PCI_Device_ID             0x1229  PCI_Subsystem_Vendor      0x8086  PCI_Subsystem_ID          0x0070  PCI_Revision_ID           0x0010  PCI_Bus                   3  PCI_Slot                  6  IRQ                       20  System_Device_Name        port4  Current_HWaddr            00:09:0F:09:00:03  Permanent_HWaddr          00:09:0F:84:76:ED  Part_Number               ffffff-0ff    Link                      up  Speed                     100  Duplex                    full  FlowControl               receive/transmit  State                     up    Rx_Packets                2877827  Tx_Packets                1022152  Rx_Bytes                  1619676618  Tx_Bytes                  452862314  Rx_Errors                 0  Tx_Errors                 10  Rx_Dropped                0  Tx_Dropped                0  Multicast                 N/A  Collisions                0  Rx_Length_Errors          0  Rx_Over_Errors            0  Rx_CRC_Errors             0  Rx_Frame_Errors           0  Rx_FIFO_Errors            0  Rx_Missed_Errors          0  Tx_Aborted_Errors         0  Tx_Carrier_Errors         10  Tx_FIFO_Errors            0  Tx_Heartbeat_Errors       0  Tx_Window_Errors          0    Rx_TCP_Checksum_Good      0  Rx_TCP_Checksum_Bad       0  Tx_TCP_Checksum_Good      0  Tx_TCP_Checksum_Bad       0    Tx_Single_Collision_Frames 0  Tx_Multi_Collision_Frames 0  Tx_Deferred               0  Rx_Symbol_Errors          0    Tx_Pause_Frames           0  Rx_Pause_Frames           0  Rx_Control_Unknown_Opcodes 0    Tx_TCO_Packets            0  Rx_TCO_Packets            0    Rx_Interrupt_Packets      0  Rx_Polling_Packets        2879708  Polling_Interrupt_Switch  0
    Rx/TxBytes and Packtes increase, other than that, only Rx_Polling_Packets increase. On the 60Bs: 
      # diagnose hardware deviceinfo nic wan1  Description             sundance Ethernet driver1.01+LK1.21  chip_id                 6                                                                     IRQ                     5                                                                     System_Device_Name      wan1  Current_HWaddr          00:09:0f:79:0a:ae  Permanent_HWaddr        00:09:0f:79:0a:ae  State                   up                                                                    Link                    up                                                                    Speed                   100                                                                   Duplex                  full                                                                  Rx_Packets              621178                                                                Tx_Packets              963582                                                                Rx_Bytes                112144056                                                             Tx_Bytes                479493981                                                             Collisions              0                                                                     Rx_Missed_Errors        0  Tx_Carrier_Errors       0    and    # diagnose hardware deviceinfo nic wan1  Description             sundance Ethernet driver1.01+LK1.21  chip_id                 6  IRQ                     5  System_Device_Name      wan1  Current_HWaddr          00:09:0f:79:05:62  Permanent_HWaddr        00:09:0f:79:05:62  State                   up  Link                    up  Speed                   100  Duplex                  full  Rx_Packets              478361  Tx_Packets              601930  Rx_Bytes                232205448  Tx_Bytes                510926723  Collisions              0  Rx_Missed_Errors        0  Tx_Carrier_Errors       0
    100/Full is correct everywhere... Thanks stephan
    rwpatterson
    rwpatterson
    New Member
    August 7, 2008
    Also check the inside port(s) the internal device is on...
    A
    Anonymous_UserAuthor
    Contributor
    August 7, 2008
    On the LAN interface of the FG400 I see these: Rx_CSum_Offload_Good 1197420231 rising at about 400/second Rx_CSum_Offload_Errors 305 Errors not rising. I see no errors on the internal interfaces of the FG60s or at the connected switches. A consultant from a Network specialist here told me they have it on ALL their Fortigate VPNs and never checked whether this could be a problem. They wondered though, but never found the time to analyze it... I promised to keep him up to date ;( No response to my ticket from yesterday yet...
    A
    Anonymous_UserAuthor
    Contributor
    August 12, 2008
    After sending some traces and some discussion with the Fortinet Support they came to this conclusion:
    The truncated-ip message is an expected behavior. What happens is that when Fortigate gets packets through the VPN it tries to match the packet header as a normal packet but it does not match thats why it shows it as truncated packet. Its normal for VPN traffic and it does not create any performance problem on the network or on the unit.
    You will not see truncated-ip messages when sniffing on interface any - this is something like when do a trace on a LAN interface with Wireshark and get currupted TCP checksums. Everything fine, just the sniffer getting something wrong :-) Thanks.
    Terms & ConditionsAccessibility statement