Description
This article describes that fragmented packets are not sent out when IPsec npu-offload is enabled. This is for an IPsec tunnel configured with vpn-id-ipip encapsulation and with NP7 Hardware model.
Scope
FortiGate.
Solution
Network Topology:
Source 10.10.11.1 -> FortiGate 601F [vrf 11-] (Spoke) [1.1.1.1] -> [1.1.1.2] Router [2.2.2.2]-> [2.2.2.1]FortiGate (Hub) -> Destination 10.10.22.2.
FortiGate 601F is a hardware model with NP7.
For a configuration guide, see SD-WAN segmentation over a single overlay - FortiGate 7.2.0 new features.
- FortiGate-601F with NPU offload enabled.
config vpn ipsec phase1-interface
edit "601F-p1"
set interface "port1"
set ike-version 2
...
set encapsulation vpn-id-ipip
...
next
end
get router info routing-table bgp
...
Routing table for VRF=11
B V 10.10.22.2/32 [200/0] via 10.10.32.1 tag 1 (recursive via 601F-p1 tunnel 1.1.1.1), 00:00:29, [100/0]
Ping Destination with data size of 56. Ping is successful.
PING 10.10.22.2 (10.10.22.2): 56 data bytes
64 bytes from 10.10.22.2: icmp_seq=0 ttl=253 time=1.2 ms
64 bytes from 10.10.22.2: icmp_seq=1 ttl=253 time=0.5 ms
64 bytes from 10.10.22.2: icmp_seq=2 ttl=253 time=0.4 ms
64 bytes from 10.10.22.2: icmp_seq=3 ttl=253 time=0.5 ms
64 bytes from 10.10.22.2: icmp_seq=4 ttl=253 time=0.5 ms
--- 10.10.22.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.6/1.2 ms
FortiGate-601F Packet Capture:
diagnose sniffer packet any 'host 10.10.22.2' 4 200 l
interfaces=[any]
filters=[host 10.10.22.2]
2025-05-30 13:37:03.776990 port16 in 10.10.11.1 -> 10.10.22.2: icmp: echo request
2025-05-30 13:37:03.777251 601F-p1 out 10.10.11.1 -> 10.10.22.2: icmp: echo request
2025-05-30 13:37:03.777661 601F-p1 in 10.10.22.2 -> 10.10.11.1: icmp: echo reply
2025-05-30 13:37:03.777688 port16 out 10.10.22.2 -> 10.10.11.1: icmp: echo reply
Ping Destination with data size of 1600. Ping failed.
PING 10.10.22.2 (10.10.22.2): 1600 data bytes
--- 10.10.22.2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
FortiGate-601F Packet Capture:
diagnose sniffer packet any 'host 10.10.22.2' 4 200 l
interfaces=[any]
filters=[host 10.10.22.2]
2025-05-30 13:38:48.351573 port16 in 10.10.11.1 -> 10.10.22.2: icmp: echo request (frag 27797:1480@0+)
2025-05-30 13:38:48.351574 port16 in 10.10.11.1 -> 10.10.22.2: ip-proto-1 (frag 27797:128@1480)
2025-05-30 13:38:48.351836 601F-p1 out 10.10.11.1 -> 10.10.22.2: icmp: echo request
Packet capture on Router:
No fragmented packets received.
- FortiGate-601F with NPU offload disabled.
config vpn ipsec phase1-interface
edit "601F-p1"
set interface "port1"
set ike-version 2
...
set npu-offload disable
set encapsulation vpn-id-ipip
...
next
end
Ping Destination with data size of 1600. Ping is successful.
PING 10.10.22.2 (10.10.22.2): 1600 data bytes
1608 bytes from 10.10.22.2: icmp_seq=0 ttl=253 time=3.6 ms
1608 bytes from 10.10.22.2: icmp_seq=1 ttl=253 time=0.7 ms
1608 bytes from 10.10.22.2: icmp_seq=2 ttl=253 time=0.7 ms
1608 bytes from 10.10.22.2: icmp_seq=3 ttl=253 time=0.6 ms
1608 bytes from 10.10.22.2: icmp_seq=4 ttl=253 time=0.7 ms
--- 10.10.22.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.6/1.2/3.6 ms
FortiGate-601F Packet Capture:
diagnose sniffer packet any 'host 10.10.22.2' 4 200
interfaces=[any]
filters=[host 10.10.22.2]
2025-05-30 13:46:30.090301 port16 in 10.10.11.1 -> 10.10.22.2: icmp: echo request (frag 47243:1480@0+)
2025-05-30 13:46:30.090302 port16 in 10.10.11.1 -> 10.10.22.2: ip-proto-1 (frag 47243:128@1480)
2025-05-30 13:46:30.090561 601F-p1 out 10.10.11.1 -> 10.10.22.2: icmp: echo request
2025-05-30 13:46:30.091357 601F-p1 in 10.10.22.2 -> 10.10.11.1: icmp: echo reply
Router packet capture:
port1 in 1.1.1.1 -> 2.2.2.2: ESP(spi=0x74310586,seq=0x125) (frag 48194:1480@0+)
port1 in 1.1.1.1 -> 2.2.2.2: ip-proto-50 (frag 48194:224@1480)
port2 out 1.1.1.1 -> 2.2.2.2: ESP(spi=0x74310586,seq=0x125) (frag 48194:1480@0+)
port2 out 1.1.1.1 -> 2.2.2.2: ip-proto-50 (frag 48194:224@1480)
port2 in 2.2.2.2 -> 1.1.1.1: ESP(spi=0x174e1ca0,seq=0x12a) (frag 20994:1480@0+)
port2 in 2.2.2.2 -> 1.1.1.1: ip-proto-50 (frag 20994:224@1480)
port1 out 2.2.2.2 -> 1.1.1.1: ESP(spi=0x174e1ca0,seq=0x12a) (frag 20994:1480@0+)
port1 out 2.2.2.2 -> 1.1.1.1: ip-proto-50 (frag 20994:224@1480)
- FortiGate-601F with NPU offload enabled and NPU ip-fragment enable.
config vpn ipsec phase1-interface
edit "601F-p1"
set interface "port1"
set ike-version 2
...
set encapsulation vpn-id-ipip
...
next
end
config system npu
config ip-reassembly
set status enable
end
end
Ping Destination with data size of 1600. Ping is successful.
PING 10.10.22.2 (10.10.22.2): 1600 data bytes
1608 bytes from 10.10.22.2: icmp_seq=0 ttl=253 time=1.2 ms
1608 bytes from 10.10.22.2: icmp_seq=1 ttl=253 time=51.4 ms
1608 bytes from 10.10.22.2: icmp_seq=2 ttl=253 time=0.7 ms
1608 bytes from 10.10.22.2: icmp_seq=3 ttl=253 time=0.6 ms
1608 bytes from 10.10.22.2: icmp_seq=4 ttl=253 time=0.6 ms
--- 10.10.22.2 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.6/10.9/51.4 ms
FortiGate-601F Packet Capture:
diagnose sniffer packet any 'host 10.10.22.2' 4 200 l
interfaces=[any]
filters=[host 10.10.22.2]
port16 in 10.10.11.1 -> 10.10.22.2: icmp: echo request
601F-p1 out 10.10.11.1 -> 10.10.22.2: icmp: echo request
601F-p1 in 10.10.22.2 -> 10.10.11.1: icmp: echo reply
Router packet capture:
port2 out 142.46.195.42 -> 206.172.39.52: ESP(spi=0x743105d0,seq=0x222) (frag 29699:1480@0+)
port2 out 142.46.195.42 -> 206.172.39.52: ip-proto-50 (frag 29699:224@1480)
port2 in 206.172.39.52 -> 142.46.195.42: ESP(spi=0x174e1cae,seq=0x208) (frag 3588:1480@0+)
port2 in 206.172.39.52 -> 142.46.195.42: ip-proto-50 (frag 3588:224@1480)
port1 out 206.172.39.52 -> 142.46.195.42: ESP(spi=0x174e1cae,seq=0x208) (frag 3588:1480@0+)
port1 out 206.172.39.52 -> 142.46.195.42: ip-proto-50 (frag 3588:224@1480)
Note:
If ping still failed after enabling NPU reassembly, perform one of the following steps:
- Flush the IPsec tunnel with the command below. The IPsec tunnel will be restarted.
diagnose vpn tunnel flush
- Reboot FortiGate. Rebooting FortiGate will cause a traffic outage.
execute reboot
- The topology shows Spoke to Hub. The issue can be seen on Spoke to Spoke as well.
- There is a known issue (1149340) that fragmented packets are not being sent out, and it is resolved on v7.4.9, v7.6.4, and v8.0.0.
Related documents:
Segmentation over single overlay - FortiGate 7.2.0
SD-WAN segmentation over a single overlay - FortiGate 7.2.0 new features
NP7 and NP7lite acceleration - FortiGate 7.6.3
