Ipsec Site 2 Site VPN - slow performance and malformed packets (Wireshark)
I have got a IPSEC Site 2 Site VPN between 2 Fortigate Appliances:
1x Fortigate 60D connected to the Internet 12/12 Mbit: v5.0,build0310 (GA Patch 11)
1x Fortigate 100D connected to the Internet 20/20 Mbit: v5.0,build0310 (GA Patch 11)
We have got a Database Application running which transfers a small amount of data through the tunnel (a few MB for login for example). Although the speed of the connection is not that slow, it lasts 2 minutes until we get the loginscreen.
So I started Wireshark and got the following capture (log.jpg).
Really don't know why there are malformed packets with IPA Protocol (192.168.10.248 is the databaseserver IP). Is Wireshirk missinterpreting the protocol eventually?
Regarding the Malformed packets. I have troubleshot this before and this was due to our Geo redundant platform setup. Where 2 mirror Fortigates one active and one standby controlled by BGP fail over. The standby peer had auto-negotiate on for the phase 1 which causes it to actively reach out to the remote peer to establish the phase1 connection, which was undesirable. This essentially causes the phase1 not to come up, which the errors being Malformed packets. I have also seen it on when the remote peer was a SonicWall and the PSK had a mismatch.
Regarding performance, try turning off the reply detection on your phase2, this will auto-enable the npu offload. When on you have to manually enabled the feature.
config system npu
set enc-offload-antireplay enble
set dec-offload-antireplay enable
set offload-ipsec-host enable
====and make sure=====
config firewall policy
set auto-asic-offload enable
next, Regarding performance, I have troubleshot some ERP Database applications through a VPN with poor performance. The fix to that issue was to increase the session TCP TTL to be like 8+ hours, you can set this for particular ports also.:
I would look at the crypto status and validate that noting is process switch via software;
diag vpn ipsec status
In my experience, various ciphers offer a slight improvement over others. You would be best to test this in a control lab using a set packet size and with udp and once again with iperf/jperf or similar if you want to gain ever inch of performance out of your ipsec tunnels.
BTW: I'm really surprised that fritzbox didn't have ikev2 support. I would thought the 7490s would have it by now.
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.