IPSec High Latency optimizations

[sainc]

Hi everyone,

We currently have two offices one in USA and one in Europe both of which have FortiGate 60F.

They are tunneled with IPSec and the bandwitdh using single iperf3 stream is extremely slow at 10Mbps but when using 10 parallel stream it is around 65Mbps.

The office in Europe has 1Gbps download and 400Mbps upload while office in USA has 300Mbps for both download and upload.

If you need any information to help me with the problem and to help me optimize the bandwidth between two offices, I will provide it.

[Christian_89]

Your diagnostic outputs provide some valuable insights into the issues. Here's a breakdown of potential causes and solutions:

### 1. **MTU Differences**:

The `dst_mtu` value for the US FortiGate is 1500, while for the Europe FortiGate, it's 1492. MTU mismatch can cause fragmentation, which may lead to performance issues.

**Solution**:
- Standardize the MTU value on both ends to ensure that they match.
- Use the `ping` command with the "Do Not Fragment" bit set to determine the maximum MTU size that can traverse the VPN without fragmentation.

### 2. **NPU Offload**:

The NPU (Network Processing Unit) offloads processing from the main CPU, offering better performance for specific tasks, like VPN traffic handling. The US FortiGate is offloading to the NPU (`npu_flag=03`), while the Europe FortiGate is not (`npu_flag=00`).

**Solution**:
- Make sure the configurations are indeed identical. Some specific configurations, like using certain security profiles, can prevent NPU offloading.
- Ensure the FortiGate in Europe has the necessary hardware (NPU) and it's functioning correctly. You might need to contact Fortinet Support for a deeper diagnosis.

### 3. **Latency**:

While 148ms latency might be stable, it's still relatively high, especially for real-time applications. However, for regular data transfer, it should be fine as long as it's consistent and there's no packet loss.

### 4. **Firmware Version**:

Both devices are running v6.4.14 build2093, which is good for consistency. However, there might be known issues or bugs in this specific build that affect VPN stability or performance.

**Solution**:
- Consider checking Fortinet's documentation or support forums to see if there are known issues with this firmware version related to VPNs.
- Consider upgrading to a newer firmware version if available, after checking its release notes and ensuring it doesn't introduce new issues.

### Additional Steps:

1. **Re-check Configurations**: Even if the configurations seem identical, double-check VPN configurations, especially advanced settings, to ensure they match.
2. **Physical Layer**: Check the physical connections, including cables and interfaces, for any signs of issues.
3. **Logs**: Examine the FortiGate logs around the times of disconnections. They might provide more insights into the reasons behind the drops.
4. **Diagnostics**: Run diagnostic commands (e.g., `diagnose debug application ike -1`) to get more detailed information about the VPN process.

Lastly, if the issue remains unresolved, consider reaching out to Fortinet Support for a more in-depth analysis. They can provide specific guidance tailored to your setup.

[sainc]

When using multiple iperf3 streams the bandwidht goes up to 150Mbps, while still when using one stream it is at 10Mbps at most. How should I approach tuning MTU Size