Fortigate IPsec tunnel slow TCP, fast UDP

[WHATisIPV4]

Good afternoon all,   I've inherited a setup that has two locations. Fortigate1 (WAN speed 1000Mbps up/down) Fortigate2 (WAN speed 200Mbps up/down)   I've ran into an issue where file transfers between the two are very slow. I tested using iperf3 and I get about 15-30Mbps no matter which side is sending/receiving. If I test using UDP, it maxes out bandwidth both ways. The tunnel is using AES128-SHA256 for phase1 and phase2. DH Group 2. I've opened a ticket with fortinet but they are pretty stumped on this. I've read similar accounts online where this was fixed by changing MTU and/or tcp-mss-receiver/sender on the policies. I've tried setting a lower tcp-mss on the incoming/outgoing policies but this has no effect. Here is an example of a similar issue I found on reddit: https://www.reddit.com/r/networking/comments/9vep0k/ipsec_speeds_are_trash_or_im_doing_something/

[ad]

What did you set the MTU to? IIRC for AES128+SHA1 it needs to be 1387, so you would need something smaller than that to prevent fragmentation. I did a few minutes googling but didnt find a definitive answer. As a guess, maybe 256-160 = 96 bits (SHA1 is 160 bits, SHA256 is 256 bits), so dividing by 8 to get bytes is 12, so maybe 1375?

Other than that, TCP gets slower as latency rises, so you might need to increase the TCP window size if the latency is high.

I remember scp/sftp being slow without the HPN patches, but that was many years ago.

[pmandava_FTNT]

Setting mss on the policies to 1350 helps sometimes.

[emnoc]

Don't guess do a path mtu ping and then adjust afterwards

ping -f -l 1488 x.x.x.x

Adjust downwards until the packet needs fragmentation errors goes away

Ken Felix