slow vxlan speed over ipsec

[Stefano_iso]

Hi,

we have speed problem on vxlan over ipsec connection between FGT100F and FGT60F. The tunnel works fine and the traffic is ok but the speed of a simple test with iperf is about only 2MB/s. (both sites have 1Gb/s symmetrical IPS connection). Any suggestions?

Thanks

Stefano

[gfleming]

What is the result of the iPerf test over the 1Gb/s links without IPsec/VXLan in the mix? In other words are you sure this is a VXLAN issue?

Also what are your Iperf settings—sometimes base/default settings do not give you the best speeds. Such as using UDP vs TCP, window sizes, parallel threads, etc...

[Stefano_iso]

The tests performed are the following:
iperf tests from windows to windows on vxlan over ipsec via iperf with various windows sizes are always no higher than 15-20 Mbits/sec.

iperf tests from fortigate console (via "diagnose traffictest run" therefore excluding vxlan/ipsec) to the same windows pc on the other side gives the following results:
with default settings no more than 30 Mbits/sec , with various windows sizes (until 8192k) I can get to 330Mbits/sec.

Thanks

Stefano

[aionescu]

Hi @Stefano_iso ,

Can you share a .pcap of the traffic captured on both clients? 

What is the MTU of the tunnel? 

[Stefano_iso]

Hi, aionescu,

i share print screen of the test:

pcap over the vxlan

pcap_vxlan

diagnose traffic test over wan from fortigate with different windows size

nperf_Test_on_fortigate

pcap of diag traffic from fortigate with windows size 2048k

pcap_size_2048

pcap of diag traffic from fortigate with default

pcap_default_setting

the MTU of tunnel is : SA: ref=6 options=10226 type=00 soft=0 mtu=1438

the MTU of software switch of vxlan is 1450

thanks,

Stefano

[gfleming]

What does the CPU look like on the 60F when you are transferring data over VXLAN? is utilization high?

[Stefano_iso]

I tried with "diag traffictest" to PC on hardware interface but I get the same low result.
and as you have seen "diag traffictest" to the same pc on software switch is capable of getting the desired results via windows size. ( vxlan is encapsulated on Loopback interface ).

Stefano

[aionescu]

@Stefano_iso hard to tell just from this information. What is the npu flag of the tunnel?

You can find with the command "diagnose vpn tunnel list"

[Stefano_iso]

Hi @aionescu , 

the flag of vpn tunnel on each firewall is "npu_flag=03" and the MTU is "SA: ref=6 options=10226 type=00 soft=0 mtu=1438".

The cpu of each firewall when i run some tests with iperf stays around 1-2% on each side.

Stefano

[Stefano_iso]

Hi ,

i'm back to this problem. I ran further tests excluding the vxlan so now we have simple ipsec tunnel ( created with vpn sdwan wizard) but I still get the same speed (slow). I've tried various MTU/MSS configurations with no improve. Any suggestions on further tests I could do? Thanks,

Stefano