slow vxlan speed over ipsec

What is the result of the iPerf test over the 1Gb/s links without IPsec/VXLan in the mix? In other words are you sure this is a VXLAN issue?

Also what are your Iperf settings—sometimes base/default settings do not give you the best speeds. Such as using UDP vs TCP, window sizes, parallel threads, etc...

The tests performed are the following:
iperf tests from windows to windows on vxlan over ipsec via iperf with various windows sizes are always no higher than 15-20 Mbits/sec.

iperf tests from fortigate console (via "diagnose traffictest run" therefore excluding vxlan/ipsec) to the same windows pc on the other side gives the following results:
with default settings no more than 30 Mbits/sec , with various windows sizes (until 8192k) I can get to 330Mbits/sec.

Thanks

Stefano

Hi @Stefano_iso ,

Can you share a .pcap of the traffic captured on both clients? 

What is the MTU of the tunnel? 

Hi, aionescu,

i share print screen of the test:

pcap over the vxlan

pcap_vxlan

diagnose traffic test over wan from fortigate with different windows size

nperf_Test_on_fortigate

pcap of diag traffic from fortigate with windows size 2048k

pcap_size_2048

pcap of diag traffic from fortigate with default

pcap_default_setting

the MTU of tunnel is : SA: ref=6 options=10226 type=00 soft=0 mtu=1438

the MTU of software switch of vxlan is 1450

thanks,

Stefano

I tried with "diag traffictest" to PC on hardware interface but I get the same low result.
and as you have seen "diag traffictest" to the same pc on software switch is capable of getting the desired results via windows size. ( vxlan is encapsulated on Loopback interface ).

Stefano

@Stefano_iso hard to tell just from this information. What is the npu flag of the tunnel?

You can find with the command "diagnose vpn tunnel list"

Hi ,

i'm back to this problem. I ran further tests excluding the vxlan so now we have simple ipsec tunnel ( created with vpn sdwan wizard) but I still get the same speed (slow). I've tried various MTU/MSS configurations with no improve. Any suggestions on further tests I could do? Thanks,

Stefano

Hi Stéfano,

I did a recent layer 2 tunnel setup (ADVPN Less). I had to activate add the following configuration :

config system global

set honor-df disable

end

And add option in my phase1-interface Tunnel

set ip-fragmentation pre-encapsulation

I hope this will fix your problem as well.

Best regards,

Wondering if you ever got this resolved?   I have the same problem with IPsec + VXLAN on a combination of 40F, 81E, and virtual VM04.  Tried every combination, initially I thought it was an MTU size issue due to IPSEC + VXLAN overhead. But I am actually able to send ping with DF bit set at 1472 payload which is the correct value using 1500 byte max minus the 8 byte ICMP and 20 byte IP header.     

I have not been able to figure this out after 2 weeks of vxlan over ipsec full mesh testing between 4 different geographic locations about 12ms apart. So i doubt its the latency because to the Internet on these same firewalls to speedtest dot net, I am getting 900Mbps on a 1 Gbps Internet circuit.   

Also like you, my CPU is not an issue. The maximum it will reach for the CPU is approximately 30% (usually less on my VM04 ) and 8% CPU on my 40F firewall.   Both platforms hardware and virtual appliance transport will not exceed 60-70 Mbps when it has Internet circuits 1 Gbps.

Wondering if you ever figured it out?   I'm at a loss at the moment and so few people are doing this function so its difficult to find any expert references with experience.