Overhead for IPsec aggregation at SD-WAN

[HT_JDC]

Dear Experts,

I want to know the theoretical overhead of  bandwidth when IPsec aggregation is used at SD-WAN.

For example, pure WAN1: 5Mbps, pure WAN2: 3Mbps and pure WAN3: 2Mbps.

Total is 10Mbps. All WANs is IPsec-aggregated as SD-WAN.

   PC1 -- FG1(IPsec Aggregated SD-WAN) -- (network) -- FG2(IPsec Aggregated SD-WAN) -- PC2

We tried a little. 8-8.5Mbps between PC1 and PC2. I do not judge if it is reasonable.

Any comments are appreciated.

[Toshi_Esumi]

Have you measured bandwidth between two FGTs 1) just over the internet on each port, and 2) a single IPsec between them on each interface? What are those numbers?

I think 2) is the deciding factor. And, I wouldn't expect much overhead by "IPsec aggregate" or "SD-WAN".
Besides, it's difficult to measure the total bandwidth over SD-WAN with 2 paths. Even if you set them to load-balance, if the source IP and destination IP is the same, the measuring traffic would take only one side of those paths.

Toshi

[HT_JDC]

Dear Toshi,

Thanks for your reply. I should have clarified more.

The values like 5Mbps means "1) just over the internet on each port" which you are saying.

We use VXLAN so that PC1 and PC2 belong to the same network. iperf is used between PC1 and PC2. Thus, no load-balance exists.

Any additional comments would be appreciated.

[Toshi_Esumi]

In other words, you need to use iPerf test between two fortigates, for:
1) wan1 - wan1 and wan2 - wan2
2) IPsec1(interface) - IPsec1 and IPsec2 - IPsec2
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-Configure-FortiGate-as-speed-test-ip...
Which would be the ceiling of the max bandwidth on each path. 

I'm not sure how FGT decide which path to pass VXLAN traffic. I'm guessing that's depending on how it's configured. Afterall VXLAN need to be forwarded from IP to IP over IPsec, right? I'm not so familier with VXLAN itself. 

Toshi

[HT_JDC]

Hello,

I did not know iperf exists within Fortigate.

We try to see if we can get useful information.

Anyway, theoretical information would be practical to us.

Best regards,

[Toshi_Esumi]

So VXLAN seems to a technology encapsulating ethernet packets over UDP.
 https://en.wikipedia.org/wiki/Virtual_Extensible_LAN

So every single ethernet frame needs to be encapsulated with at least 28 bytes header (8+20). Then IPsec adds 30 - 70 bytes depending on mode and protocol. There are some overhead calculators on the internet for IPsec, like this one:
https://ipsec-overhead-calculator.netsec.us/

Toshi