FG60E - FG60E IPSec Tunnel Slow

[remosito]

Hi everybody,

our Site2Site IPSec VPN Tunnel between two FTG-60E on 5.4.4 is slow and I have run out of stuff to try.

Both Sites are connected via Gbps Fiber Internet. Iperf3 to a public Iperf Server gave 600+ Mbps results on both ends. So we know our Internet uplinks are good. But our IPSec Tunnel tops out at 130-ish Mbps.

I have tried from tips I read in various places:

 - deleted the udp-dns session helper thingie

 - took all internal1-8 out of the internal hardware switch

 - tried different enc algos

 - set local-gw

here the sanitized relevant parts of the config from MainSite FTG-60E

config system npu
    set enc-offload-antireplay enable
    set offload-ipsec-host enable
end

config system interface
    edit "wan1"
        set vdom "root"
        set ip a.a.a.a 255.255.255.248
        set allowaccess ping ssh
        set type physical
        set estimated-upstream-bandwidth 1000000
        set estimated-downstream-bandwidth 1000000
        set role wan
        set snmp-index 1
        set secondary-IP enable
        config secondaryip
            edit 1
                set ip a.a.a.b 255.255.255.255
            next
        end
    next


    edit "internal1"
        set vdom "root"
        set ip p.p.p.p 255.255.255.0
        set allowaccess ping https ssh
        set type physical
        set snmp-index 11
    next

config vpn ipsec phase1-interface
    edit "VPN-BRN-LSN"
        set interface "wan1"
        set local-gw a.a.a.a
        set peertype any
        set comments "VPN: VPN-BRN-LSN (Created by VPN wizard)"
        set remote-gw f.f.f.f
        set psksecret ENC secretsauce
    next
end

config vpn ipsec phase2-interface
   edit "VPN-BRN-LSN"
        set phase1name "VPN-BRN-LSN"
        set proposal aes256-sha256
        set dhgrp 14
        set comments "VPN: VPN-BRN-LSN (Created by VPN wizard)"
        set src-subnet p.p.p.p 255.255.255.0
        set dst-subnet q.q.q.q 255.255.255.0
    next

and here some diag output from the same 60E:

diagnose vpn ipsec status
All ipsec crypto devices in use:
NP6LITE_0
        null: 0 0
        des: 0 0
        3des: 0 0
        aes: 1682560 1894400
        aes-gcm: 0 0
        aria: 0 0
        seed: 0 0
        null: 0 0
        md5: 0 0
        sha1: 6784 8192
        sha256: 1675776 1886208
        sha384: 0 0
        sha512: 0 0
NPU HARDWARE
        null: 0 0
        des: 0 0
        3des: 0 0
        aes: 33331 0
        aes-gcm: 0 0
        aria: 0 0
        seed: 0 0
        null: 0 0
        md5: 0 0
        sha1: 544 0
        sha256: 32754 0
        sha384: 0 0
        sha512: 0 0
SOC3:
        null: 0 0
        des: 0 0
        3des: 0 0
        aes: 20 32
        aes-gcm: 0 0
        aria: 0 0
        seed: 0 0
        null: 0 0
        md5: 0 0
        sha1: 1 2
        sha256: 18 30
        sha384: 0 0
        sha512: 0 0
SOFTWARE:
        null: 0 0
        des: 0 0
        3des: 0 0
        aes: 0 0
        aes-gcm: 0 0
        aria: 0 0
        seed: 0 0
        null: 0 0
        md5: 0 0
        sha1: 0 0
        sha256: 0 0
        sha384: 0 0
        sha512: 0 0

get system npu
enc-offload-antireplay: enable
dec-offload-antireplay: enable
offload-ipsec-host : enable


diagnose vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=VPN-BRN-LSN ver=1 serial=3 a.a.a.a:0->f.f.f.f:0
bound_if=3 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/8 options[0008]=npu
proxyid_num=1 child_num=0 refcnt=20 ilast=13 olast=13 auto-discovery=0
stat: rxp=1715632 txp=1911092 rxb=833611779 txb=1426710396
dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=1101
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=VPN-BRN-LSN proto=0 sa=1 ref=5 serial=1
  src: 0:p.p.p.p/255.255.255.0:0
  dst: 0:q.q.q.q/255.255.255.0:0
  SA: ref=6 options=2e type=00 soft=0 mtu=1438 expire=11928/0B replaywin=1024 seqno=29811 esn=0 replaywin_lastseq=00025e80
  life: type=01 bytes=0/0 timeout=43174/43200
  dec: spi=257f4fb4 esp=aes key=32 secretsauce
       ah=sha256 key=32 secretsauce
  enc: spi=8574257e esp=aes key=32 secretsauce
       ah=sha256 key=32 secretsauce
  dec:pkts/bytes=155266/49170368, enc:pkts/bytes=173054/97413586
  npu_flag=03 npu_rgwy=f.f.f.f npu_lgwy=a.a.a.a npu_selid=0 dec_npuid=1 enc_npuid=1

kinda all looks as it should be to me.

I can post the same info for the other FGT-60E if needed. Or any other output or config parts....

any help greatly appreciated

[Ricky_Fleming]

One question answered:

remaining questions I have not answered yet: - Would the NP6Lite ASIC on the 60E get higher throughput on the IPSEC VPN?

Absolutely! Almost double.

[Toshi_Esumi]

All:

I have to apologize for our false report of FG60E IPSec VPN performance issue. It turned out to be the test bed we ran iperf test was not proper and misled to the wrong conclusion. After we re-test it under a proper test environment, it didn't exhibit much difference between individual internal interface, like internal1, and hard-switch, at about 800-900Mbps through GigE test enviroment.

Only soft-switch showed a lower bandwidth at about 400Mbps, which was expected.

Toshi