Skip to main content
TheJaeene
TheJaeene
New Member
January 26, 2015
Solved

sFlow breaks IPsec Traffic (Offloading) on a 60D !?!

  • January 26, 2015
  • 9 replies
  • 14572 views

Hello Folks,

 

today I configured sFlow on a 60D, afterwards I noticed high ESP-Errors (Replayed-Packets) on a far-end VPN Tunnel Fortigate.

For me it seems that enabling sFlow on this small SoC Boxes breaks the IPSec "Acceleration".

 

The Box was running 5.2.2 and the sFlow Polling-Interval was configured quite conservative.

CPU-Load was O.K.

 

Has anyone seen similar issues yet?

 

 

Regards,

 

Jan

Best answer by Christopher_McMullan

I'll weigh in here. Neat to research the issue, so thank you for the opportunity!

 

The problem is an existing offloading situation when sFlow is applied. What initially seems counter-intuitive is that the issue is not sFlow-specific. Traffic *was* being offloaded historically, but when enabling sFlow, you are essentially queuing new traffic to be forced through the kernel instead (i.e., the CPU).

 

A reboot fixes the issue because, when the tunnels come back up, they are all functioning through the CPU - no more offloading. You may have noticed higher baseline CPU usage, depending on your traffic patterns.

 

A bug is open, but it could always use more customer reporting and buy-in for a solution. If you end up opening a ticket with TAC, quote the number here so that I can make an internal note referencing the bug.

9 replies

emnoc
emnoc
New Member
January 26, 2015

Open a case with  TAC. I'm curious if you use netflow  does the same thing happen?

 

And are you 100% sure the ESP Replayed-Packets is not cause by something else?

 

And is the far-end device a cisco ?

 

If your bored you could probably do a small packet capture of the ESP  traffic and check the sequences.

TheJaeene
TheJaeeneAuthor
New Member
January 26, 2015

Hi emnoc,

 

 

once I activated sflow sampling on the Interface (WAN, not Tunnel IF) the ESP-Errors were visible on Remote Fortigates (80C, 100D). All VPN Traffic going through that Interface was very sluggish.  (Several Tunnels)

 

Disabling the sflow sampling didn´t help, only a reboot cured the behaviour.

 

And yes, I could reproduce it several times

 

 

Once our FGT-Support account is repaired, I´ll open a Ticket.

 

Regards,

Jan

 

 

emnoc
emnoc
New Member
January 26, 2015

So  you have replay enabled on your vpn-tunnel ph2 cfgs? This could be a bug or something else going on. TAC should look into it and see what's going on.

 

What happens if you enable sFlow on the bigger fortigates, do you see errors on the opposite end ?  Might be hardware related on the smaller units.

TheJaeene
TheJaeeneAuthor
New Member
January 26, 2015

I also assume that this is a SoC FGT related Problem.

Replay-Detection ist enabled in the Ph2 Configs.

 

Right now our bigger Lab Units are not available for testing.

 

But during Research I found some old npu/nplite related stuff that I now disabled.

 

    set enc-offload-antireplay enable *gone now*     set offload-ipsec-host enable *gone now*

 

 

I´ll check again if the problem still persits without offloading antireplay to the nplite.

Christopher_McMullan
Staff
Christopher_McMullanAnswer
Staff
January 26, 2015

I'll weigh in here. Neat to research the issue, so thank you for the opportunity!

 

The problem is an existing offloading situation when sFlow is applied. What initially seems counter-intuitive is that the issue is not sFlow-specific. Traffic *was* being offloaded historically, but when enabling sFlow, you are essentially queuing new traffic to be forced through the kernel instead (i.e., the CPU).

 

A reboot fixes the issue because, when the tunnels come back up, they are all functioning through the CPU - no more offloading. You may have noticed higher baseline CPU usage, depending on your traffic patterns.

 

A bug is open, but it could always use more customer reporting and buy-in for a solution. If you end up opening a ticket with TAC, quote the number here so that I can make an internal note referencing the bug.

kelv1n
kelv1n
New Member
March 8, 2015

OMG THANK YOU!!!

 

I've been going insane for the past several days, trying to figure why our 2 sites connected via a 1Gbp/s link would carry 600Mb/s in 1 direction, but only 50-150Mb/s in the other (it is quite amazing how much performance the NPU seems to deliver).. turns out traffic was not offloading to the NPU, and removing our Netflow and Sflow configs fixed this.

Delta
Delta
New Member
June 3, 2015

I have a 90D ... just enabled sflow and ... tunnels stopped passing traffic.  Is there a workaround yet ?  (5.3)

Christopher_McMullan
Staff
Christopher_McMullan
Staff
June 4, 2015

Once it's done, it's done, unfortunately. The workaround is to reboot the FortiGate at this point.

 

The bug is open, but not fixed yet. I'd encourage you to open a TAC case to put your weight behind a quicker schedule for the fix. Not to say there is a schedule, but something about squeaky wheels comes to mind.

Delta
Delta
New Member
June 4, 2015

Do you have a bug # to reference for TAC

Terms & ConditionsAccessibility statement