V5.2.1 TCP Reset Issue

[Dickie]

Hi All,

A heads up here.  We had some downtime for a bandwidth upgrade so at the same time we thought we would upgrade our 200D to V5.2.1. - which we have working fine elsewhere.  This worked fine in most aspects BUT:

An Ironport cluster and a VMware application running over an IPsec VPN would disconnect almost every 59mins 23 (ish) seconds.  Then reconnect.  VPN's would stay up no errors or other notifications.   It was so regular we knew it must be a timer or something somewhere - but we could not find it.  We did packet traces of the disconnects and found that at the time of the disconnects 'something' was causing the application(s) to reset all its sessions. 

We chased the ISP blaming the new link and looking at all manner of things that could be causing this, all to no avail.  Today we reverted to V5.2.0 Build 589 - so far we have not had one disconnect (3hrs).  The config on the Firewall is exactly the same. 

I hope this stops someone else pulling their hair out.    If the resets 'come back' I'll update this post.

[Uwe_Sommerfeld]

Thank for the info.

Did you vary the TCP timeout settings to verify it is not related to connection tracking in any way?

[Dickie]

Hi - indeed we did to BIG numbers but it made no difference.

[Dickie]

Errors just started happening again!!!!! - Sorry turns out this was the ISP resetting THEIR router...so it still might be an issue.  I'll get less trigger happy and let you know at the end of the day.

[emnoc]

If it's regular, than it sounds like a  timer. A question or two

Q1: you say it's 60min and regular?

Q2: And the traffic is only effected over the vpn

Q3: is this by any chance a SA time count down

I would monitor the  SA lifetime and then seen any thing relates.

[Dickie]

HI - yes our thoughts were a timer.  In answer to your questions:

Q1: you say it's 60min and regular?

Never quite the full 60mins - and regular(ish) sometimes we get a rogue fail.   Q2: And the traffic is only effected over the vpn

That's as far as we can track it as these application run over the VPN   Q3: is this by any chance a SA time count down

Possibly but we've played with all manner of settings, the VPN's never drop.  AND after reverting to the earlier code the problem appears to have gone away using the same config / vpns etc 

[emnoc]

Do you have logging on if the vpn is resetting in a 60min interval or less? if the SA are being torn down and restarting to a underlaying problem, it could cause the issues that you are seeing.

Also ensue the SA lifetime on near and far peers.

e.g

show vpn  ipsec phase2 "< phase2 interface name  >" | grep key         set keylifeseconds 3600

[Dickie]

emnoc wrote:

Do you have logging on if the vpn is resetting in a 60min interval or less? if the SA are being torn down and restarting to a underlaying problem, it could cause the issues that you are seeing.

Also ensue the SA lifetime on near and far peers.

e.g

show vpn  ipsec phase2 "< phase2 interface name  >" | grep key        set keylifeseconds 3600

Emnoc,

Thanks for the reply. Would be happy to do this - but since we have downgraded the firmware we are not seeing the issues so cannot track it.  It could be related to SA's but the settings are the same as they were before, timers everything.  So if it is, it is still related to the 5.2.1 release I would think?

[snobs]

Hello, one of our customers complained about TCP RST while downloading source code (file by file) with wget from a homepage.

(Take a look at http://forums.gradle.org/gradle/topics/unable_to_pull_from_maven_central_connection_reset_by_peer)

I could confirm the behavior with wireshark. When I download the files outside the firewall I do not see any TCP RSTs within wireshark. I have 311B with 5.0.7 firmware installed. (I do not have UTM/IPS enabled for that "Port 80"-policy)

Any ideas what could cause this?

Regards

[Dickie · Accepted solution]

Hi,

I can't find the relevant article but I believe you will find that is related to interface MTU / TCP MSS - try the following:

set tcp-mss 1380

set mtu-override enable set mtu 1454

These will be set on your WAN interface.  You can play with the sizes to optimise them.

Cheers