Can consistently ping over IPSEC, but sometimes can't run programs

[ikmarwright]

I was in a remote office today where we have a 60E talking to a 100E (both 6.0.9). Using PRGT I know the lines have been consistently up. No significant latency issues, dropped packets or such.

But sometimes when I connect to the server behind the 100E (where I have a SQLExpress database, a Splashtop streamer and shared drives) , I get timeout errors -- like it can't see the server. But I can ping the server (with 22ms responses) at the same time it won't connect via any other software. 

Seconds later, everything works again.  And I haven't been able to repeat the issue if I use a SSL-VPN connection from my remote office. All communication is over a FAP221E (newest firmware) -- the remote office is all WiFi. And I can't force the issue to repeat. It could happen twice in ten minutes and then not happen for three hours.

We do have an ongoing issue in both offices (two different ISPs) where every couple days the modems have to be reset -- and neither ISP knows why. I was going to run a trace, but then it stopped happening (for now). It's driving staff nuts, and because of COVID my access is VERY limited. 

The one thing (that was supposed to be temporary) is the internal DNS server is behind the 100E, but considering half my software connects by IP, that shouldn't be the issue.

Anyone have any ideas?

[ede_pfau]

just as a hint...

ping working but 'real' protocol's payload not - sounds like an MTU issue. Lately, I encountered a similar situation and the culprit was the CPE modem's speed setting - 100/full. Switched the FGT port to 100/full and everything worked.

You could try

- using ping, disallow fragmentation ('ping -f')

- run ping with increasingly bigger packets, up to 1500 bytes

1500 should never work over a VPN, you'll probably will lose all packets at 1372 or such.

[ikmarwright]

Hi Ede

Thanks. :)

The modem and FGT are both 1000/full. And we have no problems with SSL-VPN over the same infrastructure (even though you are right in that it fragments after 1360) -- it's only the IPSEC that seems to fail.

Anyway I ran ping -f -l XXXX <ip> and it's inconsistent across IPSEC. Is that normal? 90% of the time a 1470 size packet does not need to be fragmented, but the other 10% of the time it does (at these times I have to drop it to 1400). Does that suggest an issue with a hop between the two sites? Or should I just drop the MTU to 1400 and not worry around it?

Dave

[ede_pfau]

Strange, I agree.

Well, if it fixes the problem, reduce the MTU to 1400. This should not affect throughput much (not at all in comparison to frequent failures). A working solution is not always the most satisfying solution.

[ikmarwright]

Thanks, Ede

It works a bit better. Still too many problems with our ISP to work around our communication issues completely, but losing connection once a day is better than five or six times. :)

[ede_pfau]

Meanwhile, I've read up on MTU issues on IPsec VPN and FTNT states that the MTU cannot be changed on VPN links. Only on the underlying physical port. MTU will be determined dynamically and can be shown by

diag vpn tunnel list

in the line starting with "SA:".

And with

diagnose netlink interface list <Phase 1 name>

you'll see error counters ("rxe= ...txe=").

As a last resort you may try

config sys global

   set honor-df disable

This will prevent dropping fragmented packets in IPsec traffic (default: enabled).

In my case it didn't have any positive effect, though but YMMV.