How to force SDWAN IPSec VPN Down when quota reachs its end - Starlink maritme 5TB quota

[FB]

How to force SDWAN IPSec VPN Down when quota reachs its end - Starlink maritme 5TB quota

The scenario: small office with Starlink+CG-NAT+DHCP, FG100, FortiOS 7.4 and remote office/DTC, regular ISP with static IP

I have two Starlink antenas, with a 5TB/Month quota, something around 168GB/day.

Besides that, the Starlink connectins are CG-NAT ones, with DHCP, so, not only the IP sometimes changes, but also, the NAT is under another NAT, so we need to use NAT-T on "forced" and also, use passive mode on remote datacenter, as I am using DDNS because the Starlink side needs it. (note: dial-up is not supported under SDWAN)

Because of the number of users and devices, besides controlling very strictly the traffic, we still need to "cap" the in/out max bandwidht to around 20Mbps to make sure that the Quota is not reached before the months ends

We did several tests, when something happens with the link, antenna turned off, cabling problem, DHCP error, remote SIP lack of communication error, it triggers the ISPEC to be down, and the SDWAN makes its job and everything works well.

As we have two antennas, we ´ve created a SDWAN with two IPSec VPN tunnels, one for each antenna. The remote/DTCe side is also a Fortigate, but not Starlink, instead regular ISPs, to serve as vpn endpoints with static IP

But stil, sometimes, the quota ends before the end of the month, and it´s when things get weird! As the quota ends, things like electrical crrent, link up, antenna up, no communication error happening, the tunnel still is up and running, so the ipsec tunnel neve got down and therefore, the SDWAN it doesn´t failover and everything stops. So, even with no quota available, still, the VPN is considered (on both sides) up and running, so, te SDWAN it doesn´t allow any communication to flow properly. Special Note: performace SLA is configured, with a loopback interface on each side, to test both tunnels and making sure that SDWAN culd detect the failure of the VPN channel, but, again even after end of quota, the tunnel is still up.

DTC/Main Static IP side: set passive-mode enable, phase1 keylife 28800, phase2 keylife 3600, set nattraversal forced
Small office/Starlink site: set passive-mode disable, phase1 keylife 28800, phase2 keylife 3600, set nattraversal forced

Besides that, some parametrs are as follows, i Phase1 on both sides:

monitor :
monitor-min : 0
auto-negotiate : enable
idle-timeout : disable

But for several months we´ve noticed that when the Quota ends, THE TUNNEL is still fully "functional", tehrefore, the SDWAN it does not failover properly

So, I started to think to use some special/custom parameters to tailor the system to my needs

So I got tha idea to change the keepalive to DISABLE
and maybe also the auto-negotiate also to DISABLE

keepalive : disable
auto-negotiate : disable

This could help to solve my issue?

Any other idea?

[filiaks1]

Have you reviewed shape policies that will drop the traffic over the limit?

Shared traffic shaper | FortiGate / FortiOS 7.6.3 | Fortinet Document Library

Traffic shaping | FortiGate / FortiOS 7.6.3 | Fortinet Document Library

Other than that if this is a forti security fabric you could try triggering automation or if not send the forti logs to external server that can use the fortigate API to then trigger a script to stop the interface. My examples that you can modify:

Solved: Re: Restart Fortigate http/gui processes automatic... - Fortinet Community

Fortigate CPU/memory leak automation technical tip - Fortinet Community

Old post but the traffic logs can be used:

Can We Alert Traffic Shaping Rule? - Fortinet Community

[FB]

To use shaper to drop the traffic over the limite is not an option, for several reasons: the 5TB limit is overall, for ANY traffic produced/generated, the 5 TB limit  is up+down, the traffic that goes though ipsec has overhead not related to what is being transported through the vpn, there is an IPSLA traffic using the quota as well, and finally, there are different interfaces using the quota besides the IPsec vpn, like regular internet access trhough another SDWAN, as I have 2 antennas, I use for internet access AND corporate access on both SDWANs

And sending logs to an external system is not an option

I was hoping that the dpd/auto-negotiate/keep-alive might be more useful in this particular case, as the defaults are tied to standard environments, which is not the case here

[filiaks1]

That are the options I am aware of and usually even small companies have if not SIEM a systlog server as for specific use cases like yours API Automations (Python, Terraform, Forti stiches and triggers) is usually the go to.

[FB]

Yes, sure, maybe it could be done, but now we´re talking about a solution which requires skills that we don´t have and of course, included that there is no budget to hire someone to do

I´m puzzled by why the "auto-negotiate" and "keep alive" disabled are not being able to for the tunnel to get down, whe the quota reachs its limit and if not really, if there are other parameters that could be configured to do so