How to stop FortiGate to failback to 4G

[MadDog_2023]

Hi guys, 

I have FortiGate 30E.

4G device is connected to LAN1 interface.

Primary NBN link is connected to the WAN port.

Recently, NBN went down and FortiGate switch to 4G. 

However, when NBN link was restored:

a) FortiGate doesn't failback to NBN

b) If disconnect 4G (to make NBN active) and then reconnect, FortiGate fails over to 4G connection.

WAN has priority 0.

4G has priority 10.

Could you please help to stop it. 

[syordanov]

Hello MadDog_2023,

As far as i understand you have 2 links to internet - NBN and 4G, NBN has priority 0(more preferable than 4G). When NBN is down all outgoing traffic is moved to 4G which is normal, but when NBN is back again the outgoing traffic is not switched back to NBN?
If you have SNAT (Source NAT) for outgoing traffic when 4G is operating , then please check the KB's bellow :
https://community.fortinet.com/t5/FortiGate/Technical-Tip-Using-SNAT-route-change-to-update-existing...
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-Routing-Changes-and-SNAT-snat-route-...

When you have session with SNAT , the action that FortiGate takes after a routing change(like adding new better route NBN for example) depends on the snat-route-change settings.

If you have 'disable' the default one, when route changes sessions with SNAT continue using the same outbound interface as log as the old route is still active(4G interface and associated routes with that interface). If you have 'snat-route-change enable' , when route changes (add route with better metric or in your case NBN is up again) , Fortigate flushes routing information from existing SNAT session, so the existing SNAT sessions can use any new best route.

So my suggestion is to change 'snat-route-change' to ' enable'   if you have SNAT for outgoing traffic and wants to change outgoing interface back to NBN when is recovered .

[MadDog_2023]

Hi syordanov,

Thanks for your reply.

As far as i understand you have 2 links to internet - NBN and 4G, NBN has priority 0(more preferable than 4G). When NBN is down all outgoing traffic is moved to 4G which is normal, but when NBN is back again the outgoing traffic is not switched back to NBN?

In addition to that when 4G cable is unplugged and NBN is active as soon as 4G is plugged back connection switches to 4G.

I enabled snat-route change but Fotigate still switches back to 4G.

Attached screenshots of the configured static routes.

[gfleming]

What version of FortiOS?

Can you show the output of "get router info routing-table all"?

Are you using SD-WAN or Link Load Balancing at all? Have you considered it?

Do you have any link monitors set up that dynamically update your static routes?

[MadDog_2023]

Hi @gfleming 

What version of FortiOS?

v5.6.2 build1486 (GA)

Can you show the output of "get router info routing-table all"?

COMPANY-FG # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default

S* 0.0.0.0/0 [10/0] via 60.XXX.XXX.XXX, ISP VLAN100
C 10.10.10.0/24 is directly connected, Guest Wireless
C 60.XXX.XXX.XXX/30 is directly connected, ISP VLAN100
C 192.168.125.0/24 is directly connected, lan

Are you using SD-WAN or Link Load Balancing at all?

Neither SD-WAN nor load balancing is configured on the FortiGate.

Do you have any link monitors set up that dynamically update your static routes?

I don't think so. I would say no.

[gfleming]

Can you upgrade to 6.2? Once you've done that you could configure SD-WAN to handle the failover to 4G when needed using health-checks on the WAN links.

Can you also help clarify something: does the link ever come back on its own if you just leave it?

I wonder if you're hitting an issue with existing sessions choosing to use the 4G but new sessions will use the NBN?

Also I feel like you must be using some form of link monitoring because unless the link goes down on the FortiGate the default route is statically configured and will stay in the table. I assume you failover to 4G even if the physical link to NBN is up and green?

See page 109 for details here:

https://docs.fortinet.com/document/fortigate/5.4.13/fortios-handbookhttps://docs.fortinet.com/document/fortigate/5.4.0/cli-reference

Best course of action IMO is to upgrade to 6.2 and leverage the SD-WAN features which will be a bit better to manage. Plus 5.4 is way out of support and may be vulnerable due to a lack of updates.

[MadDog_2023]

Hi @gfleming,

Can you also help clarify something: does the link ever come back on its own if you just leave it?

I wonder if you're hitting an issue with existing sessions choosing to use the 4G but new sessions will use the NBN?

This is what happens now. 

At the moment 4G link is disconnected from FortiGate because as soon as the 4G modem cable is plugged in to the FortiGate the router switches or fails over to 4G instead of remaining on primary WAN connection. 

Also I feel like you must be using some form of link monitoring because unless the link goes down on the FortiGate the default route is statically configured and will stay in the table. I assume you failover to 4G even if the physical link to NBN is up and green?

I checked  in CLi and as far as I can see link minotoring is not configured. 

Facade-FG # show full-configuration system link-monitor
config system link-monitor
end

I will try to update FortiOS and update you here. 

[gfleming]

Interesting. Can you show the output of the routing table when the 4G modem is plugged in?

[MadDog_2023]

Resolved by increasing Distance value on the interface 4G modem is connected to.