HA - Monitoring

[Domsi]

Hi. It is the first time I have setup a FortiGate 100F Cluster (FortiOS 6.2.3). I followed the tutorials for "HA" and selected "active-passive" for the FortiGate. I have setup the "ha1, ha2" interfaces an connected them. Then I have selected the "wan1" interface for monitoring. Basically the HA-Settings are working - I have got the master and the slave unit. If "wan1" loosing the connection (pulling cable out / or restart of master) it switches to slave which becomes new primary. But if "wan1" of old primary is restored I will get no connection from outside - only if I'm pulling out "wan1" cable of slave.

F1 = master -> monitoring "wan1" F2 = slave -> monitoring "wan1"

F1 > wan1 is lost > F2 = primary, F1 = slave ... all connections are now running correctly over F2. Then F1 > wan1 is restored > F1 = primary, F2 = slave ... I can only connect to F1 via MGMT (MGMT of F2 is not responding).. but I'm not able to ping the public IP of wan1, and I'm also not able to connect via SSL-VPN. I have pull out "wan1-cable" of F2 > then I'm able to connect to the F1 from public (ping on public IP, VPN)...

Is there something I have to consider or there are some settings missing?

[Toshi_Esumi]

If you want the previous master to take the master roll over when its wan1 recovered, you need to set priority on that unit higher to override. 

https://help.fortinet.com/fos60hlp/60/Content/FortiOS/fortigate-high-availability/HA_FGCP_override.h...

[Domsi]

F1 => Priority = 250

F2 => Priority = 200

(Screenshot attached) --> edge-primary = master = higher serial number

The F1 becomes, after restored "wan1", correctly the master. I can only connect to F1 via MGMT (F2 MGMT not respondig), the ha status (GUI and CLI) shows F1 as master. But I can't reach the FortiGate from public (no ping on public IP, no VPN connection possible). I have to pull out "wan1 cable" of F2 => now I can access the F1 from public. It looks like that F1 = primary but F2 is still active > because if I'm connected to an internal port of the F2 the traffic still goes over this F2 => Ping to internal LAN port is possible, traffic to the inernet is still possible. The same happens If I reboot the F1. It comes up again, becomes the master and I can never connect from public. I have to pull out "wan1 cable" of F2 => then I can connect again.... they are in an external datacenter, so I have to drive there every time...

[Toshi_Esumi]

After setting priorities then enabling override, what's in under "config sys ha" now?

You can see what's going on on either side with "diag sys ha history read" with timestamps. They can probably tell why they don't fail back.

[Domsi]

Hi. Now I have enabled the override setting. As I can see F1 becomes correctly the master, I can also connect via MGMT-Interface. BUT it is not accessible from public. Same as before:

[ul]

F1 master > pull out WAN of F1 > F2 = master (able to PING and connect with VPN).

restore WAN on F1 > F1 = master, but non of both fortigates are accessible from public (permanent PING stops responding, no VPN connection possible)

I have to pull out WAN of F2 > now F1 accessible[/ul]

I have attached the CLI output (config sys ha, diag sys ha history read):

[ol]

F1 = FG100XXXXXXXXX52

F2 = FG100XXXXXXXXX32[/ol]

As you can see F1 becomes correctly the master. But it looks like as F2 WAN is still "online" > which will result in two public interfaces with the same IP.

[Toshi_Esumi · Accepted solution]

You're not enabling "ha-mgmt-status" to use out-of-band MGMT interfaces. But it shouldn't affect to the WAN connectivity issue. Aslo you're not enabling "session-pickup". I would enable it for faster swap-over.

But otherwise, I don't see particular reasons for the behavior unless the uplink switch, which is terminating both wan1s is affecting to it.

I would open a ticket at TAC to get it looked into. Could be 100F specific with 6.2.3.

[Domsi]

Thank you, I have created a ticket. I will update this thread if there are any results.

[Domsi]

Now we found out (togehter with TAC Engineer) that this isn't an issue of the FortiGate. The ISP is blocking the "gratuitous arp" for security reasons (housing switch where multiple customers located, they block the gratuitous arp so that a foreign device can't allocate the mac address). The ISP will check if they can open this behaviour for my housing-system. They alternative solution is to disabel the ha override and set an equal priority, so that the last master stays the last master.

[ede_pfau]

...which often is preferable anyway, as it minimizes the traffic disruptions due to failover.