Upgrade Firmware on HA cluster issue on fail over

[Virgule59]

Hi Guys,

I've upgrade my cluster of 2 Fortigates 100E from 5.4.2 to 6.0.9 and follow upgrade path to do this.

Cluster Ha are configure like this

4 VDOM + Root Vdom

2 vclusters

Root Vdom and 2 Vdom are on vcluster 0

2 others vdom are on vcluster1

Primary Master is master on vcluster0 and slave on vcluster1

Secondary Slave is master on vlcuster1 and slave on vcluster0

Pimary master and secondary slave are on 2 differents datacenters distants from less than 62 Miles and ping less or equal to 1 ms.

HA1 and HA2 are on differents vlan

As i'm in maintenance Windows, before i'll launch first upgrade i'll reboot the cluster, wait for cluster checksum are Ok, and let's go for upgrade.

If i understand the KB on upgrade the cluster

[ol]

Connect on master, backup config, launch update update automatically do this operation : [/ol][ul]

load and upgrade firmware on secondary slave 

reboot slave

slave become master

load and upgrade  firmware primary master

reboot master

Primary master become master, slave go back to secondary slave (depend on priority, ages...)[/ul]

The first upgrade and other take place like that,

1) load and upgrade slave

[ul]

Lost secondary site, no ping on management interface, no ping on device[/ul]

2) slave become master

[ul]

ping secondary site ok, ping primary ok[/ul]

3) master upgrade

[ul]

lost ping on master and slave no site where available[/ul]

4) Master become master

[ul]

Site 1 are available and site 2 are not available

site 2 become available after 4-5 minutes[/ul]

5) let's take a look a checksum before upgrade a new firmware,

get sys ha status

not the same, try a calculate

diagnose sys ha checksum recalculate

Ok cluster are in sync.

[size="2"]Humm, cluster have to warm up , let's go for the round two....[/size]

Same issue... round 3 same issue, round 4 same issue ...[size="2"]After 3 hours of interruptible upgrade  i'm on 6.0.9. OK guy let's try a ha failover, connect on slave cli exec reboot, lost one ping... VPN are UP, device and equipement ping are OK. all seems good on two site [/size][size="2"]connect on master cli exec reboot lost one ping... VPN are UP, device and equipement ping are OK. all seems good on two site.[/size]

ha uninterruptible-upgrade are enable (default).

What i forgot to do before the upgrade ?

best regards

[lobstercreed]

I'd be curious to know if you find out.  This sounds like normal behavior to me, but maybe I misunderstood something.  I assume these are active-active clusters?

[Virgule59]

Hi,

Do you think that failover doesn't work on firmware upgrade ? This is not what i'v found on this forum and not what's fortinet KB say : Upgrade firmware.

Cluster HA are in Active-Passive mode BUT:.

Vcluster1 are master on the ACTIVE and Vcluster2 are master on the PASSIVE. In this type of configuration slave are not really in passive mode because it's master on the vcluster2 and slave on vcluster1.

And has I said if i reboot a member of the cluster the failover mechanism work as expected.

For me this sound strange

As noted in the KB i understand now why i've lost the passive firewall, because of the override. but it's doesn' tell why i've lost the cluster during upgrade.

Regards

[lobstercreed]

I think I must have misunderstood.  I thought you were saying that you had only 1 ping loss during the failover (which is basically hitless) and when you said one or the other was down I thought you meant during the upgrade process (rebooting the hardware, switching HA).  If you're getting a different result during upgrade than during a "normal" reboot other than maybe longer boot time, I don't know why.

[KDalbjerg]

Did you ever found out why this happen?