HA status of cluster after upgrade

[darrencarr]

Hi All, I have a question. I am working on my test lab in preparation for a system upgrade. In my test lab I have 2 x Fortigate 60B' s, configured in a HA cluster. I am looking at applying patch MR1 Patch 8 to MR1 Patch 4. My procedure is as follows: - connect to to the console of the cluster - apply the patch - from the console monitor the upgrade as it is taking place (CLI) When I do this I obviously see the firmware being upgraded on the Master, then the Slave. Throughout the upgrade I see messages stating ' slave is out of sync with master' and then eventually ' slave synced successfully with master' . I then leave it a while and then the messages appear again ' slave is out of sync with master' and then ' slave is synced with master' . My question is when do you know that the slave has completely synced with the master? I thought after seeing the first successful ' synced with master' that that was it. However it happens again. Can anyone please explain? Thanks Darren

[ede_pfau]

Hi, when upgrading a cluster of 2 units, and if there is no override parameter set, first the slave unit is upgraded by the master unit and reboots. Then the cluster fails over, the slave becomes master and the former master unit is upgraded and reboots. Hence the 2 identical messages. I just wonder: without override set, the old master would then remain the slave unit. That is, when upgrading a cluster you exchange the units' roles. I' ll have a close look next time as I haven' t noticed this consciously yet. (anyway, who cares...)

[darrencarr]

Hi, Thanks for the response. How do you know when the Master/Slave are in complete sync though? Are you able to compare two values that show the units are now idenitical (fully sync' d)? Thanks, Darren

[billp]

Darren, Below is my experience with a 111C HA cluster. With my HA setup, I can tell when the sync is complete by logging into the Fortigate and looking at the Config tab. If it shows two Fortigates, then I can assume the sync is complete. For further insurance, I click on the View HA Statistics link. During the reboot, my HA slave and master reverse rolls. I will usually do a second reboot to put back the natural order of the slave/master units. This also gives me a chance to verify that the firmware installed properly by viewing the Status page for each unit in the cluster before/after the second reboot. In one instance, the slave unit failed to upgrade properly, and I had to reinstall the firmware.

[darrencarr]

Hi Bill, Thanks for the information. I was hoping to see something that allowed you to compare two values (in the CLI) that would allow you to easily distinguish if the devices were in sync (same number) using ' get sys ha status' this would be good :) Last time when I did the upgrade, I saw the two units in the command tab, and also checked the HA stats, even after this I got a couple of messages :) I' ll give it another go in my test lab and see. Thanks again Darren

[ede_pfau]

@billp: you might enable " Override" in the HA config of your primary HA unit with the benefit of running the cluster upgrade automatically. With override enabled the unit will always be the master unit, even after temporarily being the slave. You can always check its status from GUI or CLI, be it primary or secondary. Fortinet removed the default setting of override enabled some time ago because customers wanted to have the fewest interruptions possible when upgrading.

[ede_pfau]

that allowed you to compare two values (in the CLI) that would allow you to easily distinguish if the devices were in sync

from the HA Guide: " Comparing checksums of cluster units You can use the

diagnose sys ha showcsum

command to compare the configuration checksums of all cluster units. The output of this command shows checksums labelled global and all as well as checksums for each of the VDOMs including the root VDOM. The primary unit and subordinate unit checksums should be the same. If they are not you can use the

execute ha synchronize

command to force a synchronization." Usually you compare the " all" checksum.

[billp]

Ede, Thanks for the tips. Appreciate it!

[darrencarr]

Yeah thanks Ede

[ede_pfau]

hey, it' s always fun to share with friends!