Active - Passive Firmware upgrade Process

[bhagat_sudhir]

Hi Experts,

We are running Fortigate 300C firewall in HA (Active - Passive). Pls share step-by-step process for upgrading the Firmware for Active Passive HA (remotely).

Regards

SB 

[bhagat_sudhir]

please reply

[awasfi_FTNT · Accepted solution]

Hello,

Please refer to the following document (page 176):

http://docs.fortinet.com/d/fortigate-high-availability-ha-2

Recommend before the upgrade:

1) Check the release notes for supported upgrade path, special notices, product integration, known issues and limitations if any.

2) Backup configuration before and after each upgrade.

3) Plan a maintenance window for the upgrade.

3) Have some one available on the remote site in case something went wrong during the upgrade.

4) Make sure check sum is matching between cluster members using the following CLI commands:

# get sys ha status # diag sys ha showcsum

# execute ha manage <Slave ID>   <<-- could be 0 or 1, check "get sys ha status" results

$ diag sys ha showcsum

$ exit

The results of "diag sys ha showcsum" should be the same on all levels (all/global/vdoms)

Regards,

[ede_pfau]

As best practice, reboot the cluster before upgrading.

For a A/P cluster the actual downtime during a cluster reboot is quite short, maybe 9 pings.

[emnoc]

Ede i would have to disagree on a pre-upgrade reboot. This might be a "feel good method",  but it's not required. What  awasfi wrote, is what I typically do, but you also add to that;

[ul]

obtain a copy of the current fortiOS-version just encase you need to reformat and fallback

have a post upgrade means for contact support if your network goes 100% down ( very unlikely but you should have fallback plan that includes support access )

Acquire the AV/IPS definitions for backup and manual pushing for any post install issues such as fortigaurd service are hosed up ( optional)[/ul]

The reason for the last part, I upgrade a 1000C and had trouble gaining  access to fds network due to a combination of issues ( both fortios, fds network, and my provider  ). So our AV was not updated and we ran for 6-8 hours with an not up to date  definitions.

[ede_pfau]

@emnoc, my advice stems from experience. In the old days, I believed every word of the FortiOS lore where the Handbook stated to just upgrade and everything would be fine. Until some day I upgraded a FGT with an uptime of 200+ days which just didn't take. It didn't reboot properly, and when it finally did after pulling the power cord twice, half of the config was gone...especially the static routes near the end of the config file. That was very unpleasant.

Then one day I read about this in the forums (a post by rwpatterson IIRC) that others had had the same experience as me. If you've got a cluster, the more so - I've seen so many FW upgrades with clusters that wouldn't 100% work at the first attempt. Compared to the minimal effort a reboot takes it has proven to be worth it since.

But, to each his own - I'll keep it on my list of BP.

[gschmitt]

ede_pfau wrote:

@emnoc, my advice stems from experience. In the old days, I believed every word of the FortiOS lore where the Handbook stated to just upgrade and everything would be fine. Until some day I upgraded a FGT with an uptime of 200+ days which just didn't take. It didn't reboot properly, and when it finally did after pulling the power cord twice, half of the config was gone...especially the static routes near the end of the config file. That was very unpleasant.

Then one day I read about this in the forums (a post by rwpatterson IIRC) that others had had the same experience as me. If you've got a cluster, the more so - I've seen so many FW upgrades with clusters that wouldn't 100% work at the first attempt. Compared to the minimal effort a reboot takes it has proven to be worth it since.

But, to each his own - I'll keep it on my list of BP.

Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

[emnoc]

Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

Than that's just bad upgraded and management of a post-upgrade imho.  After completing the upgrade you should always validate the FortiOS was actually upgrade & ensure the haardware models are the same.

e.g ( for starters )

  get system status

  diag sys  flash list

  get system startup-error-log

  get system ha-nonsync-csum

  and a  diag sys  csum  between pairs in a cluster if they are exact in the images installed.

What I've found out from experience. A lot of older devices and older fortiOS has hadproblem when jumping to the latest  fotriOS version. I 've computed  this to be because of the  linux FS type difference. So be prepared to  reformat. if anything I would suggest a reformat  if your upgrading between  3.0 and let's say 5.0 for extendedtyepFS. This would ensure the system flash is freshly format and no bad blocks & for the multiple partitions.

As far as rebooting a long uptime cluster, that's an extra steps that's not really called for but optional. YMMV

ken

[gschmitt]

emnoc wrote:

Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

Than that's just bad upgraded and management of a post-upgrade imho.  After completing the upgrade you should always validate the FortiOS was actually upgrade & ensure the haardware models are the same.

I did fix it during the upgrade by seperating the cluster, upgrading them individually and reforming the cluster, that's not the point.

The point is that simply "click update and it runs without a real downtime" is not always the case 

[rwpatterson]

For what it's worth, rebooting each member of a cluster adds so little downtime, I don't see what the issue is... The time saving ROI is good. Clear the memory, start the day off with that cold one in the near future. Try the upgrade after being up 300+ days and then ask yourself after messing with the thing for hours, "why didn't I just reboot the darn thing first?"....