Fortinet Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
bhagat_sudhir
New Contributor

Active - Passive Firmware upgrade Process

Hi Experts,

 

We are running Fortigate 300C firewall in HA (Active - Passive). Pls share step-by-step process for upgrading the Firmware for Active Passive HA (remotely).

 

Regards

SB 

1 Solution
awasfi_FTNT

Hello,

 

Please refer to the following document (page 176):

http://docs.fortinet.com/d/fortigate-high-availability-ha-2

 

Recommend before the upgrade:

1) Check the release notes for supported upgrade path, special notices, product integration, known issues and limitations if any.

2) Backup configuration before and after each upgrade.

3) Plan a maintenance window for the upgrade.

3) Have some one available on the remote site in case something went wrong during the upgrade.

4) Make sure check sum is matching between cluster members using the following CLI commands:

# get sys ha status # diag sys ha showcsum

# execute ha manage <Slave ID>   <<-- could be 0 or 1, check "get sys ha status" results

$ diag sys ha showcsum

$ exit

 

The results of "diag sys ha showcsum" should be the same on all levels (all/global/vdoms)

 

Regards,

View solution in original post

9 REPLIES 9
bhagat_sudhir
New Contributor

please reply

awasfi_FTNT

Hello,

 

Please refer to the following document (page 176):

http://docs.fortinet.com/d/fortigate-high-availability-ha-2

 

Recommend before the upgrade:

1) Check the release notes for supported upgrade path, special notices, product integration, known issues and limitations if any.

2) Backup configuration before and after each upgrade.

3) Plan a maintenance window for the upgrade.

3) Have some one available on the remote site in case something went wrong during the upgrade.

4) Make sure check sum is matching between cluster members using the following CLI commands:

# get sys ha status # diag sys ha showcsum

# execute ha manage <Slave ID>   <<-- could be 0 or 1, check "get sys ha status" results

$ diag sys ha showcsum

$ exit

 

The results of "diag sys ha showcsum" should be the same on all levels (all/global/vdoms)

 

Regards,

ede_pfau
Esteemed Contributor III

As best practice, reboot the cluster before upgrading.

For a A/P cluster the actual downtime during a cluster reboot is quite short, maybe 9 pings.


Ede

"Kernel panic: Aiee, killing interrupt handler!"
emnoc
Esteemed Contributor III

Ede i would have to disagree on a pre-upgrade reboot. This might be a "feel good method",  but it's not required. What  awasfi wrote, is what I typically do, but you also add to that;

[ul]
  • obtain a copy of the current fortiOS-version just encase you need to reformat and fallback
  • have a post upgrade means for contact support if your network goes 100% down ( very unlikely but you should have fallback plan that includes support access )
  • Acquire the AV/IPS definitions for backup and manual pushing for any post install issues such as fortigaurd service are hosed up ( optional)[/ul]

    The reason for the last part, I upgrade a 1000C and had trouble gaining  access to fds network due to a combination of issues ( both fortios, fds network, and my provider  ). So our AV was not updated and we ran for 6-8 hours with an not up to date  definitions.

     

     

     

     

  • PCNSE 

    NSE 

    StrongSwan  

    ede_pfau
    Esteemed Contributor III

    @emnoc, my advice stems from experience. In the old days, I believed every word of the FortiOS lore where the Handbook stated to just upgrade and everything would be fine. Until some day I upgraded a FGT with an uptime of 200+ days which just didn't take. It didn't reboot properly, and when it finally did after pulling the power cord twice, half of the config was gone...especially the static routes near the end of the config file. That was very unpleasant.

     

    Then one day I read about this in the forums (a post by rwpatterson IIRC) that others had had the same experience as me. If you've got a cluster, the more so - I've seen so many FW upgrades with clusters that wouldn't 100% work at the first attempt. Compared to the minimal effort a reboot takes it has proven to be worth it since.

    But, to each his own - I'll keep it on my list of BP.


    Ede

    "Kernel panic: Aiee, killing interrupt handler!"
    gschmitt
    Valued Contributor

    ede_pfau wrote:

    @emnoc, my advice stems from experience. In the old days, I believed every word of the FortiOS lore where the Handbook stated to just upgrade and everything would be fine. Until some day I upgraded a FGT with an uptime of 200+ days which just didn't take. It didn't reboot properly, and when it finally did after pulling the power cord twice, half of the config was gone...especially the static routes near the end of the config file. That was very unpleasant.

     

    Then one day I read about this in the forums (a post by rwpatterson IIRC) that others had had the same experience as me. If you've got a cluster, the more so - I've seen so many FW upgrades with clusters that wouldn't 100% work at the first attempt. Compared to the minimal effort a reboot takes it has proven to be worth it since.

    But, to each his own - I'll keep it on my list of BP.

    Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

    emnoc
    Esteemed Contributor III

    Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

     

    Than that's just bad upgraded and management of a post-upgrade imho.  After completing the upgrade you should always validate the FortiOS was actually upgrade & ensure the haardware models are the same.

     

    e.g ( for starters )

     

      get system status

      diag sys  flash list

      get system startup-error-log

      get system ha-nonsync-csum

     

      and a  diag sys  csum  between pairs in a cluster if they are exact in the images installed.

     

     

    What I've found out from experience. A lot of older devices and older fortiOS has hadproblem when jumping to the latest  fotriOS version. I 've computed  this to be because of the  linux FS type difference. So be prepared to  reformat. if anything I would suggest a reformat  if your upgrading between  3.0 and let's say 5.0 for extendedtyepFS. This would ensure the system flash is freshly format and no bad blocks & for the multiple partitions.

     

    As far as rebooting a long uptime cluster, that's an extra steps that's not really called for but optional. YMMV

     

    ken

    PCNSE 

    NSE 

    StrongSwan  

    gschmitt
    Valued Contributor

    emnoc wrote:

    Indeed, I got a cluster under my care which simply ran two different firmware images after an upgrade

     

    Than that's just bad upgraded and management of a post-upgrade imho.  After completing the upgrade you should always validate the FortiOS was actually upgrade & ensure the haardware models are the same.

    I did fix it during the upgrade by seperating the cluster, upgrading them individually and reforming the cluster, that's not the point.

     

    The point is that simply "click update and it runs without a real downtime" is not always the case 

    rwpatterson
    Valued Contributor III

    For what it's worth, rebooting each member of a cluster adds so little downtime, I don't see what the issue is... The time saving ROI is good. Clear the memory, start the day off with that cold one in the near future. Try the upgrade after being up 300+ days and then ask yourself after messing with the thing for hours, "why didn't I just reboot the darn thing first?"....

    Bob - self proclaimed posting junkie!
    See my Fortigate related scripts at: http://fortigate.camerabob.com