Firmware " FortiOS" Updates on a large scale

[ZeeshanMoe]

Hi everyone
Could anyone please share their experience or best practice regarding firmware updates for a very big envirnoment (50 plus firewall clusters including Fortigates 200F, 600E, 1200D, 1500D and 3980E?
is it prefered to have the lastest FortiOS Version? Or a bit older but mature version?
Do you perform the firmware updates manually or do it automatically / schedule it?

 what else useful tipps can you give?

It would  be great to hear from IT experts in other big companies.

Best regards

[spoojary]

It is recommended to balance between having the latest features and stability. Opt for a version that is mature and has been tested in similar environments to minimize potential issues.  For better control and to ensure minimal disruption, consider performing firmware updates manually. This allows you to schedule updates during maintenance windows and monitor the process closely. Always back up the configuration before initiating any firmware updates Ensure you follow the recommended upgrade path for each FortiGate model to avoid compatibility issues.

About firmware installations | FortiGate / FortiOS 7.4.3 | Fortinet Document Library

[hbac]

Hi @ZeeshanMoe,

Please refer to this article: https://community.fortinet.com/t5/FortiGate/Technical-Tip-Recommended-Release-for-FortiOS/ta-p/22717...

Regards, 

[ZeeshanMoe]

according to the link this list is updated quarterly that would mean that the recommended version for a certain fortigate could potentially changes quite frequently and that one should update the fortigates at least 2 times a year. And to do that in huge envirnoments with large number of firewalls it is quite difficult. but that itseld is a separate issue :)

[kinodmer]

7.2.6 is very nearly mature. 7.2.7 probably will be. I think new systems may have slightly different considerations than upgrades, depending upon how you do it. If you're able to do a full parallel setup, you may be in a better position to test all the features for a new system. Using 7.2 postpones doing a major version upgrade longer. For obvious reasons it's hard to do a full test of a new version before upgrading to it.

[Yurisk]

Well, any such info will be opinionated, naturally, and will differ depending on specific topology/models/tolerance to the down time. So, with this disclaimer, my experience. 

Cluster - tread carefully. The issue is that upgrading members is easy, but downgrading (if issues appear) is not. To be exact - downgrading is relatively easy if upgrading only 1 version up, because on such upgrade FGTs keep current version as back up OS in back up partition. I wrote more info about that https://yurisk.info/2023/06/18/tips-on-upgrading-fortigate-in-ha-cluster/ . E.g. (imagined) say if you upgrade 7.0.6 -> 7.0.12 -> 7.0.14 -> 7.0.15 and turns out things do not work, downgrade to 7.0.14 would be easy, but back to original 7.0.6 - not so, to downgrade multiple versions you would need to dismantle the cluster, downgrade each member to 7.0.6 individually then create cluster again, brr. After all these years I still get jitters upgrading clusters. 

Of course, back up configuration each step - when doing multi-step upgrades, as this will allow you to downgrade back to any interim version, if need arises.

Always follow the official Upgrade Path on Fortinet site if you want to keep the configuration. When upgrading in multi-step, Fortigate converts its configuration to the upgrade-to FortiOS version, but knows to do it only according to the Upgrade Path (reliably). 

Reference for the Recommended FortiOS versions has been already mentioned. The higher model - the more conservative you should be. I keep 1500D on 7.0.15 until either critical CVE for it announced or End of support comes.  Standalone FGTs and smaller models 100/200/400 I keep at 7.2.8. The general wisdom has been to wait until x.x.6 version of FortiOS, but experience shows it varies. E.g. 6.0 was stable quite early, while 6.2 had issues beyond, the 7.2.x also had issues until 7.2.7/7.2.8. 

Make sure to disable auto-upgrade on SMB models, unless it is some lab/home environment.

I, personally, also do mostly manual upgrades - download beforehand all needed FOrtiOS images to local PC and upload them locally as well, not using "Upgrade from FortiCloud". As sometimes Fortinet servers can be overloaded/bad connectivity and this would add unnecessary stress.

Always read Release Notes - if to pick number 1 reason for troubles with/after upgrades this would be it - people don't read Release Notes for the version they upgrade to. 

Always prefer to have OOB access to the FGT or have someone with console cable and laptop available. FGTs with years got really stable in upgrades, but better be on safe side. I remember "bug" in earlier versions FGT 60A when doing Reboot from the device GUI could easily cause device not to boot or come up with empty config :), this doesn't happen anymore. 

Worth reading https://www.reddit.com/r/fortinet/ for personal experiences/opinions on the new FortiOS versions. 

HTH and good luck with upgrades, 98% of upgrades pass very smoothly actually.

[ZeeshanMoe]

Hi Yuri
Thanks for the detailed explanination and tipps & hints. you are right one should always be very careful at every step and good preparation will make the update process easy and successful.

[soamanpo1]

Just Google it, it's people's anecdotal experience with instability and bugs around specific features. This is unscientific approach but this is how it is with brand new releases - if there are bugs, you don't have them documented, you can only research the rumors and decide for yourself if you want to stick with the more conservative release (unless you need a specific fix that is in the new release).