Making sense of "dedicated management" features in relation to HA etc.

[jmlux]

Hi,

Currently running FOS 5.4 on a 100D using a-p HA I'd like to find out and document the EXACT differences and compatibility between the following features concerning HA/clustering and dedicated management. Unfortunately, asking support about things like these is an impossible task. They have no idea what you are talking about. I am actually wondering what person in the company would be able to answer these. Maybe I am also asking too unclearly and too broadly.

Intro

Concerning the features in a standalone setup (no HA) I can find: * config system interface -> edit mgmt -> set dedicated-to management (only available on "mgmt" interface) * config system dedicated-management -> set interface mgmt, set default-gateway 1.2.3.4 (only available on "mgmt" interface too) (this seems to put the interface into a hidden VDOM called "dmgmt_vdom") Now, if we add VDOMs to the game, you can configure a separate mgmt vdom instead of the default one (root): * config system global -> set management-vdom XXX   Now, when HA/clusters come into play, then there is the reserved manamagent interface feature: * config system ha -> set ha-mgmt-interface which puts two interfaces across a cluster into a hidden VDOM called "vsys_hamgmt" which makes individual access to the appliance possible. Now, if you combine HA/VDOMs, there seems to be a "standalone-mgmt-vdom" option in HA configuration so you can have a mgmt VDOM other than root and which is NOT synchronized across the cluster (cluster member units have individual IPs). Finally, there is "ha-direct" to make the chaos perfect. I am really missing an overall view of all these things, interactions and conflicts documented somewhere together. Questions 1) What exactly is the effect of "config sys interface, edit mgmt, set dedicated-to management" vs. "config system dedicated-mgmt, set interface mgmt"? What if only either one vs. both are set? 2) What is the difference between "config system dedicated-mgmt" (which automatically creates a hidden mgmt vdom "dmgmt_vdom") vs. "config system global -> set management-vdom" which achieves the same thing? (except that you may be able to create more complicated setups using the latter) 3) How would "config system dedicated-management" be compatible with a cluster/HA setup? I guess it wouldn't, but you can configure it and create havoc (same IP on both sides using different MAC adresses). 4) How would "ha-mgmt-interface" and "standalone-management-vdom" work together / create conflicts? (I have not tried.) How would "ha-direct" and "standalone-management-vdom" work together / create conflicts? (especially since "ha-direct" somehow tells the ha-mgmt-interface to be used for management traffic egress, and I don't see how there could be several) Please also nitpick and correct or enhance the following statements: First we need to get our terms straight: "Management VDOM". This designates the vdom the internal CPU is attached to, and which is used to initiate EGRESS management traffic (syslog, Fortiguard, NTP, e-mail, SNMP traps, ...). By default, management VDOM = root, no matter if it is a standalone unit or a cluster. INGRESS management traffic on the other hand (GUI, CLI, SNMP polling) is configured per interface (be it standalone or not) and can be enabled anywhere. It is not actually related to any "Management VDOM". Now, in a HA context, and in order to be able to manage cluster units individually you can either: 1) use "ha-mgmt-interface" statement, which makes the given interface's configuration standalone on each cluster unit, and which automatically creates "vsys_hamgmt" VDOM (+ inter-VDOM links to management VDOM for egress management traffic) ; 2) use "standalone-management-vdom" statement (together with a previously manually created management VDOM), which can contain individual IP addresses per unit. There are two properties here to take into account and not to be confused: * VDOM is standalone (individual IP addresses can be configured per unit) * VDOM is the management VDOM (or not) In case #1 above, it is implicit that the vsys_hamgmt VDOM is standalone. If you want it to become the management VDOM (like #2) (see definition of "Management VDOM" above), you need to use "ha-direct". Still, management traffic would always egress the master unit only. In case #2 above, it is implicit that the manually created VDOM is the management VDOM, since you created it as such. If you want it to be standalone in a HA context (like #1), you need to use "standalone-management-vdom". Management traffic would exit the individual cluster member in this case. In case of #1, there can be only one interface for management. For more advanced setups you'd need to use #2, which will also make it more complex.

Bye,

Marki

Links Some links I found, feel free to complete the list. * http://docs-legacy.fortin...p/VDOM-NAT.178.03.html * http://docs-legacy.fortin...fig_system.23.017.html * http://docs-legacy.fortin....23.035.html#ww2922028 * http://kb.fortinet.com/kb....do?externalID=FD34744 * http://help.fortinet.com/...peratingInVcluster.htm * http://help.fortinet.com/...peratingReservedMg.htm

[whatz]

Hi Marki,

Great summary of a confusing topic for me too.  We use most of the times a clustered Fortigate setup in our environment.  In this case it is possible to select a dedicated HA management interface in the HA config which will result in the "dedicated HA mgmt" interface being shifted into a hidden management VDOM.  Because of that it is not participating in the regular routing table of standard traffic.  More about this here:

Technical Tip: HA Reserved Management Interface (fortinet.com)

Another interesting concept that was introduced with FortiOS 6.2 is the "split task VDOM".  In this case you have an officially support VDOM that is specifically only for out of band management traffic:

Technical Tip: Configuring split-task VDOM mode With Fortinet Security Fabric

I am still pondering on what the best way is to split management/out of band traffic from the rest of the data forwarding traffic / routing tables when a firewall is not in a HA setup.  I am currently leaning to use the "config system dedicated-mgmt" feature for this:

FortiGate dedicated-mgmt feature - Out-of-band Management (fortinet.com)

Just using a separate virtual routing table (VRF) could also be an option but I am more leaning to the dedicated-mgmt for standalone firewalls personally at the moment.

Mike

[Robinson_Roca]

ITs too bad, Fortinet doesn't monitor this board.  This should have been something Fortinet answered for us.  I too asked support for clarification, and they have NO idea what they are saying.  They talk themselves into circles, instead of actually trying to investigate and find someone who knows the answer.  Kind of sad

[Anthony_E]

Hello all,

Sorry for the late reply and thank you for your patience.

Be sure that we will take care of this and be sure somebody will reply to your post.

Regards,

[pwinthrop]

Been close to 2 months now and to be honest, the OP's inquiry on a topic that is rather convoluted definitively needs better documentation, explanation, clarification, etc...  "I'm sure someone will reply" seems lacking, and 2 months from that statement with nothing from nobody is a failing of Fortinet. Would be nice to get some attention here.

[Debbie_FTNT]

Hey pwinthrop,

sorry, the thread must have fallen through the crags somewhere.

I've read through the above posts, and to me it seemed as if whatz had provided a decent overview of the different features mentioned in the OP's original comment.

Do you still have questions regarding HA/dedicated management?