Fortigate HA cluster and reserved management interface

[kritt]

Hello community, This is my first post on this forum :-) I need some help about the reserved management interfaces concept and there access from/to the outside. My objectives are : - having a cluster of 2 fortigate 1500D in active/passive mode - aggregated interfaces "inside" and "outside" - single reserved management interfaces for syslog, snmp, ntp,dns,...(logs sent to FortiManager) - using mgmt1 as reserved mgmt intf - they are on the same network - No specific management vdom, all in vdom root (but vdom-admin activated for futures configurations) - 2 heartbeat interfaces - 2 session sync interfaces Here is a sample of the architecture :                                                   Outside

               _____________________________________                                      ||                                   ||                     agg_if   ||                                   || agg_if                               ||                                   ||                               ||       [size="1"]2_hb_intf [/size]                 ||           [size="1"] mgmt1[/size]   -----------===========----------[size="1"]mgmt1                [/size] mgmt-----------|   FW1   |                         |   FW2  |-------------mgmt network [size="1"]network            [/size]    -----------===========----------                                          ||        [size="1"]2_session_sync_intf      [/size] ||                     agg_if ||                                     || agg_if                             ||                                     ||                             ||                                     ||                         ______________________________________                                                      inside First, I've configured the 2 Fortigate 1500D in cluster active/passive mode with a very basic configuration: - factoryreset - deleting all default routes, dhcp, ip,... - configuring an IP for each mgmt1 (2 differents IP on the same network) and allow-access https,ssh,snmp,ping - configuring date/time, ntp (without source-ip parameter) - disable hb interfaces and session interfaces of FW2 - cluster configuration - enable hb interfaces and session interfaces of FW2 - configuring syslog server (without source-ip parameter) And that's it for the moment. No default/special route, no aggregated interfaces first Cluster sync is ok. Here is the ha configuration : config system ha     set group-id 1     set group-name "my_cluster"     set mode a-p     set password <my_pass>     set hbdev "port_x" 50 "port_y" 50     set session-sync-dev "port_a" "port_b"     set session-pickup enable     set ha-mgmt-status enable     set ha-mgmt-interface "mgmt1"     set ha-mgmt-interface-gateway x.x.x.x     set override disable     set priority 200/100 end My questions :

- I can ping mgmt1 from the management network or from any other vlan connected to the mgmt network : Why does it works while there is no route (default or specific) to the different vlan network connected to the mgmt net? I've checked the routing static table and routes kernel table and there is no route via interface mgmt1. I guess it works anyway because of the specific gateway setting in the ha configuration, is it correct ? - I can't ping the interface-gateway sets in the ha config from the vdom root ( execute ping x.x.x.x) With a packet sniffer command, no ping traffic is going out the firewall. Why ? How does the firewall decide which traffic to send through mgmt1 and which other traffic to send via the routing table ? Does the firewall allow only internal processes to send traffic from mgmt1 (snmp, syslog,...) and forbid other traffic ? - With this configuration, I see no mgmt traffic initiated from the firewalls (no syslog messages from mgmt1) If I add the "set ha-direct" command in the cluster ha config, the firewalls send syslog messages but no ntp traffic (and syslog message indicate "ntp server is determinated unreachable"). I have not found a lot of details about this parameter except "Enable/disable sending of messages (logs, SNMP, RADIUS) directly from ha-mgmt interface" and that it's a special parameter for snmp configuration. What does this command exactly ? I would like to know how the firewall chooses which traffic is sent via this reserved management interface. Is all the traffic initiated by the firewall sent from this interface (dns, syslog, snmp, logs,...) or is there any configuration parameters that could allow to define which traffic to sent via mgmt1 or not ? ???

[jhouvenaghel_FTNT]

I can ping mgmt1 from the management network or from any other vlan connected to the mgmt network : Why does it works while there is no route (default or specific) to the different vlan network connected to the mgmt net?

--> you don't see the route in the routing table as the ha-managment interface is in a dedicated vdom. This interface will use the gateway you configured under "system ha" to reply to a remote station.

I've checked the routing static table and routes kernel table and there is no route via interface mgmt1. I guess it works anyway because of the specific gateway setting in the ha configuration, is it correct ? --> correct

- I can't ping the interface-gateway sets in the ha config from the vdom root ( execute ping x.x.x.x) With a packet sniffer command, no ping traffic is going out the firewall. Why ? How does the firewall decide which traffic to send through mgmt1 and which other traffic to send via the routing table ? Does the firewall allow only internal processes to send traffic from mgmt1 (snmp, syslog,...) and forbid other traffic ?

--> as far as I remember, what you see is expected. You can not ping from the ha-management intrerface to an external host. However, if you use ha-direct (under config system ha) , then logs can be sent from the ha-management interface of each cluster unit - With this configuration, I see no mgmt traffic initiated from the firewalls (no syslog messages from mgmt1) If I add the "set ha-direct" command in the cluster ha config, the firewalls send syslog messages but no ntp traffic (and syslog message indicate "ntp server is determinated unreachable"). I have not found a lot of details about this parameter except "Enable/disable sending of messages (logs, SNMP, RADIUS) directly from ha-mgmt interface" and that it's a special parameter for snmp configuration. What does this command exactly ? I would like to know how the firewall chooses which traffic is sent via this reserved management interface. Is all the traffic initiated by the firewall sent from this interface (dns, syslog, snmp, logs,...) or is there any configuration parameters that could allow to define which traffic to sent via mgmt1 or not ?

--> initially, ha-management interface was to be able to access using ssh, https each cluster unit on a dedicated IP address not synced between cluster members and to be able to poll using snmp each cluster unit separately. for this last step, you need to enable ha-direct which is under the snmp configuration part.

Then another ha-direct parameter (under config system ha) was available to allow to connect to each cluster unit separately using radius, tacacs, .. authentication and have the radius, tacacs, ... auth request sent to the radius, tacacs, ... server directly from each ha-management interface., Plus the possibility to send log directly from the ha-management interface for each cluster unit

[kritt]

Thank you Jhouvenaghel_FTNT.

"--> you don't see the route in the routing table as the ha-managment interface is in a dedicated vdom. This interface will use the gateway you configured under "system ha" to reply to a remote station."

Ok. I guess this is due to the "ha-mgmt-interface" setting under "config system ha". I have not see reference to this dedicated vdom in the ha documentation even if it seems to be evidence. The ha documentation refers to an hidden v_sys ha but this v_sys ha is intended for routing management traffic via the master unit. But here, the management traffic is routed via the reserved management interface of each firewall. Something I noticed is that the mgmt1 setting "vdom" disappeared when I've enabled the "ha-mgmt-interface" setting. That confirms the hidden vdom when "ha-mgmt-interface" setting is enabled under "config system ha".

"--> initially, ha-management interface was to be able to access using ssh, https each cluster unit on a dedicated IP address not synced between cluster members and to be able to poll using snmp each cluster unit separately. for this last step, you need to enable ha-direct which is under the snmp configuration part. Then another ha-direct parameter (under config system ha) was available to allow to connect to each cluster unit separately using radius, tacacs, .. authentication and have the radius, tacacs, ... auth request sent to the radius, tacacs, ... server directly from each ha-management interface., Plus the possibility to send log directly from the ha-management interface for each cluster unit" Ok. I have done complementary tests : ==>  basic cluster config without ha-mgmt-status enable : all management trafic is routed via the mgmt1 master interface ==> cluster config with ha-mgmt-status enable + ha-direct : management trafic is sent from each nodes and I can access each nodes separately (https/ssh/ping) ==> cluster config with ha-mgmt-status enable but no ha-direct : I can access each nodes separately (https/ssh/ping) but there is no management trafic going out the mgmt1 interface. The management traffic (syslog) used the routing table .

Thank you for your help.