Technical Tip: The FortiGate HA hidden VDOMs 'vsys_ha' and 'vsys_hamgmt'
Description
This article describes the purpose and difference between the hidden VDOMs used for HA communication on FortiGates in a FortiGate Clustering Protocol (FGCP) HA cluster, the conditions under which they are available, and provides some basic diagnostic commands for inspecting the VDOM's routing and connectivity.
Scope
FortiOS.
Solution
FortiGate uses two different hidden VDOMs for HA communication and reserved management interface access, vsys_ha and vsys_hamgmt.
Summary of differences:
vsys_ha | vsys_hamgmt | |
|---|---|---|
Purpose | Internal HA communication and relay of secondary unit management traffic | Out-of-band management access through HA reserved management interfaces |
Availability | Used for internal FGCP HA operation. Always present if FGCP is configured. | Only present when HA reserved management interface is configured. |
IP Addressing | By default, uses IPv4 addresses in the 169.254.0.0/26 range. If 'set unicast-hb enable' is configured, uses IP address configured in 'config system interface'. | Uses IP address configured in 'config system interface'. |
Relevant interfaces | Internal interfaces such as port_ha and HA inter-VDOM links (havdlink) | HA reserved management interfaces. |
Diagnostic use | Inspect internal HA IP communication. If unicast-hb is enabled, inspect internal HA routing. | Inspect reserved management routing, ARP, and connectivity. |
Example VDOM list:
Note: This output is taken from a firewall with a reserved management interface configured. If it is not configured, the 'name=vsys_hamgmt/vsys_hamgmt' entry will not appear.
diagnose sys vd list
system fib version=94
list virtual firewall info:
name=root/root index=0 enabled fib_ver=195 rpdb_ver=0 use=52 rt_num=9 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0, ipd_tuple_timeout=248
ecmp=source-ip-based, ecmp6=source-ip-based asym_rt6=0 rt6_num=18 strict_src_check=0 src_check_reply=0 dns_log=1 ses_num=3 ses6_num=0 pkt_num=51637
tree_flag=1 tree6_flag=1 traffic_log=1 extended_traffic_log=0
deny_tcp_with_icmp=0 ses_denied_traffic=no mc_ses_denied_traffic=no tcp_no_syn_check=0 central_nat=0 policy_mode_ngfw=0 block_land_attack=0 link_check_local_in=1
fw_session_hairpin=no keep-PRP-trailer=0 auxiliary_ses=0 dup_num=2
ipv4_rate=0, ipv6_rate=0, mcast6-PMTU=0, allow_linkdown_path=0
per_policy_disclaimer=0 pcp=0
mode=AP ha_state=work prio=0 vid=1
name=vsys_ha/vsys_ha index=1 enabled fib_ver=31 rpdb_ver=2 use=22 rt_num=12 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0, ipd_tuple_timeout=248
ecmp=source-ip-based, ecmp6=source-ip-based asym_rt6=0 rt6_num=3 strict_src_check=0 src_check_reply=0 dns_log=0 ses_num=13 ses6_num=0 pkt_num=23737
tree_flag=1 tree6_flag=0 traffic_log=0 extended_traffic_log=0
deny_tcp_with_icmp=0 ses_denied_traffic=no mc_ses_denied_traffic=no tcp_no_syn_check=0 central_nat=0 policy_mode_ngfw=0 block_land_attack=0 link_check_local_in=1
fw_session_hairpin=no keep-PRP-trailer=0 auxiliary_ses=0 dup_num=0
ipv4_rate=0, ipv6_rate=0, mcast6-PMTU=0, allow_linkdown_path=0
per_policy_disclaimer=0 pcp=0
ha_flags={no-ses-sync,no-ha-stats} mode=standalone ha_state=work prio=0 vid=1
name=vsys_fgfm/vsys_fgfm index=2 enabled fib_ver=7 rpdb_ver=0 use=6 rt_num=0 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0, ipd_tuple_timeout=248
ecmp=source-ip-based, ecmp6=source-ip-based asym_rt6=0 rt6_num=1 strict_src_check=0 src_check_reply=0 dns_log=0 ses_num=0 ses6_num=0 pkt_num=1
tree_flag=0 tree6_flag=0 traffic_log=0 extended_traffic_log=0
deny_tcp_with_icmp=0 ses_denied_traffic=no mc_ses_denied_traffic=no tcp_no_syn_check=0 central_nat=0 policy_mode_ngfw=0 block_land_attack=0 link_check_local_in=1
fw_session_hairpin=no keep-PRP-trailer=0 auxiliary_ses=0 dup_num=0
ipv4_rate=0, ipv6_rate=0, mcast6-PMTU=0, allow_linkdown_path=0
per_policy_disclaimer=0 pcp=0
mode=standalone ha_state=work prio=0 vid=1
name=vsys_hamgmt/vsys_hamgmt index=3 enabled fib_ver=12 rpdb_ver=0 use=16 rt_num=9 asym_rt=0 sip_helper=0, sip_nat_trace=1, mc_fwd=0, mc_ttl_nc=0, tpmc_sk_pl=0, ipd_tuple_timeout=248
ecmp=source-ip-based, ecmp6=source-ip-based asym_rt6=0 rt6_num=1 strict_src_check=0 src_check_reply=0 dns_log=0 ses_num=5 ses6_num=0 pkt_num=10782
tree_flag=1 tree6_flag=0 traffic_log=0 extended_traffic_log=0
deny_tcp_with_icmp=0 ses_denied_traffic=no mc_ses_denied_traffic=no tcp_no_syn_check=0 central_nat=0 policy_mode_ngfw=0 block_land_attack=0 link_check_local_in=1
fw_session_hairpin=no keep-PRP-trailer=0 auxiliary_ses=0 dup_num=0
ipv4_rate=0, ipv6_rate=0, mcast6-PMTU=0, allow_linkdown_path=0
per_policy_disclaimer=0 pcp=0
mode=standalone ha_state=work prio=0 vid=1
vf_count=5 vfe_count=0
Note: The hidden VDOM vsys_fgfm is another expected hidden VDOM, even on devices that do not have HA configuration. It handles the FortiGate -> FortiManager or FortiGate -> FortiGate Cloud management tunnel when central-management is enabled, but is also present when no central-management is configured. See Technical Tip: CLI Behavior When Using FortiGate Cloud to Manage a FortiGate on v7.6.4.
Troubleshooting vsys_ha:
The vsys_ha VDOM is enabled when a FortiGate HA mode is set from standalone to active-active or active-passive. It handles most HA-related functions including HA heartbeat, administrator CLI login from one cluster member to another using SSH (see Technical Tip: How to access secondary unit of HA cluster via CLI), as well as HA configuration sync and relayed outbound requests from the secondary. See Technical Tip: Information about the protocols and ports used in FortiGate High Availability for more information.

Example configuration:
config system ha
set group-id 50
set group-name "90G"
set mode a-p
set password ENC
set hbdev "b" 0
set override disable
set priority 200
end
By default, the vsys_ha VDOM does not appear in CLI hints.
FGT91G-1(Primary) # execute enter ?
<name> vdom name
root
However, vsys_ha can still be accessed with 'execute enter vsys_ha' and further diagnostics can be observed:
execute enter vsys_ha
current vdom=vsys_ha:1
diagnose ip address list
IP=10.21.4.62->10.21.4.62/255.255.252.0 index=5 devname=port1
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=15 devname=root
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=23 devname=vsys_ha
IP=169.254.0.2->169.254.0.2/255.255.255.192 index=24 devname=port_ha
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=25 devname=vsys_fgfm
IP=169.254.0.65->169.254.0.65/255.255.255.192 index=26 devname=havdlink0
IP=169.254.0.66->169.254.0.66/255.255.255.192 index=27 devname=havdlink1
get router info routing-table all
No route available
diagnose ip route list
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=23(vsys_ha)
tab=65535 vf=1 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=23(vsys_ha)
tab=65535 vf=1 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=23(vsys_ha)
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=23(vsys_ha)
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.0/32 pref=169.254.0.2 gwy=0.0.0.0 dev=24(port_ha)
tab=65535 vf=1 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.2/32 pref=169.254.0.2 gwy=0.0.0.0 dev=24(port_ha)
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.63/32 pref=169.254.0.2 gwy=0.0.0.0 dev=24(port_ha)
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.64/32 pref=169.254.0.65 gwy=0.0.0.0 dev=26(havdlink0)
tab=65535 vf=1 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.65/32 pref=169.254.0.65 gwy=0.0.0.0 dev=26(havdlink0)
tab=65535 vf=1 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.127/32 pref=169.254.0.65 gwy=0.0.0.0 dev=26(havdlink0)
tab=65534 vf=1 vrf=0 scope=0 type=1 proto=18 prio=0 169.254.0.0/255.255.255.192/0->0.0.0.0/0 pref=0.0.0.0 gwy=169.254.0.66 dev=26(havdlink0)
tab=65534 vf=1 vrf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.0/26 pref=169.254.0.2 gwy=0.0.0.0 dev=24(port_ha)
tab=65534 vf=1 vrf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->169.254.0.64/26 pref=169.254.0.65 gwy=0.0.0.0 dev=26(havdlink0)
Troubleshooting vsys_ha with unicast heartbeat enabled:
If unicast-hb is enabled (only available for FortiGate VM platforms, see HA for cloud and virtual environments), the vsys_ha VDOM has additional routing configuration, and heartbeats are sent using UDP port 730.
Example configuration:
config system ha
set group-name "HA"
set mode a-p
set hbdev "port3" 50
set priority 200
set unicast-hb enable
set unicast-hb-peerip 10.1.98.5
end
Example diagnostic output:
FGT-A # execute enter vsys_ha
current vdom=vsys_ha:2
FGT-A # get router info routing-table all
C 10.1.98.0/26 is directly connected, port3
FGT-A # diagnose sniffer packet port3 '' 4 0 1
Using Original Sniffing Mode
interfaces=[port3]
filters=[none]
0.089894 port3 -- 10.1.98.4.730 -> 10.1.98.5.730: udp 412
0.211577 port3 -- 10.1.98.5.730 -> 10.1.98.4.730: udp 401
0.841392 port3 in 10.1.98.5.730 -> 10.1.98.4.730: udp 401
7.465990 port3 -- 10.1.98.5.2379 -> 10.1.98.4.53: udp 81
7.466022 port3 -- 10.1.98.5.2379 -> 10.1.98.4.53: udp 81
7.480302 port3 -- 10.1.98.4.53 -> 10.1.98.5.2379: udp 156
7.480324 port3 -- 10.1.98.4.53 -> 10.1.98.5.2379: udp 156
Troubleshooting vsys_hamgmt:
The hidden VDOM vsys_hamgmt is only present when a reserved management interface is configured. It handles administrative management traffic (HTTPS, HTTP, and SSH) to the reserved management interface and is available even when the device is acting as a secondary HA cluster member.
If ha-direct is enabled, vsys_hamgmt is also used for some outbound management traffic, see Out-of-band management with reserved management interfaces.

Example configuration:
config system ha
set group-id 50
set group-name "90G"
set mode a-p
set password ENC
set hbdev "b" 0
set ha-mgmt-status enable
config ha-mgmt-interfaces
edit 1
set interface "port1"
set gateway 10.21.7.254
next
end
set override disable
set priority 200
end
Example diagnostic output:
FGT91G-1(Primary) # execute enter ?
<name> vdom name
root
vsys_hamgmt
FGT91G-1(Primary) # execute enter vsys_hamgmt
FGT91G-1(Primary) # diagnose ip address list
IP=10.21.4.62->10.21.4.62/255.255.252.0 index=5 devname=port1
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=15 devname=root
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=23 devname=vsys_ha
IP=169.254.0.2->169.254.0.2/255.255.255.192 index=24 devname=port_ha
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=25 devname=vsys_fgfm
IP=169.254.0.65->169.254.0.65/255.255.255.192 index=26 devname=havdlink0
IP=169.254.0.66->169.254.0.66/255.255.255.192 index=27 devname=havdlink1
IP=127.0.0.1->127.0.0.1/255.0.0.0 index=28 devname=vsys_hamgmt
FGT91G-1(Primary) # get router info routing-table all
Routing table for VRF=0
C 10.21.4.0/22 is directly connected, port1
FGT91G-1(Primary) # get system arp
Address Age(min) Hardware Addr Interface
10.21.7.254 0 00:09:0f:09:b4:14 port1
FGT91G-1(Primary) # diagnose ip route list
tab=65535 vf=3 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.21.4.0/32 pref=10.21.4.62 gwy=0.0.0.0 dev=5(port1)
tab=65535 vf=3 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.21.4.62/32 pref=10.21.4.62 gwy=0.0.0.0 dev=5(port1)
tab=65535 vf=3 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.21.7.255/32 pref=10.21.4.62 gwy=0.0.0.0 dev=5(port1)
tab=65535 vf=3 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=28(vsys_hamgmt)
tab=65535 vf=3 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=28(vsys_hamgmt)
tab=65535 vf=3 vrf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=28(vsys_hamgmt)
tab=65535 vf=3 vrf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=28(vsys_hamgmt)
tab=65534 vf=3 vrf=0 scope=0 type=1 proto=18 prio=0 0.0.0.0/0.0.0.0/0->0.0.0.0/0 pref=0.0.0.0 gwy=10.21.7.254 dev=5(port1)
tab=65534 vf=3 vrf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.21.4.0/22 pref=10.21.4.62 gwy=0.0.0.0 dev=5(port1)
Related articles:
