Description
This article describes the HA heartbeat packets' Ethertypes for FortiOS.
Scope
FortiGate.
Solution
Normal 802.3 IP packets have an Ethernet type (Ethertype) field value of 0x0800. Ethertype values other than 0x0800 are understood as layer 2 frames rather than IP packets.
HA heartbeat packets use the following Ethertypes:
Ethertype 0x8890 - NAT/Route Mode Heartbeat:
HA heartbeat packets for NAT/Route mode clusters use Ethertype 0x8890. These packets are used by cluster units to find other cluster units and to verify the status of other cluster units while the cluster is operating.
diagnose sniffer packet any 'ether proto 0x8890' 6 0 l
diagnose sniffer packet <heartbeat_interface> '' 6 0 l
The Ethertype of these packets can be changed using the ha-eth-type option of the config system ha command.
Ethertype 0x8891 - Transparent Mode Heartbeat.
HA heartbeat packets for Transparent mode clusters use Ethertype 0x8891. These packets are used by cluster units to find other cluster units and to verify the status of other cluster units while the cluster is operating.
The Ethertype of these packets can be changed using the ha-eth-type option of the config system ha command.
Ethertype 0x8892 - Session Synchronization:
Session synchronization packets use Ethertype 0x8892. The interfaces used for session synchronization must be connected either directly using the appropriate cable (possible if there are only two units in the cluster) or using switches. If one of the interfaces becomes disconnected, the cluster uses the remaining interfaces for session synchronization. If all the session synchronization interfaces become disconnected, session synchronization reverts to using the HA heartbeat link.
All session synchronization traffic is between the primary unit and each subordinate unit.
Since large amounts of session synchronization traffic can increase network congestion, it is recommended to keep this traffic off the network by using dedicated connections for it:
config system ha
set session-sync-dev port10 port12
end
Session synchronization is always using UDP 708, but this will be encapsulated differently depending on the 'session-sync-dev' setting. If session-sync-dev is being specified, the packets will use 0x8892 and will exit over the mentioned port. If session-sync-dev is not being specified, the packets will use 0x8893 and will exit the heartbeat port.
Ethertype 0x8893 - HA telnet sessions (Configuration synchronization):
HA telnet sessions between cluster units over HA heartbeat links use Ethertype 0x8893. The telnet sessions are used to synchronize the cluster configurations. Telnet sessions are also used when an administrator uses the execute ha manage command to connect from one cluster unit CLI to another. The Ethertype of these packets can be changed using the l2ep-eth-type option of the config system ha command.
Understanding the different types of heartbeat packets will ease troubleshooting. Heartbeat packets are recognized as layer 2 frames; the switches and routers on the heartbeat network that connect to heartbeat interfaces must be configured to allow them to pass through. If layer 2 frames are dropped by these network devices, heartbeat traffic will not be allowed between the cluster units.
For example, some third party network equipment may not allow Ethertype 0x8893, therefore the unit can still be found in the HA cluster, but the configuration will never synchronize, and it will not be able to 'execute ha manage' the other unit, or the checksum of the secondary unit may not be shown accordingly on the Primary unit. In this case, sniff Ethertype 0x8893 to see if it is one-way traffic:
diagnose sniffer packet any 'ether proto 0x8893' 4
As a workaround, change the Ethertypes used for Heartbeat:
config system ha
set ha-eth-type 8895
set l2ep-eth-type 889f
end
The above example changes the Ethertype of the HA heartbeat packets from 0x8890 to 0x8895 and changes the Ethertype of HA Telnet session packets from 0x8891 to 0x889f.
Note: Ethertype 0x8895 is also used for the 6000/7000 series.
Technical Tip: FPC/FPMs stuck at 'Waiting for data heartbeat'
Related article:
Technical Tip: HA session-sync-dev configuration
