Description
This article provides troubleshooting commands that can be used when facing LACP (Link Aggregation Control Protocol) issues on a FortiGate.
The related articles provide additional information about LACP.
Scope
FortiGate supporting LACP: Models 310B (Recommended on port handled by the same NP2), 300A, 400A, 500A, and 800 or higher.
Solution
There are three modes of LACP on the FortiGate:
- Active: actively use LACP to negotiate 802.3ad aggregation.
- Passive: passively use LACP to negotiate 802.3ad aggregation.
- Static: use static aggregation, do not send and ignore any LACP messages (all ports in the LAG will send traffic).
Depending on the remote device, it may be necessary to adapt the LACP mode appropriately.
The 'lacp-ha-slave enable' command allows subordinate units in HA Cluster to participate in LACP negotiation by allowing them to send/receive LACP messages. When disabled, it blocks HA slave units from sending/receiving LACP messages.
The lacp-speed determines how often the interface sends LACP messages. By default, it is set to slow which sends LACP messages every 30 seconds. However, when it is set to fast it sends LACP message every second.
There are three types of traffic distribution across the ports in the LACP bundle. Distribution of sessions uses a hash of either L2 / L3 / L4 header fields divided by the number of physical interfaces in the link aggregation group to determine a remainder value that identifies the link number to use.
Example of an LACP configuration.
config system interface
edit "lacp_ports"
set vdom "root"
set type aggregate
set member "port1" "port2"
set description 'lacp_example'
set lacp-mode active <----- Default.
set lacp-ha-slave enable <----- Default.
set lacp-speed slow <----- Default.
set algorithm L4 <----- Default.
next
end
The following CLI commands can be used to check the ports and LAG (Link Aggregation Group) status.
- Example of LACP operational information when ports are up and in the LAG.
diag netlink aggregate name your_aggregate_link
LACP flags: (A|P)(S|F)(A|I)(I|O)(E|D)(E|D)
(A|P) - LACP mode is Active or Passive
(S|F) - LACP speed is Slow or Fast
(A|I) - Aggregatable or Individual
(I|O) - Port In sync or Out of sync
(E|D) - Frame collection is Enabled or Disabled
(E|D) - Frame distribution is Enabled or Disabled
status: up
distribution algorithm: L4
LACP mode: active
LACP speed: slow
LACP HA: enable
aggregator ID: 1
ports: 2
actor key: 17
actor MAC address: 00:09:0f:68:35:94
partner key: 17
partner MAC address: 00:09:0f:68:37:d8
slave: port7
status: up
link failure count: 3
permanent MAC addr: 00:09:0f:68:35:94
actor state: ASAIEE
partner state: ASAIEE
aggregator ID: 1
slave: port8
status: up
link failure count: 2
permanent MAC addr: 00:09:0f:68:35:95
actor state: ASAIEE
partner state: ASAIEE
aggregator ID: 1
In this example, the aggregator IDs have the same value on both ports and globally (ID=1). This means that both ports are operational in the LAG.
- Example of LACP operational information when both ports are up, but there is no LACPDU exchange on port 5(*).
diag netlink aggregate name your_aggregate_link
LACP flags: (A|P)(S|F)(A|I)(I|O)(E|D)(E|D)
(A|P) - LACP mode is Active or Passive
(S|F) - LACP speed is Slow or Fast
(A|I) - Aggregatable or Individual
(I|O) - Port In sync or Out of sync
(E|D) - Frame collection is Enabled or Disabled
(E|D) - Frame distribution is Enabled or Disabled
status: up
distribution algorithm: L3
LACP mode: active
LACP speed: slow
LACP HA: enable
aggregator ID: 1
ports: 1
actor key: 17
actor MAC address: 00:09:0f:71:1f:22
partner key: 45
partner MAC address: 00:0d:66:2f:2b:40
slave: port5
status: up
link failure count: 19
permanent MAC addr: 00:09:0f:71:1f:22
actor state: ASAIDD <----- DISABLED.
partner state: ASIODD <----- OUT OF SYNC / DISABLED.
aggregator ID: 2
slave: port6
status: up
link failure count: 2
permanent MAC addr: 00:09:0f:71:1f:23
actor state: ASAIEE
partner state: ASAIEE
aggregator ID: 1
Note that port5 is in a different aggregator (2) above than the global aggregator ID (1). In this case, only port6 is operational in the LAG.
(*) If both FortiGates or equipment are connected via an intermediate L2 switch, make sure that is passes LACPDU packets.
Link Aggregation Control Protocol "LACPDU" packet format and how to get a sniffer trace from the CLI:
diag sniffer packet your_aggregate_link
2.546898 aggreg_link -- 802.3ad LACPDU (65535,00-09-0F-68-37-D8,0017,0255,0002) ASAIEE (65535,00-09-0F-68-35-94,0017,0255,0002) ASAIEE
0x0000 0180 c200 0002 0009 0f68 37d9 8809 0101 .........h7......
Dst Multicast - src=lowest MAC of all ports in the LAG - Eth frame type.
If there is a lot of traffic, only LACP traffic can be fully captured with:
diagnose sniffer packet any "ether proto 0X8809" 6 0 a
The following information should be provided when opening a ticket with TAC Support for an LACP issue:
- The FortiGate configuration file.
- Information about how the two devices are connected together for this LACP bundle (direct cables or fibers/Intermediate L2 or metro device between the FortiGate and the other device).
- Results of the following CLI commands:
diag netlink aggregate name your_aggregate_link
diagnose hardware deviceinfo nic <all_interface_in_your_aggregation>
diag sniff packet your_aggregate_link " " 6 0 l
- Output of the sniffer trace gathered on the other end (port-mirroring or PCAP).
- Also, for a single session, LACP can distribute traffic across multiple interfaces, which may lead to packet reordering issues for applications that require ordered packet delivery. If this is causing problems, consider using static aggregation, aligning hash settings, or temporarily disabling one interface to ensure that all traffic for a session is handled consistently.
Related articles:
