Troubleshooting Tip: High CPU due to a network loop in FortiGate

[tino_p]

Description	This article describes how to detect the potential network loop, which causes high CPU usage in the FortiGate firewall.
Scope	FortiGate before v7.6.0.
Solution	Seeing a high CPU because of softirq may be a sign of a potential network loop, especially in a FortiGate that has Transparent mode OR a switch-interface. For example, this is the output from the command 'diagnose sys mpstat'.   In this case, it is recommended to check the outputs of these commands to identify the bridging information: diagnose netlink brctl list   Perform further checks with each of the bridges by inputting the following command several times in a specific interval (such as 5 times every 5 seconds): diagnose netlink brctl name host [name] This will make it possible to check if any MAC address appears in different interfaces. In such cases, it is recommended to review the network design. By right, a MAC address should appear and stick to only 1 interface. For example, MAC addresses 24:5e:be:12:da:88 being on both the interfaces b and dmz will indicate a network loop.    From v7.6.0, there is a new feature where FortiOS logs MAC address flapping events to aid in quickly identifying this behavior. (Feature ID = 974975.) It is possible to use 'diagnose netlink interface packet-rate' to see the PPS (packets per second) for each interface. This may also help narrow down which interface has a loop. See this KB article for more info: Troubleshooting Tip: How to Monitor Incoming (RX) and Outgoing (TX) Packets Per Second (PPS) on Fort...  Related documents: FortiOS 7.6.0 release notes - new features. Logging MAC address flapping events  Troubleshooting Tip: Check SoftIrq increments (recommended when experiencing high CPU usage) Technical Tip: Software switch causing high CPU softirq usage and network downtime Checking the bridging information in transparent mode