Troubleshooting Tip: FortiAnalyzer Fabric management troubleshooting steps

[tthrilok]

Description	This article describes the initial troubleshooting steps to be taken for FortiAnalyzer Fabric management supervisor and member connectivity issues.
Scope	FortiAnalyzer.
Solution	To configure Fabric management for supervisors and members, follow the instructions in the deployment guide. FortiAnalyzer members and supervisors use TCP ports 514 and 6443 (default port) to get into Fabric management.  If FortiAnalyzer members and supervisors are running over different networks, make sure to whitelist TCP ports 514 and 6443, or any configured port for Fabric management on any firewalls in transit. Member to supervisor Fabric management flow: Members first initiate TCP port 514 connection to the supervisor in order to be connected to the supervisor OFTP. A separate connection is initiated by a member on TCP port 6443 or any configured port to the supervisor's IP (configured in the member). Once the TCP connection on 6443 or any configured port is successful, the member is available for authorization on the supervisor. After the authorization of the member on the supervisor under System Settings -> Fabric Management, the member can be seen under the Device Manager and logs are received from the member on the supervisor. Follow these troubleshooting steps once Fabric management is configured for the supervisor and members: If members display 'wrong fabric config' at the bottom right in red ribbon, and if the supervisor shows the 'Auth Status' of a member is unknown:  Confirm that the 'Cluster Name' matches on both the supervisor and member. Once the Cluster Name is corrected, select Apply to apply it to the member. The member FortiAnalyzer should display 'Successfully joined Fabric FAZ cluster' at the right bottom in green ribbon. At this stage, the member will be waiting for authorization to be completed on the supervisor. The member authorization section will display 'Pending'. After authorization, the member will display 'Accepted' and the supervisor will display 'In Sync'. If the member is not displayed in the supervisor for authorization, and if the member displays 'Unknown' with a 'Failed to join Fabric FAZ Cluster' error: Validate the IP of the supervisor on the member. Confirm that TCP port 514 is allowed from the member to the supervisor. If the member is not displayed in the supervisor for authorization, and if the member displays 'Unknown': It could be because the supervisor is not able to authenticate members with the OFTP daemon. Validate whether OFTP is running on the supervisor if the issue is with all the members. If it is specific to one member or a small number of members, validate the certificate sent by the members to the supervisor: the members' serial numbers should match the certificate name in the certificate that each member is sending to supervisor. Confirm this using OFTPD debugging on the supervisor. The debug output should display 'Error invalid member SN: FAZ-VM00000XYZAB, cert sn: FAZ-VM0000000001.' If this error is observed, see Technical Tip: FortiAnalyzer certificate does not reflect the correct serial number. The scenario mentioned above is one possibility. Normally, the situation happens if the supervisor is unable to authenticate the member's certificate. If the member displays 'Unknown' authorization with 'Failed to join Fabric FortiAnalyzer Cluster' at the bottom right in red ribbon, and the supervisor displays members as Down: Confirm whether the supervisor is able to receive traffic from the Fabric management port 6443 (default) or any non-default port traffic from members. If the supervisor is receiving the traffic and trying to reset, confirm whether there are any local-in policies on the supervisor. Confirm that 'FortiAnalyzer Fabric' is enabled under the Administrative Access section of the supervisor interface. Confirm whether any trusted-list is configured in the supervisor under 'config system soc-fabric'. Confirm whether the trusted hosts are configured under admin settings. If so, add the member IP under trusted hosts of any admin in the supervisor. If issues are still encountered, validate fabricsyncd is running on the supervisor. Raise a FortiCare ticket after collecting the output of the following debug commands from the supervisor and member: diagnose test app fabricsyncd 1 diagnose test app fabricsyncd 3 diagnose test app fabricsyncd 4 diagnose test app fabricsyncd 82 nodes diagnose test app fabricsyncd 82 auth info diagnose test app fabricsyncd 82 auth stats