Description
This article provides method to resolve FortiAnalyzer Big Data failed to collect diagnostic logs.
Scope
FortiAnalyzer Big Data.
Solution
FortiAnalyzer Big Data provides features of diagnostic logs to be collected via GUI for troubleshooting purposes. If users notice the error and fail to complete the task, the troubleshooting guidelines below are required.
In the GUI, go under Cluster Manager -> Top right corner (heartbeat icon) will record all the jobs performed on the FortiAnalyzer Big Data.
In the View all commands, a list of recent commands is provided. Select the magnifying glass icon in the 'Result' column to view the error details:
In the command result, the job has failed due to error below:
failed: [198.18.1.3] (item=bd-management-server-54fb495b7-v2lgg) =>
{
"ansible_loop_var":
"item"
"changed":
true
"cmd":
"kubectl --kubeconfig=/root/.kube/config cp /tmp/fazbd-diagnostic-logs.tar.gz bd-management-server-54fb495b7-v2lgg:/tmp/fazbd-diagnostic-logs.tar.gz"
"delta":
"0:00:00.124217"
"end":
"2025-03-05 15:36:14.996476"
"item":
"bd-management-server-54fb495b7-v2lgg"
"msg":
"non-zero return code"
"rc":
1
"start":
"2025-03-05 15:36:14.872259"
"stderr":
"error: cannot exec into a container in a completed pod; current phase is Failed"
"stderr_lines":[
0
:
"error: cannot exec into a container in a completed pod; current phase is Failed"
]
"stdout":
""
"stdout_lines":[]
}
Note:
The Ansible script will run a command in Master Blade leader -> default namespace -> bd-management-server pods. However, the command failed to be executed. Further troubleshooting is required in Master Blade leader.
- In GUI, go under Cluster Manager -> Hosts -> Master blade leader (Host with cluster icon circled in red box) -> CLI Console.
- Execute into the shell to further troubleshoot via Kubernetes
#fazbdctl execute shell
#kubectl get pods | grep bd-management-server
Note:
Kubernetes deployment has configured a scale of 2 for the bd-management server. The pods with 'Shutdown' status are not removed. As a result, the diagnostic logs ansible command is trying to access those pods and and failed to collect logs.
- Proceed to remove 'Shutdown' pods:
# kubectl delete pods <pod name>
After deleting those pods, FortiAnalyzer Big Data can collect the diagnostic logs via GUI successfully.
Note:
This issue is fixed with FortiAnalyzer Big Data v7.4.2 and above. It is recommended to upgrade the instances to fix permanently.
Related document:
