Technical Tip: FortiSIEM Java & opensdk symlink solution
Description
This article describes a method to resolve symlinks in FortiSIEM for Java & opensdk.
Scope
FortiSIEM.
Solution
FortiSIEM Java and opensdk symlink may break after an upgrade is completed or changes are made to the backend. In some cases, SVNlite and FortiInsightAI processes will be impacted and will fail to start. Run the following command to verify the status of the symlink:
[root@supervisor]# ls -la /opt
total 13068
lrwxrwxrwx 1 root root 12 Feb 29 22:26 Java -> /opt/openjdk
drwxr-xr-x 5 admin admin 225 Dec 9 08:57 jsreport
drwxr-xr-x 7 admin admin 206 Feb 6 12:54 node-rest-service
lrwxrwxrwx 1 root root 40 Feb 6 12:44 openjdk -> /etc/alternatives/java_sdk_1.8.0_openjdk
If both lines above are highlighted in red, it means that the symlink is broken. In the log, it will affect multiple processes and functions. Example of logs that may be encountered:
- Symlink missing for /opt/openjdk:
2024-04-12T11:22:57.260588+08:00 PCSGFSMC phDiscover[707017]: [PH_DISCOV_RUN_VM_TEST_CONN_ERROR]:[eventSeverity]=PHL_WARNING,[procName]=phDiscover,[fileName]=discover.cpp,[lineNumber]=2379,[hostIpAddr]=172.20.1.30,[exitValue]=32512,[phLogDetail]=Failed to execute runVmTestConn.sh
It can be further verified under this path:
# ls -la /etc/alternatives/java_sdk*
[root@supervisor alternatives]# ls -la java_sdk*
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
lrwxrwxrwx 1 root root 54 Dec 9 08:44 java_sdk_17 -> /usr/lib/jvm/java-17-openjdk-17.0.9.0.9-2.el8_8.x86_64
lrwxrwxrwx 1 root root 54 Dec 9 08:44 java_sdk_17_openjdk -> /usr/lib/jvm/java-17-openjdk-17.0.9.0.9-2.el8_8.x86_64
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk_openjdk -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
From the output above, java_sdk_1.8.0_openjdk is missing from the list. Hence, users need to rebuild the symlink with the command below:
#ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.<version>.el8.x86_64 /etc/alternatives/java_sdk_1.8.0_openjdk
Example:
#ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.382.b05-2.el8.x86_64 /etc/alternatives/java_sdk_1.8.0_openjdk
Note:
The Java version can be verified from other FortiSIEM instances with the same firmware version.
After the symlink is fixed, verify the output for /etc/alternatives and /opt:
[root@supervisor]# ls -la /etc/alternatives/java_sdk*
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
lrwxrwxrwx 1 root root 54 Dec 9 08:44 java_sdk_17 -> /usr/lib/jvm/java-17-openjdk-17.0.9.0.9-2.el8_8.x86_64
lrwxrwxrwx 1 root root 54 Dec 9 08:44 java_sdk_17_openjdk -> /usr/lib/jvm/java-17-openjdk-17.0.9.0.9-2.el8_8.x86_64
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk_1.8.0 -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk_1.8.0_openjdk -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
lrwxrwxrwx 1 root root 60 Dec 9 08:44 java_sdk_openjdk -> /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.392.b08-4.el8_8.x86_64
[root@supervisor]# ls -la /opt
total 13068
lrwxrwxrwx 1 root root 12 Feb 29 22:26 Java -> /opt/openjdk
drwxr-xr-x 5 admin admin 225 Dec 9 08:57 jsreport
drwxr-xr-x 7 admin admin 206 Feb 6 12:54 node-rest-service
lrwxrwxrwx 1 root root 40 Feb 6 12:44 openjdk -> /etc/alternatives/java_sdk_1.8.0_openjdk
Note:
SSH client, such as Putty or MobaXterm, will highlight the symlink in RED (broken) or BLUE (exists).
-
Symlink is broken for Java:
2024-04-12T11:22:57.260588+08:00 PCSGFSMC phAgentManager[3782]: [PH_AGENTMGR_JAVA_PIPE_FAILED]:[eventSeverity]=PHL_ERROR,[procName]=phAgentManager,[fileName]=phJavaAgentMonitor.cpp,[lineNumber]=1146,[exitValue]=24,[phLogDetail]=Pipe() failed.
2024-04-12T11:22:57.260588+08:00 PCSGFSMC phAgentManager[3782]: [PH_AGENTMGR_JAVA_AGENT_START_FAILED]:[eventSeverity]=PHL_ERROR,[procName]=phAgentManager,[fileName]=phJavaAgentMonitor.cpp,[lineNumber]=1221,[phLogDetail]=Failed to start Java agent, will retry.
The error above indicates the Java files are completely missing:
[root@supervisor]# ls -la /opt
total 13068
drwxr-xr-x 5 admin admin 225 Dec 9 08:57 jsreport
drwxr-xr-x 7 admin admin 206 Feb 6 12:54 node-rest-service
lrwxrwxrwx 1 root root 40 Feb 6 12:44 openjdk -> /etc/alternatives/java_sdk_1.8.0_openjdk
In that case, users need to rebuild the symlink for /opt/Java and reboot the nodes:
# ln -s /opt/openjdk /opt/Java
# phxctl reboot
