This issue is mainly reported for a single worker among other workers and the Supervisor, it is also noticed for the query worker as well.  The following errors have been observed in backend logs.  tail -f /opt/phoenix/log/phoenix.log | grep -i 'PH_DATAMANAGER_EVTIDX_WRITE_POST_ERROR'
2025-07-13T11:51:11.869198+03:00 fsiem-qwkr phDataManager[86900]: [PH_DATAMANAGER_EVTIDX_WRITE_POST_ERROR]:[eventSeverity]=PHL_ERROR,[procName]=phDataManager,[fileName]=EventIndexFile.cpp,[lineNumber]=548,[fileName]=/data/eventdb/CUSTOMER_1/internal/20279/486696-486719-2886866843/tmpseg-5-5811-2147894505-1752105600-1752191999/index/1039n.pst,[errReason]=Current posting is less than last posting,[phLogDetail]=Event index file write posting error
 The error can be resolved using the mentioned steps and approach.  Index file and prepare a list of index files, e.g., from above errors, /data/eventdb/CUSTOMER_1/internal/20279/486696-486719-2886866843/tmpseg-5-5811-2147894505-1752105600-1752191999/index/1039n.pst  The directory starting from tmpseg- does not contain data and only index files, which can be safely removed Take a backup of the index files. Remove the index files.
 rm -rf data/eventdb/CUSTOMER_1/internal/20279/486696-486719-2886866843/tmpseg-5-5811-2147894505-1752105600-1752191999/index/1039n.pst
 If the index files from the same directory have the same errors, if multiple index files have corruption, it would be best to remove the whole directory. Restart the phDataManager process using the commands below.
 su admin
phtools --stop phDataManager
phtools --start phDataManager
 Check again if the same errors have been observed If the errors are repetitive for multiple index files, the same steps need to be followed to remove corruption in the index files. The worker's performance needs to be checked. The query worker can be replaced with the new worker. Check if NFS performance is optimum; slow disk read/write operations may cause such an issue. Threads can be increased on NFS if necessary. The output below provides details if there is a latency observed for the NFS mount point.
nfsiostat 3 5
|