Troubleshooting Tip: phParser not starting on collector after disk cleanup removes /var/log/httpd

Description

This article describes how to resolve an issue on a FortiSIEM collector where the phParser process fails to start after a disk full condition. In some cases, users may accidentally remove the /var/log/httpd directory while attempting to free disk space, resulting in multiple service failures including Apache (httpd) and phParser.

Scope

FortiSIEM Collector v7.x+.

Solution

Symptoms:

One or more of the following symptoms may be observed:

• phParser process is not running.

• Apache (httpd) service fails to start.

• phoenix.log is empty.

• /var/log/httpd/ directory is missing.

• /etc/rsyslog.conf contains invalid or modified configuration entries.

• Port 514 is occupied by rsyslog instead of phParser.

Example:

ss -ltnp | grep :514

LISTEN 0 25 0.0.0.0:514 0.0.0.0:* users:(("rsyslogd",pid=1398,fd=6))
LISTEN 0 25 [::]:514    [::]:*    users:(("rsyslogd",pid=1398,fd=7))

Expected behavior:

• Port 514 should be owned by the phParser process.

Root cause: This issue may occur after manually clearing disk space on the collector, where critical directories or log files required by Apache (httpd) are accidentally removed.

As a result:

• httpd cannot start properly.

• phParser fails to initialize.

• rsyslog incorrectly binds to port 514, causing a conflict.

Solution: Follow the steps below to restore the required directories, permissions, and service configurations.

1. Recreate the required Apache log directory and files:

mkdir -p /var/log/httpd
touch /var/log/httpd/access_log
touch /var/log/httpd/error_log
touch /var/log/httpd/modsec_audit.log
touch /var/log/httpd/modsec_debug.log
chown -R root:root /var/log/httpd

2. Restore correct ownership and permissions:

chmod 755 /var/log/httpd
chmod 644 /var/log/httpd/access_log
chmod 644 /var/log/httpd/error_log
chmod 640 /var/log/httpd/modsec_audit.log
chmod 640 /var/log/httpd/modsec_debug.log

3. Restart and Verify the Apache (httpd) service:

systemctl restart httpd
systemctl status httpd

4. Correct the rsyslog configuration. Edit the /etc/rsyslog.conf file.

Replace the following:

<truncated>
. @127.0.0.1:6100
<truncated>
# Save ACE log messages to phoenix.log. The facility should be as the same as defined in phoenix/ext/ACE/src/ace/Default_Constants.h
user.*
<truncated>

With:

<truncated>
*.info;cron.none                                        @127.0.0.1:6100
<truncated>
# Save ACE log messages to phoenix.log. The facility should be as the same as defined in phoenix/ext/ACE/src/ace/Default_Constants.h
user.*                                                     /opt/phoenix/log/phoenix.log
<truncated>

5. Restart the rsyslog service:

systemctl restart rsyslog

6. Verify phParser status:

ps -ef | grep -i phParser
ss -ltnp | grep :514

Expected output should show phParser instead of rsyslog.

Result:

After completing the above steps:

• Apache (httpd) should start successfully.

• phParser should bind to port 514.

• Log ingestion should resume normally.

• The collector should recover from the disk full condition.