Troubleshooting Tip: PostgreSQL docker container fails to start after unexpected reboot due to stale PID file on FortiClient EMS server

Description:

After an unexpected reboot of a server running FortiClient EMS and PostgreSQL inside a Docker container, the EMS service may recover successfully, but the PostgreSQL container may fail to start or remain running.

This behavior is often caused by a stale 'repmgrd' PID file left over from before the reboot, preventing PostgreSQL from initializing properly.

After the reboot, the PostgreSQL container repeatedly fails to start and enters a restart loop. Log shows the following error:

[ERROR] PID file "/tmp/repmgrd.pid" exists and seems to contain a valid PID
[HINT] if repmgrd is no longer alive, remove the file and try to restart repmgrd

This error occurs because the 'repmgrd' daemon detects a stale process ID file and incorrectly assumes an instance is still running.

Solution:

• Access the problematic PostgreSQL container manually:

docker exec -it <container_name> bash

• Remove the stale PID file:

rm -f /tmp/repmgrd.pid

• Exit the container:

Exit

• Restart the container:

docker restart container_name

• Verify the cluster membership:

docker exec -it <container_name> bash -c "/opt/bitnami/scripts/postgresql-repmgr/entrypoint.sh repmgr -f /opt/bitnami/repmgr/conf/repmgr.conf cluster show"

The node should be online and properly rejoined as a standby.

Additional Notes:

Due to the restart loop of the container, it may be necessary to perform the above actions quickly before the container is restarted again.