Troubleshooting Tip: Resolving ClickHouse Error “Too many parts. Merges are processing significantly slower than inserts”

[aebadi]

Description	This error happens when ClickHouse creates too many small data parts in a table. The background merge process cannot keep up, causing performance issues. As a result, queries or data ingestion may fail with the error: Uploading events to ClickHouse failed. respCode:500 resp:Code: 252. DB::Exception: Too many parts
Scope	FortiSIEM environments that use ClickHouse as the backend database.
Solution	This error typically happens in any of the following conditions: Too many small inserts: frequent, tiny write batches instead of larger, optimized ones. Insert rate exceeds merge rate: spikes in data volume or collector backlog cause merges to fall behind. Inefficient merge operations: high-cardinality partitions, poor table design, or slow disk I/O. Broken parts: corrupted or incomplete parts that block merges. Initial Investigation: Check ClickHouse logs for the TOO_MANY_PARTS error: grep -i "TOO_MANY_PARTS" /var/log/clickhouse-server/clickhouse-server.log ClickHouse failed. respCode:500 resp:Code: 252. DB::Exception: Too many parts (3001 with average size of 282.52 KiB) in table 'fsiem.events_replicated (75dbec0-3f31-4405-b818-63161137fcac)'. Merges are processing significantly slower than inserts. (TOO_MANY_PARTS) (version 23.8.16.40 (official build)) Root Cause: ClickHouse uses the MergeTree engine, which writes each insert as a small part on disk. Background threads merge these parts into larger files. If inserts arrive faster than merges complete, the table grows thousands of tiny parts, triggering the TOO_MANY_PARTS exception. The current max part setup on each click house node is set to 5 located here:  cat /etc/clickhouse-server/config.xml | grep -i max_suspicious_broken_parts <max_suspicious_broken_parts>5</max_suspicious_broken_parts> This means ClickHouse will tolerate up to 5 broken parts before halting. Broken parts may result from disk issues, unclean shutdowns, or corrupted merges. Recommended Fix:  Set this parameter to '0' to avoid processing any corrupted data parts.  Back up the ClickHouse config: cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config.xml_Back Edit the configuration file: vi /etc/clickhouse-server/config.xml Update the parameter to avoid broken parts (default is 5): <max_suspicious_broken_parts>0</max_suspicious_broken_parts> Restart the ClickHouse service: systemctl start clickhouse-server Summary: This issue stems from ClickHouse being overwhelmed by too many small parts and not merging them fast enough. The fix involves: Identifying large numbers of unmerged parts. Enforcing stricter corruption checks by  setting <max_suspicious_broken_parts>0</max_suspicious_broken_parts> ClickHouse refuses to process any broken part. The benefit is: Fail fast: As soon as one corrupted or inconsistent part appears, ClickHouse halts instead of silently continuing. Protects data integrity: Prevents broken parts from being merged into healthy data. Speeds up resolution: Engineers immediately see the problem instead of chasing slow merges or unexplained ingestion failures later. As a result, changing the value to '0' does not directly make merges faster — it ensures that merges are not blocked or delayed by bad parts hiding in the system.