Troubleshooting Tip: Resolving ClickHouse Error “Too many parts. Merges are processing significantly slower than inserts”

This error typically happens in any of the following conditions:

• Too many small inserts: frequent, tiny write batches instead of larger, optimized ones.

• Insert rate exceeds merge rate: spikes in data volume or collector backlog cause merges to fall behind.

• Inefficient merge operations: high-cardinality partitions, poor table design, or slow disk I/O.

• Broken parts: corrupted or incomplete parts that block merges.

Initial Investigation:

Check ClickHouse logs for the TOO_MANY_PARTS error:

grep -i "TOO_MANY_PARTS" /var/log/clickhouse-server/clickhouse-server.log

ClickHouse failed. respCode:500 resp:Code: 252. DB::Exception: Too many parts (3001 with average size of 282.52 KiB) in table 'fsiem.events_replicated (75dbec0-3f31-4405-b818-63161137fcac)'. Merges are processing significantly slower than inserts. (TOO_MANY_PARTS) (version 23.8.16.40 (official build))

Root Cause:

ClickHouse uses the MergeTree engine, which writes each insert as a small part on disk. Background threads merge these parts into larger files. If inserts arrive faster than merges complete, the table grows thousands of tiny parts, triggering the TOO_MANY_PARTS exception.

The current max part setup on each click house node is set to 5 located here: 

cat /etc/clickhouse-server/config.xml | grep -i max_suspicious_broken_parts
<max_suspicious_broken_parts>5</max_suspicious_broken_parts>

This means ClickHouse will tolerate up to 5 broken parts before halting. Broken parts may result from disk issues, unclean shutdowns, or corrupted merges.

Recommended Fix: 

Set this parameter to '0' to avoid processing any corrupted data parts. 

1. Back up the ClickHouse config:

cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config.xml_Back

2. Edit the configuration file:

vi /etc/clickhouse-server/config.xml

3. Update the parameter to avoid broken parts (default is 5):

Summary:

This issue stems from ClickHouse being overwhelmed by too many small parts and not merging them fast enough. The fix involves:

• Identifying large numbers of unmerged parts.

• Enforcing stricter corruption checks by  setting <max_suspicious_broken_parts>0</max_suspicious_broken_parts>

ClickHouse refuses to process any broken part. The benefit is:

• Fail fast: As soon as one corrupted or inconsistent part appears, ClickHouse halts instead of silently continuing.

• Protects data integrity: Prevents broken parts from being merged into healthy data.

• Speeds up resolution: Engineers immediately see the problem instead of chasing slow merges or unexplained ingestion failures later.

As a result, changing the value to '0' does not directly make merges faster — it ensures that merges are not blocked or delayed by bad parts hiding in the system.