Summary

The v4.4.6 self-hosted Docker stack's ClickHouse container sustained significant CPU while there was no application data or workload. The CPU was spent flushing and merging ClickHouse's own default system.*_log telemetry.

This matches ClickHouse/ClickHouse#60016, which was closed after a configuration workaround rather than a ClickHouse code fix.

Environment

• Trigger.dev v4.4.6
• bitnamilegacy/clickhouse:latest, resolving to ClickHouse 25.5.2.47
• Linux arm64
• Trigger ClickHouse application tables present but empty
• Only recurring client query: the Compose healthcheck's SELECT 1 every 5 seconds

Measurements

Before changing the configuration:

• ClickHouse CPU: 23.63% of one core over a quiet 30-second sample
• system.trace_log: 100.9 million rows / 2.01 GiB
• system.asynchronous_metric_log: 782.1 million rows / 459 MiB
• system.metric_log: 5.2 million rows / 970 MiB
• system.latency_log: 5.2 million rows / 88 MiB
• system.processors_profile_log: 2.5 million rows / 64 MiB
• ClickHouse data volume: 3.9 GiB

Thread sampling showed MergeMutate as the dominant CPU owner. Live system.merges samples repeatedly caught system.asynchronous_metric_log and occasionally system.latency_log.

Configuration findings

The shipped hosting/docker/clickhouse/override.xml contains a <profiles><default> block, but it is mounted under ClickHouse config.d. In this Bitnami image, those user-profile values did not become effective:

• max_block_size remained at its default rather than 8192
• log_processors_profiles remained enabled
• query CPU/real-time profiling remained at 1 second
• memory_profiler_step remained 4 MiB

Mounting a separate file under /opt/bitnami/clickhouse/etc/users.d/ made the profile values effective.

The 5-second healthcheck also generated query/profile/trace records. The client accepts per-invocation settings that prevent this:

• --log_queries=0
• --log_processors_profiles=0
• --memory_profiler_step=0

Result after local deployment policy

We:

1. Disabled persistent asynchronous_metric_log, metric_log, latency_log, processors_profile_log, and trace_log through a config.d removal overlay.
2. Disabled the corresponding query/server profilers through a users.d profile and server config.
3. Retained slow/error query diagnostics with a 1-second threshold.
4. Made the healthcheck non-logging and changed its interval to 30 seconds.
5. Pinned the existing compatible ClickHouse image instead of inheriting latest.

Results:

• First quiet 5-minute CPU average: 2.22%
• Subsequent 9-minute average: 1.86%
• Post-cleanup 1-minute average: 1.83%
• ClickHouse data volume after dropping only the disabled historical telemetry tables: 183 MiB
• Trigger health remained green throughout verification

Suggested upstream changes

Could the self-hosted Docker configuration:

1. Move query-profile settings from config.d to a mounted users.d file so the advertised low-memory defaults are effective?
2. Disable or substantially reduce unconsumed high-frequency ClickHouse system telemetry for self-hosted installations?
3. Make the ClickHouse healthcheck non-logging and less frequent?
4. Pin a tested ClickHouse image tag rather than defaulting to bitnamilegacy/clickhouse:latest?

I can prepare a PR if the maintainers agree with the intended policy and retention trade-off.