> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-mintlify-fbfa8bee.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> Monitor ClickPipes using Prometheus metrics.

# Monitoring ClickPipes

In addition to in-console monitoring, ClickPipes exposes metrics to a [Prometheus-compatible endpoint](/products/cloud/features/monitoring/prometheus) for scraping. These metrics are published with other ClickHouse Cloud service metrics, and allow you to integrate ClickPipes monitoring with your existing observability stack (e.g., [Grafana](/products/cloud/features/monitoring/prometheus#integrating-with-grafana), [Datadog](/products/cloud/features/monitoring/prometheus#integrating-with-datadog)).

<h2 id="metric-labels">
  Metric labels
</h2>

ClickPipes metrics use the [standard service-level labels](/products/cloud/features/monitoring/prometheus#metric-labels) (`clickhouse_org`, `clickhouse_service`, `clickhouse_service_name`) plus the following ClickPipes-specific labels:

| Label              | Description                                                             |
| ------------------ | ----------------------------------------------------------------------- |
| `clickpipe_id`     | The unique identifier of the ClickPipe                                  |
| `clickpipe_name`   | The name of the ClickPipe                                               |
| `clickpipe_source` | The source type (e.g., `kafka`, `postgres`, `s3`)                       |
| `clickpipe_state`  | The state of the ClickPipe (e.g., `Stopped`, `Provisioning`, `Running`) |

<h3 id="sample-response">
  Sample response
</h3>

The following shows what ClickPipes metrics look like in a Prometheus scrape response:

```response theme={null}
# HELP ClickPipes_Info Always equal to 1. Label "clickpipe_state" contains the current state of the pipe: Stopped/Provisioning/Running/Paused/Failed
# TYPE ClickPipes_Info gauge
ClickPipes_Info{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent",clickpipe_state="Running"} 1

# HELP ClickPipes_FetchedEvents_Total Total number of records fetched from the source.
# TYPE ClickPipes_FetchedEvents_Total counter
ClickPipes_FetchedEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5535376

# HELP ClickPipes_SentEvents_Total Total number of records sent to ClickHouse
# TYPE ClickPipes_SentEvents_Total counter
ClickPipes_SentEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5534250

# HELP ClickPipes_FetchedBytes_Total Total uncompressed bytes fetched from the source.
# TYPE ClickPipes_FetchedBytes_Total counter
ClickPipes_FetchedBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 873286202

# HELP ClickPipes_SentBytes_Total Total uncompressed bytes sent to ClickHouse.
# TYPE ClickPipes_SentBytes_Total counter
ClickPipes_SentBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 477187967

# HELP ClickPipes_Errors_Total Total errors ingesting data.
# TYPE ClickPipes_Errors_Total counter
ClickPipes_Errors_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 0

# HELP ClickPipes_Latency Time in milliseconds between when the record was produced to when it was written to ClickHouse.
# TYPE ClickPipes_Latency gauge
ClickPipes_Latency{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 340
```

<h2 id="available-metrics">
  Available metrics
</h2>

<h3 id="metrics-data-transfer">
  Data transfer
</h3>

| Metric                                     | Type    | Description                                                                                                                        |
| ------------------------------------------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `ClickPipes_FetchedBytes_Total`            | Counter | Total uncompressed bytes fetched from the source.                                                                                  |
| `ClickPipes_FetchedBytesCompressed_Total`  | Counter | Total compressed bytes fetched from the source. If the source data is uncompressed, equivalent to `ClickPipes_FetchedBytes_Total`. |
| `ClickPipes_FetchedBytesInitialLoad_Total` | Counter | **CDC ClickPipes** Total uncompressed bytes fetched during the initial load phase.                                                 |
| `ClickPipes_FetchedBytesResync_Total`      | Counter | **CDC ClickPipes** Total uncompressed bytes fetched during a resync operation.                                                     |
| `ClickPipes_SentBytes_Total`               | Counter | Total uncompressed bytes sent to ClickHouse.                                                                                       |
| `ClickPipes_SentBytesCompressed_Total`     | Counter | Total compressed bytes sent to ClickHouse.                                                                                         |

<h3 id="metrics-events">
  Events and records
</h3>

| Metric                              | Type    | Description                                                                                                                                     |
| ----------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `ClickPipes_FetchedEvents_Total`    | Counter | Total number of records fetched from the source.                                                                                                |
| `ClickPipes_SentEvents_Total`       | Counter | Total number of records sent to ClickHouse.                                                                                                     |
| `ClickPipes_FetchedEvents_PerTable` | Gauge   | **CDC ClickPipes** Number of records fetched per destination table and event type. Uses additional `destination_table` and `event_type` labels. |

<h3 id="metrics-errors">
  Errors
</h3>

| Metric                    | Type    | Description                                                                                 |
| ------------------------- | ------- | ------------------------------------------------------------------------------------------- |
| `ClickPipes_Errors_Total` | Counter | Total [errors](/integrations/clickpipes/home#error-reporting) encountered during ingestion. |

<h3 id="metrics-latency">
  Latency
</h3>

| Metric                                    | Type          | Description                                                                                                               |
| ----------------------------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------- |
| `ClickPipes_Latency`                      | Gauge (*avg*) | Time in milliseconds between when a record was produced at the source and when it was written to ClickHouse.              |
| `ClickPipes_SourceReplicationLatency_MiB` | Gauge (*avg*) | **CDC ClickPipes** Replication lag at the source in MiB. For Postgres ClickPipes, this reflects the replication slot lag. |

<h3 id="metrics-resources">
  Resource usage
</h3>

| Metric                           | Type           | Description                                                    |
| -------------------------------- | -------------- | -------------------------------------------------------------- |
| `ClickPipes_Replica_CPUUsage`    | Gauge (*avg*)  | Average CPU usage of the ingestion replicas, in millicores.    |
| `ClickPipes_Replica_CPULimit`    | Gauge (*last*) | Configured CPU limit of the ingestion replicas, in millicores. |
| `ClickPipes_Replica_MemoryUsage` | Gauge (*avg*)  | Average memory usage of the ingestion replicas, in bytes.      |
| `ClickPipes_Replica_MemoryLimit` | Gauge (*last*) | Configured memory limit of the ingestion replicas, in bytes.   |

<h3 id="metrics-state">
  State and progress
</h3>

| Metric                          | Type  | Description                                                                            |
| ------------------------------- | ----- | -------------------------------------------------------------------------------------- |
| `ClickPipes_Info`               | Gauge | Always equal to `1`. Use the `clickpipe_state` label to filter or alert on pipe state. |
| `ClickPipes_LastFetchedBatchId` | Gauge | ID of the last batch fetched from the source.                                          |
| `ClickPipes_LastSentBatchId`    | Gauge | ID of the last batch sent to ClickHouse.                                               |

<h2 id="related">
  Related pages
</h2>

* [Prometheus endpoint](/products/cloud/features/monitoring/prometheus) — Endpoint reference, authentication, and setup for external tools (e.g., Grafana, Datadog)
* [Error reporting](/integrations/clickpipes/home#error-reporting) — Where record and system errors are stored