Observability

Lance can publish operational metrics to your monitoring stack. The table below is the authoritative catalogue of the metrics Lance emits, shared verbatim with the Rust lance::metrics module documentation.

Lance publishes metrics through the metrics crate facade. Install any recorder (Prometheus, OpenTelemetry, etc.) in your application and Lance will emit into it; when no recorder is installed, emission is a cheap no-op. Metrics are only emitted when Lance is built with the metrics feature.

Object store metrics

These track I/O against the underlying object store. The base label identifies the store; its cardinality is controlled by the LANCE_OBJECT_STORE_METRICS_LABEL environment variable:

• scheme (default) — the scheme only (s3, gs, az, file, memory); low, bounded cardinality.
• full — the store's unique prefix (s3$my-bucket, az$container@account where Azure's account also matters), so multiple buckets on the same cloud can be told apart. Cardinality grows with the number of stores accessed.
• off — omit the base label entirely.

operation is one of get, put, put_part, head, list, delete, copy, rename, complete_multipart, or abort_multipart.

Request counts are per logical operation: a list or delete that spans many objects is one request, matching how backends batch them.

Metric	Type	Labels	Description
lance_object_store_requests_total	counter	operation, base	Object store requests issued.
lance_object_store_request_bytes_total	counter	operation, base	Bytes transferred by get/put requests. A get is counted once its response body has been fully read.
lance_object_store_request_duration_seconds	histogram	operation, base	Per-request latency, in seconds. For get this covers the full body transfer, not just time-to-first-byte.
lance_object_store_errors_total	counter	operation, base	Requests that returned an error.
lance_object_store_in_flight_requests	gauge	operation, base	Requests currently in flight.
lance_object_store_throttle_total	counter	status, base	Throttle responses (HTTP 429 / 503) seen at the HTTP layer, counted per attempt including retries. The status label is the numeric HTTP status.
lance_object_store_retryable_responses_total	counter	status, base	Retryable responses (HTTP 5xx / 429 / 408) seen at the HTTP layer, counted per attempt including retries. A superset of throttle_total; 409 (conflict) is excluded so commit conflicts are not counted.

lance_object_store_throttle_total and lance_object_store_retryable_responses_total are recorded only for the native cloud stores (S3, GCS, Azure); Opendal-backed stores bypass the HTTP client where the counters are installed, so they report the other object store metrics but not throttle/retryable counts.

Collecting metrics

Lance emits through the metrics crate facade, so it is not tied to a specific backend — you install a recorder/exporter and route the metrics wherever you like. Metrics are available from both the Rust and Python APIs.

Rust

Enable the metrics feature on the lance crate:

lance = { version = "...", features = ["metrics"] }

Then install any metrics-compatible recorder once at startup, before opening datasets. For example, with metrics-exporter-prometheus:

metrics_exporter_prometheus::PrometheusBuilder::new()
    .install()
    .expect("install Prometheus recorder");

Any recorder works — Prometheus, StatsD, an OpenTelemetry bridge, and so on. When no recorder is installed, emission is a cheap no-op.

Python

Unlike Rust, the Python bindings do not let you plug in an arbitrary recorder: bridging one across the FFI boundary into the Rust metrics facade would be complicated and inefficient. Instead pylance standardizes on OpenTelemetry, which has good Python support, as its recorder.

The pylance wheels are built with the metrics feature enabled. Install the OpenTelemetry extra and call instrument_lance_metrics, which registers Lance's metrics as observable instruments on your OpenTelemetry MeterProvider:

pip install "pylance[otel]"

from lance.otel import instrument_lance_metrics

# Uses the global MeterProvider; pass meter_provider=... to target a specific one.
instrument_lance_metrics()

From there the metrics flow through whatever OpenTelemetry pipeline you have configured (OTLP, Prometheus, console, …). Because OpenTelemetry has no asynchronous histogram instrument, histograms are exported Prometheus-style as three observable counters: <name>_bucket, <name>_count, and <name>_sum. Each <name>_bucket sample carries an le ("less than or equal") attribute giving that bucket's inclusive upper bound in the metric's unit; the bucket count is cumulative, covering every observation at or below le. For example, a lance_object_store_request_duration_seconds_bucket sample with le="0.5" counts all requests that completed in 0.5 seconds or less, while le="+Inf" is the total count.