Ingest data from StatsD

Deprecated

Starting with Dynatrace version 1.300, histogram ingestion is supported directly via OTLP ingest API. Consequently, using the Collector as a workaround for StatsD histograms is not required. Dynatrace recommends ingesting histograms directly without additional data manipulation.

The following configuration example shows how you configure a Collector instance to ingest data from an existing StatsD setup and import it as an OTLP request into Dynatrace.

Prerequisites

Demo configuration

receivers:
  statsd:
    endpoint: 0.0.0.0:8125
    timer_histogram_mapping:
      - statsd_type: "histogram"
        observer_type: "histogram"
        histogram:
          # max size for the auto-scaling exponential histogram OTLP metric
          # see below for details
          max_size: 100
      - statsd_type: "timing"
        observer_type: "histogram"
        histogram:
          max_size: 100
      - statsd_type: "distribution"
        observer_type: "histogram"
        histogram:
          max_size: 100


# Processors for histogram observers
# See histogram summaries documentation linked below for details
processors:
  transform:
    metric_statements:
      - context: metric
        statements:
          - extract_count_metric(true) where type == METRIC_DATA_TYPE_HISTOGRAM
          - extract_sum_metric(true) where type == METRIC_DATA_TYPE_HISTOGRAM
          - convert_sum_to_gauge() where IsMatch(name, ".*_sum")
  filter:
    metrics:
      metric:
        - type == METRIC_DATA_TYPE_HISTOGRAM


exporters:
  otlphttp:
    endpoint: ${env:DT_ENDPOINT}
    headers:
      Authorization: "Api-Token ${env:DT_API_TOKEN}"


service:
  pipelines:
    metrics:
      receivers: [statsd]
      processors: [transform,filter]
      exporters: [otlphttp]
Configuration validation

Validate your settings to avoid any configuration issues.

Components

For our configuration, we configure the following components.

Receivers

Under receivers, we specify the statsd receiver as active receiver component for our Collector instance. We configure the receiver to listen on all network interfaces to port 8125, which is the port typically used for StatsD.

The receiver is configured to aggregate histogram, timer, and distribution messages into exponential histograms, which are later processed for Dynatrace ingestion. The receiver uses auto-scaling exponential histograms, and we have selected a maximum size of 100. This means that the histogram will begin with very granular bucket boundaries and automatically re-scale itself if it receives data points that would result in more than 100 buckets.

For a full list of configuration parameters and supported StatsD metric types, see the StatsD receiver documentation.

Processors

Under processors, we configure the transform and filter processors to prepare the histogram OTLP metrics for Dynatrace. For details, see the histogram summaries documentation.

Exporters

Under exporters, we specify the default otlphttp exporter and configure it with our Dynatrace API URL and the required authentication token.

For this purpose, we set the following two environment variables and reference them in the configuration values for endpoint and headers.

Service pipeline

Under service, we assemble our receiver, processors, and exporter objects into a metrics pipeline, which accepts StatsD data and ingests it into Dynatrace.

Limits and limitations

Metrics are ingested using the OpenTelemetry protocol (OTLP) via the Dynatrace OTLP API and are subject to the API's limits and restrictions. For more information see:

Related topics