Enrich from Kubernetes

The following configuration example shows how you configure a Collector instance to enrich OTLP request with additional Kubernetes metadata.

Prerequisites

A deployed ActiveGate for Kubernetes API monitoring
One of the following Collector distributions with the Kubernetes Attributes and Transform processors
Collector deployed in agent mode
The API URL of your Dynatrace environment
An API token with the relevant access scope
Kubernetes configured for the required role-based access control

Demo configuration

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  k8sattributes:
    extract:
      metadata:
        - k8s.pod.name
        - k8s.pod.uid
        - k8s.deployment.name
        - k8s.statefulset.name
        - k8s.daemonset.name
        - k8s.cronjob.name
        - k8s.namespace.name
        - k8s.node.name
        - k8s.cluster.uid
    pod_association:
      - sources:
        - from: resource_attribute
          name: k8s.pod.name
        - from: resource_attribute
          name: k8s.namespace.name
      - sources:
        - from: resource_attribute
          name: k8s.pod.ip
      - sources:
        - from: resource_attribute
          name: k8s.pod.uid
      - sources:
        - from: connection
  transform:
    error_mode: ignore
    trace_statements:
      - context: resource
        statements:
          - set(attributes["dt.kubernetes.workload.kind"], "statefulset") where IsString(attributes["k8s.statefulset.name"])
          - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.statefulset.name"]) where IsString(attributes["k8s.statefulset.name"])
          - set(attributes["dt.kubernetes.workload.kind"], "deployment") where IsString(attributes["k8s.deployment.name"])
          - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.deployment.name"]) where IsString(attributes["k8s.deployment.name"])
          - set(attributes["dt.kubernetes.workload.kind"], "daemonset") where IsString(attributes["k8s.daemonset.name"])
          - set(attributes["dt.kubernetes.workload.name"], attributes["k8s.daemonset.name"]) where IsString(attributes["k8s.daemonset.name"])
          - set(attributes["dt.kubernetes.cluster.id"], attributes["k8s.cluster.uid"]) where IsString(attributes["k8s.cluster.uid"])

exporters:
  otlphttp:
    endpoint: ${env:DT_ENDPOINT}
    headers:
      Authorization: "Api-Token ${env:DT_API_TOKEN}"

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [k8sattributes, transform]
      exporters: [otlphttp]
    metrics:
      receivers: [otlp]
      processors: [k8sattributes, transform]
      exporters: [otlphttp]
    logs:
      receivers: [otlp]
      processors: [k8sattributes, transform]
      exporters: [otlphttp]

Kubernetes configuration

Configure the following rbac.yaml file with your Kubernetes instance, to allow the Collector to use the Kubernetes API with the service-account authentication type.

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app: collector
  name: collector
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: collector
  labels:
    app: collector
rules:
  - apiGroups:
      - ''
    resources:
      - 'pods'
      - 'namespaces'
    verbs:
      - 'get'
      - 'watch'
      - 'list'
  - apiGroups:
      - 'apps'
    resources:
      - 'replicasets'
    verbs:
      - 'get'
      - 'list'
      - 'watch'
  - apiGroups:
      - 'extensions'
    resources:
      - 'replicasets'
    verbs:
      - 'get'
      - 'list'
      - 'watch'
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: collector
  labels:
    app: collector
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: collector
subjects:
  - kind: ServiceAccount
    name: collector
    namespace: default

Components

For our configuration, we configured the following components.

Receiver

Under receivers, we specify the standard otlp receiver as an active receiver component for our Collector instance.

This is mainly for demonstration purposes. You can specify any other valid receiver here (for example, zipkin).

Processor

Under processors, we specify the k8sattributes processor with the following parameters:

extract—Specifies which information should be extracted.
pod_association—Specifies how the pod information is linked to attributes.

We also configure the transform processor to have Kubernetes cluster information automatically added as resource attributes.

Exporter

Under exporters, we specify the default otlphttp exporter and configure it with our Dynatrace API URL and the required authentication token.

For this purpose, we set the following two environment variables and reference them in the configuration values for endpoint and Authorization.

DT_ENDPOINT contains the base URL of your ActiveGate (for example, https://{your-environment-id}.live.dynatrace.com/api/v2/otlp)
DT_API_TOKEN contains the API token

Service pipeline

Under service, we assemble our receiver, processor, and exporter objects into pipelines for traces, metrics, and logs. These pipelines allow us to send OpenTelemetry signals via the Collector instance and have them automatically enriched with additional Kubernetes-specific details.

Limits and limitations

Data is ingested using the OpenTelemetry protocol (OTLP) via the Dynatrace OTLP APIs and is subject to the API's limits and restrictions. For more information see: