Try it free

Ingest FluentD data with the OTel Collector

  • How-to guide
  • 1-min read
  • Published Jan 26, 2024

The following configuration example shows how to configure a Collector instance to accept FluentD events via the Fluent Forward protocol and ingest them as OTLP requests into Dynatrace.

Prerequisites

  • One of the following Collector distributions with the Fluent Forward receiver:
    • The Dynatrace OTel Collector
    • OpenTelemetry Contrib
    • A custom Builder version
  • The Dynatrace API endpoint URL to which the data should be exported
  • An API token with the relevant access scope (only required for SaaS and ActiveGate)

See Collector Deployment and Collector Configuration on how to set up your Collector with the configuration below.

Demo configuration

receivers:
fluentforward:
endpoint: 0.0.0.0:8006
exporters:
otlp_http:
endpoint: ${env:DT_ENDPOINT}
headers:
Authorization: "Api-Token ${env:DT_API_TOKEN}"
service:
pipelines:
logs:
receivers: [fluentforward]
processors: []
exporters: [otlp_http]
Configuration validation

Validate your settings to avoid any configuration issues.

Components

For our configuration, we configure the following components.

Receivers

Under receivers, we specify the fluentforward receiver as the active receiver component for our Collector instance and configure it to listen on specified ports.

Exporters

Under exporters, we specify the default otlp_http exporter and configure it with our Dynatrace API URL and the required authentication token.

For this purpose, we set the following two environment variables and reference them in the configuration values for endpoint and Authorization.

  • DT_ENDPOINT contains the base URL of the Dynatrace API endpoint (for example, https://{your-environment-id}.live.dynatrace.com/api/v2/otlp)
  • DT_API_TOKEN contains the API token

Service pipelines

Under service, we assemble our receiver and exporter objects into a logs pipeline, which will listen on the configured address for FluentD logs and ingest the data into Dynatrace.

Limits and limitations

Logs are ingested using the OpenTelemetry protocol (OTLP) via the Dynatrace OTLP API and are subject to the API's limits and restrictions. For more information, see Ingest OpenTelemetry logs.

Related topics

  • Enrich ingested data with Dynatrace-specific fields
  • Enrich OTLP requests with Kubernetes data
  • Ingest logs from files with the OTel Collector
  • Ingest syslog data with the OTel Collector
Related tags
Application Observability