Batch OTLP requests
The following configuration example shows how you configure a Collector instance and its native batch processor to queue and batch OTLP requests and improve throughput performance.
1receivers:2 otlp:3 protocols:4 grpc:5 http:67processors:8 batch:9 send_batch_max_size: 100010 timeout: 30s11 send_batch_size : 8001213exporters:14 otlphttp:15 endpoint: $DT_ENDPOINT/api/v2/otlp16 headers:17 Authorization: "Api-Token $DT_API_TOKEN"1819service:20 pipelines:21 traces:22 receivers: [otlp]23 processors: [batch]24 exporters: [otlphttp]25 metrics:26 receivers: [otlp]27 processors: [batch]28 exporters: [otlphttp]29 logs:30 receivers: [otlp]31 processors: [batch]32 exporters: [otlphttp]
For our configuration, we configure the following components.
receivers, we specify the standard
otlp receiver as active receiver component for our Collector instance.
This is for demonstration purposes. You can specify any other valid receiver here (for example,
processors, we specify the
batch processor with the following parameters:
send_batch_max_sizeconfigured for a maximum of 1,000 entries per batch
timeoutconfigured to always send data after 30 seconds, regardless of any other batch limits
send_batch_sizeconfigured to always send data after 800 entries, regardless of any other batch limits
With this configuration, the Collector queues telemetry entries in batches and sends a batch either after 30 seconds have passed or at least 800 entries are queued.
exporters, we specify the default
otlphttp exporter and configure it with our Dynatrace API URL and the required authentication token.
For this purpose, we set the following two environment variables and reference them in the configuration values for
service, we assemble our receiver and exporter objects into pipelines for traces, metrics, and logs and enable our batch processor by referencing it under
processors for each respective pipeline.