Scaling & Resiliency

Depending on your use case, you can deploy the Dynatrace Collector with different scaling approaches in mind.

Commonly used deployment architectures

There are a few commonly used ways OpenTelemetry data makes its way into Dynatrace through Collectors. The common scenarios are explained here.

Simple deployment

The simplest approach to get OTel data into Dynatrace is to leave out the Collector completely and send data directly from an in-app OTel SDK over OTLP.

Another option is to have a standalone gateway Collector in between that acts as a central ingest point for OpenTelemetry data and forwards it to Dynatrace over OTLP while also adding the possibility to add data processing in the Collector.

Large-scale deployment

Larger scale Collector deployments have different Collector deployment modes combined to achieve more scalability and flexibility for teams.

For example, there could be a Collector DaemonSet per Kubernetes cluster that does some initial processing specific to the workloads on that cluster, like sensitive data filtering. The in-cluster Collectors then forward to a set of gateway Collectors (for example, managed by a platform team) for general data processing (such as sampling and batching) and ingest into Dynatrace.

A setup like this ensures that development teams have control over their domain-specific Collector setups while also allowing a different team to manage general OpenTelemetry data processing and Dynatrace token management at a central point.

Scaling the Collector

Collector scaling options depend on your specific use case and scenarios. This section mostly applies to the gateway deployment mode described above.

Having gateway collectors in front of Dynatrace makes sense if you want to preprocess or otherwise transform your OpenTelemetry data before finally sending it to Dynatrace. Preprocessing your data could involve, for example, transforming and filtering or sampling the data.

Send separate signals to different Collectors

If your use case has higher load on specific signals (for example, far more logs than traces), you could split out signals with higher load into separate Collector instances and scale those on their own.

This has the benefit that you can fine-tune your resource consumption for your specific amount of telemetry data per signal, which results in high flexibility.

Scaling stateless Collectors

Most Collector components are stateless, which makes the Collector an ideal fit for both vertical and horizontal scaling.

Scaling stateful Collectors

If you have components in your Collector that have some form of state, vertical scaling is a good option. Horizontal scaling can still be used with special handling of the state (for example, special load balancing).