In addition to the application-side configuration, several Dynatrace-specific settings let you control how OpenTelemetry data is used in Dynatrace.
These can be configured at Settings > Collect and capture > General monitoring settings > OneAgent features.
To learn how to send OpenTelemetry data to a Dynatrace OneAgent, see Use OneAgent with OpenTelemetry data.
Monitoring framework | Versions |
---|---|
OpenTelemetry | 1.0 - 1.31, 1.4 - 1.541 |
OpenTracing | 0.33, 0.32, 0.31 |
To enable OpenTelemetry Java
Existing tracers are replaced and will no longer work after you enable OpenTelemetry Java.
The OneAgent code module's OpenTelemetry Span Sensor automatically captures all OpenTelemetry attributes. If you want to prevent the accidental storage of personal data, you can exclude specific attribute keys for which the values must not be persisted. By omitting attributes containing personal data, you can meet your organization's privacy requirements and control the scope of stored monitoring data.
To configure attribute storage and masking settings for your environment
Only one setting preference is possible.
Allowed attributes list Dynatrace recommends a few basic attributes to generally be included, such as service.name
or service.version
. For ease of use, Dynatrace comes with a default configuration that can be adjusted.
You can then find the attribute key on the Distributed traces page on the Summary tab.
Searching by resource attribute is limited to the service name: filter by Service name
on the Distributed traces page.
Searching by span attribute is limited to the span name: filter by Request
on the Distributed traces page.
For more information about the OneAgent code module's OpenTelemetry Span Sensor, see Detect OpenTelemetry spans using the OneAgent code module's OpenTelemetry Span Sensor.
To avoid possible conflicts with existing PurePath distributed traces, OneAgent ingests by default only spans with a span kind of Server
or Consumer
. This usually is not an issue, as instrumentation libraries typically configure the appropriate span kind, however something to take into account if your application fully uses manual instrumentation.
This behavior can be customized with an entry point rule. To do that, in Dynatrace go to Settings > Collect and capture > OpenTelemetry > Span entry points and create a new rule with the appropriate action and matcher entry.
Depending on your setup, you may experience "flat span enrichment". This is when spans are displayed in Dynatrace as a flat list instead of a tree hierarchy. While this generally is the default behavior with OneAgent ingestion of OpenTelemetry traces, the hierarchy may still reflect actual span relations as defined by the instrumentation, depending on the involved OneAgent code modules and their support for the instrumented technologies.
When merging OpenTelemetry spans into OneAgent sensor traces, make sure that OpenTelemetry spans are leaf spans and not in-between OneAgent spans.
As OneAgent ingests spans already upon their creation, not all eventual attributes may be already present at the initial ingest. Any attributes added at a later point are highlighted in Dynatrace with an initial value not set
note and cannot be used for span capture rules, as they were not yet available when the rules were being evaluated.
When ingesting OpenTelemetry traces automatically with the OneAgent Span Sensor, there is a difference between the context propagation of OpenTelemetry traces and OneAgent traces.
While propagation of OpenTelemetry traces may be already handled properly by your application, it is also important to consolidate them with the OneAgent-specific trace. This can be achieved with a context propagation rule. To configure this, in Dynatrace go to Settings > Collect and capture > OpenTelemetry > Span context propagation and create a context propagation rule with a
Propagate
action and a matcher entry for the span in question (for example, based on the span name or instrumentation library).
Try to avoid trace consolidation for technologies already covered natively by OneAgent sensors. Merging such OpenTelemetry spans into a OneAgent trace may lead to undefined states.
While OpenTelemetry traces are always exported to other backends as is, a small data adjustment takes place when your OneAgent-instrumented application starts a fresh OpenTelemetry trace. This applies only to new traces and not when a trace is continued via context propagation.
In that case, OneAgent may have already created a new trace object when OpenTelemetry was initialized. If these two traces (with separate IDs) are not reconciled, telemetry data might be duplicated or fragmented. To mitigate this and still keep Dynatrace PurePath traces consistent, OneAgent uses the following approach:
To enable correlation between these two IDs, Dynatrace creates additional span links for each span, linking to the OpenTelemetry trace.
The ID rewrite applies only to newly started traces (not context propagation) and to the OpenTelemetry SDKs for Go, Java, JavaScript, PHP, and Python but not to .NET.
TracerProvider
, Propagator
, and ContextManager
.
Therefore, with OpenTelemetry Java enabled, traces are no longer seen by this SDK or exported to backends like Jaeger.TracerProvider
, Propagator
, and ContextManager
.
Therefore, with OpenTelemetry Node.js enabled, traces are no longer seen by this SDK or exported to backends like Jaeger.OTEL_SERVICE_NAME
and OTEL_RESOURCE_ATTRIBUTES
environment variables. When using the OpenTelemetry trace ingest API, this limitation doesn't apply.OneAgent version 1.259+
To avoid possible span duplicates for areas covered by OpenTelemetry and OneAgent, OneAgent skips spans from the following automatic instrumentation Java libraries if OneAgent is configured to instrument your Java application and ingest OpenTelemetry spans.
Such spans are skipped only by OneAgent. Exports to third parties (for example, other backends or the Collector) remain unaffected.
io.opentelemetry.akka-http-10.0 | io.opentelemetry.apache-dbcp-2.0 | io.opentelemetry.apache-httpasyncclient-4.1 |
io.opentelemetry.apache-httpclient-2.0 | io.opentelemetry.apache-httpclient-4.0 | io.opentelemetry.apache-httpclient-4.3 |
io.opentelemetry.apache-httpclient-5.0 | io.opentelemetry.async-http-client-1.9 | io.opentelemetry.async-http-client-2.0 |
io.opentelemetry.c3p0-0.9 | io.opentelemetry.cassandra-3.0 | io.opentelemetry.cassandra-4.0 |
io.opentelemetry.cassandra-4.4 | io.opentelemetry.cxf-jaxrs-3.2 | io.opentelemetry.google-http-client-1.19 |
io.opentelemetry.grpc-1.6 | io.opentelemetry.http-url-connection | io.opentelemetry.java-http-client |
io.opentelemetry.jaxrs-1.0 | io.opentelemetry.jaxrs-1.0-common | io.opentelemetry.jaxrs-2.0-annotations |
io.opentelemetry.jaxrs-2.0-common | io.opentelemetry.jaxrs-2.0-cxf-3.2 | io.opentelemetry.jaxrs-2.0-jersey-2.0 |
io.opentelemetry.jaxrs-2.0-resteasy-3.0 | io.opentelemetry.jaxrs-2.0-resteasy-3.1 | io.opentelemetry.jaxrs-3.0-annotations |
io.opentelemetry.jaxrs-3.0-jersey-3.0 | io.opentelemetry.jaxrs-3.0-resteasy-6.0 | io.opentelemetry.jaxrs-annotations-2.0 |
io.opentelemetry.jaxrs-annotations-3.0 | io.opentelemetry.jaxrs-client-1.1 | io.opentelemetry.jaxrs-client-2.0 |
io.opentelemetry.jaxrs-client-2.0-resteasy-3.0 | io.opentelemetry.jaxws-2.0 | io.opentelemetry.jaxws-2.0-axis2-1.6 |
io.opentelemetry.jaxws-2.0-cxf-3.0 | io.opentelemetry.jaxws-2.0-metro-2.2 | io.opentelemetry.jaxws-cxf-3.0 |
io.opentelemetry.jaxws-common | io.opentelemetry.jaxws-jws-api-1.1 | io.opentelemetry.jdbc |
io.opentelemetry.jedis-1.4 | io.opentelemetry.jedis-3.0 | io.opentelemetry.jedis-4.0 |
io.opentelemetry.jersey-2.0 | io.opentelemetry.jetty-11.0 | io.opentelemetry.jetty-8.0 |
io.opentelemetry.jetty-httpclient-9.2 | io.opentelemetry.jms-1.1 | io.opentelemetry.jms-3.0 |
io.opentelemetry.jsp-2.3 | io.opentelemetry.kafka-clients | io.opentelemetry.kafka-clients-0.11 |
io.opentelemetry.kafka-clients-2.6 | io.opentelemetry.kafka-streams-0.11 | io.opentelemetry.lettuce-5.1 |
io.opentelemetry.liberty | io.opentelemetry.liberty-20.0 | io.opentelemetry.mongo-3.1 |
io.opentelemetry.netty-3.8 | io.opentelemetry.netty-4.0 | io.opentelemetry.netty-4.1 |
io.opentelemetry.okhttp-2.2 | io.opentelemetry.okhttp-3.0 | io.opentelemetry.orcale-ucp-11.2 (sic!) |
io.opentelemetry.rabbitmq-2.7 | io.opentelemetry.reactor-kafka-1.0 | io.opentelemetry.reactor-netty-1.0 |
io.opentelemetry.resteasy-3.0 | io.opentelemetry.resteasy-3.1 | io.opentelemetry.resteasy-6.0 |
io.opentelemetry.rmi | io.opentelemetry.servlet-2.2 | io.opentelemetry.servlet-3.0 |
io.opentelemetry.servlet-5.0 | io.opentelemetry.servlet-javax-common | io.opentelemetry.spring-jms-2.0 |
io.opentelemetry.spring-jms-6.0 | io.opentelemetry.spring-kafka-2.7 | io.opentelemetry.spring-rabbit-1.0 |
io.opentelemetry.spring-rmi-4.0 | io.opentelemetry.spring-webflux-5.0 | io.opentelemetry.spring-webflux-5.3 |
io.opentelemetry.spring-ws-2.0 | io.opentelemetry.tomcat-10.0 | io.opentelemetry.tomcat-7.0 |
io.opentelemetry.tomcat-jdbc | io.opentelemetry.undertow-1.4 | io.opentelemetry.vibur-dbcp-11.0 |
OneAgent version 1.313+
To avoid possible span duplicates for areas covered by OpenTelemetry and OneAgent, OneAgent skips spans from the following automatic instrumentation PHP libraries if OneAgent is configured to instrument your PHP application and ingest OpenTelemetry spans.
Such spans are skipped only by OneAgent. Exports to third parties (for example, other backends or the Collector) remain unaffected.
io.opentelemetry.contrib.php.curl | io.opentelemetry.contrib.php.laravel |
io.opentelemetry.contrib.php.mongodb | io.opentelemetry.contrib.php.mysqli |
io.opentelemetry.contrib.php.pdo | io.opentelemetry.contrib.php.slim |
io.opentelemetry.contrib.php.symfony | io.opentelemetry.contrib.php.wordpress |