Distributed traces concepts
In this article, you will learn about the most important concepts and terminology for distributed tracing in Dynatrace.
Distributed trace
A distributed trace is a sequence of spans, identified by a unique identifier—trace ID. Each distributed trace traverses at least one service and, in a modern microservice environment, typically multiple services. It contains semantically different attributes that make it possible to interpret and understand the collected data for a request.
Use cases
- Quickly understand how each microservice is performing.
- Understand how requests propagate across distributed systems and microservices.
- Use high-quality data generated by distributed systems and microservices for request analysis.
- Follow Davis AI root cause analysis drill-downs to identify cause-effect relationships between events.
Best practices
Information collected by distributed traces varies based on the instrumentation. Choose the instrumentation that best fits your needs.
Span
A span is a record of an interaction between a request and a service, triggered by the activity of a request moving through and across multiple services. Spans collect information in the form of attributes, including the name, start and end timestamps, a list of attributes as key/value pairs, the parent's span identifier, and the span kind.
When the activity— parent span—is completed, the next activity passes to its child span. A span without a parent span is called a root span and indicates the start of a trace.
Learn more about span semantic fields.
Span context
The span context is needed to put all spans and events in context with each other. The span context allows a child span to relate to the trace and its parent span. Therefore, the context needs to be propagated within a service (across different threads) but also across services and process boundaries. This typically happens via HTTP headers (like the W3C trace context) or via unique IDs in messaging systems.
Attribute
Attributes are key-value pairs that provide details about a span, request, or resource such as response codes, HTTP methods, and URLs. Via attributes, you can group, query, find, and analyze your traces and spans.
Use cases
Dynatraces uses attribute metadata to
- Detect and name services.
- Gather data on the trace context and relationships with other entities for Smartscape topology.
- Connect log data to traces for Logs or Logs Classic.
- Understand how the duration of a span is affected by service timings (for example, CPU time, network time, or just waiting for other threads) and analyze which code was executed in the context of the span.
Best practices
Depending on your instrumentation, define attribute settings for captured spans and requests.
Learn more about request attributes and captured attributes semantic fields.
Service
Services are traversed by distributed traces. On horizontally scaled services, specific Service Instances process each span. Services are determined and named based on available attributes or properties that are collected along with the spans.
Use cases
Instrumentation
Instrumentation for distributed tracing means adding code to your application to gain visibility on its behavior and interactions with services in a distributed system. The code collects and propagates trace data, including request and response times, a unique identifier for a specific request, service dependencies, and other relevant metadata in the form of attributes. Depending on the instrumentation, the information collected for distributed traces varies.
To get started instrumenting your traces, see
Best practices
Because information collected for distributed traces varies based on the instrumentation, you must choose the instrumentation that best fits your needs.
Use OpenTelemetry in combination with OneAgent to enhance your observability coverage, using the best of both.
Instrumentation
Automatic or manual
Automatic
Capturing
Automatic collection of allowed span attributes.
Automatic collection of several request attributes, including HTTP method, URL, response codes, topology data, and details about the underlying technologies.
Context
Automatically or manually contextualized log entries, depending on the instrumentation library.
Automatically contextualized
- Log entries produced by prominent log frameworks.
- Traces in Smartscape and Davis AI.
PurePath® technology
Dynatrace patented PurePath® technology for distributed tracing, since 2006. PurePath® technology combines distributed tracing information with additional insights like user experience information, logs, metrics, topology information, metadata, and even code-level profiling information to provide the highest level of data fidelity and granularity.
Use cases
Analyze data down to code-level detail without ever losing the full context around your environment during drill down, with the highest level of data granularity and fidelity of monitored transactions.