Retain trace data for long periods

Long-term data retention provides comprehensive trend analysis and performance monitoring over extended periods. For industries with regulatory requirements for auditing and compliance purposes, storing trace data for several years can be a requirement. Additionally, it supports teams in identifing recurring issues, troubleshooting processes that take longer than expected, and investigating and resolving issues that may not be immediately apparent, especially those that develop over time.

With distributed tracing powered by Grail, you can store trace data in Grail buckets with custom retention time, from 10 days up to 10 years, shaping trace data retention and storage according to team ownership or time requirements.

Who is this for

This article is intended for SRE and architects who want to store trace data to compare the behavior of services over long periods of time for troubleshooting or compliance purposes.

What you will learn

In this article you will learn how to create a custom bucket with a 5-year data retention period and assign production trace data of a Kubernetes namespace to it.

Before you begin

  • Dynatrace SaaS environment powered by Grail and AppEngine.
  • Dynatrace Platform Subscription (DPS) with Traces powered by Grail capabilities.
  • openpipeline:configurations:write and openpipeline:configurations:read permissions. To learn how to set up the permissions, see Permissions in Grail.
Pipeline

Collection of processing instructions to structure, separate, and store data.

Stage

Phase in a pipeline sequence, focused on a task and defined by processors.

Processor

Pre-formatted processing instruction.

Routing

Assignation of data to a pipeline, based either on matching conditions (dynamic routing) or directly configured (static).

Steps

To create a custom bucket with a long retention period

  1. Go to Settings Settings > Storage management > Bucket storage management > Bucket.
  2. Define the new bucket
    • Name: long_term_spans
    • Retention period (days): 3650
    • Record type: spans
  3. Select Create and then refresh the list ( ).

You created a new bucket to store trace data for 5 years. The bucket will remain empty until you assign records to it via OpenPipeline app OpenPipeline.

To assign records from a production namespace to the long_term_spans bucket via OpenPipeline app OpenPipeline

  1. Go to OpenPipeline app OpenPipeline > Spans > Pipelines.
  2. Choose an existing pipeline or create a new one.
  3. In the Storage stage, select Processor > Bucket assignement.
  4. Define the new processor
    • Name: Store prod spans (5 years)
    • Condition: k8s.namespace.name==prod
    • Bucket: long_term_spans
  5. Select Save.

If an existing route points to the pipeline, the new processor will start assigning trace data from the production namespace to the long_term_spans bucket. If you created a new pipeline for the processor, make sure to route trace data to it.

Make sure your pipeline is receiving records via a dynamic route.

  1. In the OpenPipeline app OpenPipeline go to Dynamic routing.
  2. Choose an existing dynamic route or create a new one.
  3. Define the route by entering a route name, a matching condition (for example true), and the target pipeline name.
  4. Select Save.

Conclusion

You created a new bucket with a custom retention period of 5 years. The new bucket is the storage option for production-related spans. Spans are routed to the pipeline and assigned to the bucket according to matching condition.