GenAI spans carry prompts, completions, and token usage. With Dynatrace, you can tag GenAI spans to answer two of the most common questions that come up when sharing one Dynatrace environment with other teams:
Whenever possible, set these tags at the source, for example via OpenTelemetry resource attributes or OneAgent host tags.
If it's not possible to set tags at the source, or when your GenAI spans need different values than other telemetry from the same host or service, you can use OpenPipeline to set tags. This page shows you how to tag your GenAI spans to answer these questions, without changing any AI applications or their instrumentations.
This tutorial is for platform and security administrators who manage OpenPipeline configurations. You don't need to change your AI applications or their instrumentation.
By the end of this tutorial, you'll have an OpenPipeline configuration that processes and tags GenAI spans before they're stored in Grail.
In this tutorial, you'll learn how to:
OpenPipeline processes data at ingest. Therefore, you can't use OpenPipeline to modify spans that are already ingested and stored.
The general steps are as follows:
The dt.security_context tag enables record-level access control in Grail.
Set it on your GenAI spans to configure read access for prompts and completions.
If GenAI spans need a different context than the other telemetry from the same host or service, configure dt.security_context in OpenPipeline.
OpenPipeline routing conditions only select which spans receive that value; it isn't an isolation boundary.
For per-team isolation, set dt.security_context at the source, for example in an OTel Collector you control.
In your pipeline, go to the Permission tab and select Processor > Set security context.
Configure the processor as follows.
| Field | Value |
|---|---|
Name | GenAI security context |
Matching condition |
|
Security context value | Choose Static value and enter a value that you control, for example Don't derive the security context value from a field that the instrumented application sets, such as |
Select Save.
Cost allocation tags let you attribute GenAI costs to specific products or organizational units. To set these tags, use the Cost Allocation and Product Allocation stages in OpenPipeline.
For more information about these stages, see Cost allocation via OpenPipeline.
In your pipeline, go to the Cost Allocation stage and select Processor > DPS Cost Allocation - Cost Center.
Configure the processor as follows.
| Field | Value |
|---|---|
Name | GenAI cost center |
Matching condition |
|
Value | Choose Field value and set the field name to |
Select Save.
Go to the Product allocation stage and select Processor > DPS Cost Allocation - Product.
Configure the processor as follows.
| Field | Value |
|---|---|
Name | GenAI product |
Matching condition |
|
Value | Choose Static value and enter |
Select Save.
A pipeline only processes records that a dynamic route sends to it. For details, see Route data.
GenAI span routingisNotNull(gen_ai.provider.name) or isNotNull(gen_ai.system)New GenAI spans now carry the fields that you just configured. Verify this by running the following DQL, for example in
Notebooks, and analyze the output to make sure you're seeing the expected spans.
fetch spans| filter isNotNull(gen_ai.provider.name) or isNotNull(gen_ai.system)| fields start_time, service.name, dt.security_context, dt.cost.costcenter, dt.cost.product| limit 20
You've configured OpenPipeline to set security context and cost allocation on GenAI spans. Specifically, you learned how to:
Set dt.security_context with the Set security context processor.
Set dt.cost.costcenter and dt.cost.product in the Cost Allocation and Product allocation stages.
Route GenAI spans to the pipeline and verify the result with DQL.