Frequently asked questions about AI Observability and Dynatrace

• Latest Dynatrace
• Overview
• 8-min read
• Updated on Oct 30, 2026

This FAQ page provides answers to your most common questions about how AI Observability works within Dynatrace.

How do I instrument my services and send data?

Use the AI Observability onboarding page to configure OpenInference, the recommended OpenTelemetry-based ingest path, or direct OpenTelemetry instrumentation, then define permissions, sampling strategies, and tokens. It includes scenario‑based guidance (for example, "data not in", "no access") and validation tools to confirm successful ingestion.

Which metrics are available?

Within AI Observability, use the Explorer tab to see AI performance metrics such as volume (LLM request counts), errors and success/failure rates, latency and traffic by model, tokens (input, completion, and total), cost (per model, input and output totals), and guardrails (invocation counts and provider-specific dimensions).

Select any service or model in the Explorer table to open the service detail view, which also shows service-level metrics (open problems, endpoints, vulnerabilities, running processes, pods, and clusters) and debug information (logs, prompts, traces, and additional collected telemetry).

Views are customizable, and you can add DQL-based metrics.

Behind these dashboards, the underlying data comes from three distinct sources. Knowing the source helps you troubleshoot when a metric is missing or empty:

For the full list of standard metrics, their instrument types, and which tile each one feeds, see AI Observability metrics reference.

How do guardrail metrics work? Who provides guardrail runtime protection?

Dynatrace does not execute or enforce guardrail runtime protection. Guardrail enforcement happens at the model/provider level during inference (for example, Amazon Bedrock Guardrails). These then exposes results (such as whether a guardrail intervened) via response payloads and/or provider metrics.

AI Observability instruments these provider signals and surfaces them for monitoring and analysis. This includes pre-filled alert creation flows and real‑time monitoring with customizable views.

You need to configure guardrails with your model provider. Once configured, Dynatrace ingests and displays the resulting guardrail outcomes and metrics so you can observe behavior and trends centrally.

You can build on top of these guardrail metrics in Dynatrace just like with other AI observability signals: create custom alerts and notifications, add tiles to dashboards, and trigger workflows.

How do I evaluate LLM applications and agents?

Use Arize AX to run and analyze evaluations of your LLM applications, agents, prompts, and AI workflows.

Evaluation results already written to Dynatrace remain available as business events for as long as your bucket retention keeps them. To query them, use DQL:

fetch bizevents
| filter event.type == "gen_ai.evaluation.result"

What happens when dt-evals and the Evaluations tab reach end of life?

dt-evals and the Evaluations tab in AI Observability are being retired.

If you still run dt-evals, see Run LLM-as-a-judge evaluations with dt-evals and the dt-evals README. For details, see the dt-evals deprecation notice.

Does Dynatrace support agent frameworks and protocols?

Yes. OpenInference and OpenTelemetry integrations cover:

For examples in GitHub, see Dynatrace AI Agent instrumentation examples.

Can I configure proactive alerts?

Yes. You can create custom alerts directly in AI Observability. These include context with pre‑filled fields, embed notifications (Slack/email), and link back to investigate. You also get a centralized view to review all related alerts that you and your teams have created.

What attributes do I need to add to my GenAI spans?

For dashboards to render data properly, add all GenAI Required, Conditionally required, and Recommended attributes to your spans.

For more information, see Semantic Conventions for GenAI agent and framework spans

What if I am already using ready-made dashboards?

You can continue using your existing ready‑made dashboards, which remain available until the AI Observability is generally available. The dashboards are tagged for discoverability and include navigation that redirects into AI Observability for deeper, contextual analysis. For a more integrated and centralized workflow (instrumentation guidance, insights, proactive alerts, and in‑context prompt/log/trace debugging), we recommend using AI Observability as your primary entry point, because it provides a dedicated end‑to‑end experience purpose‑built for AI workloads. Dashboards alone can be limited or inconsistent for GenAI‑specific workflows.

Does AI observability generate additional query cost?

Some out-of-the-box AI Observability dashboards use span queries, which consume Traces powered by Grail - Query. This is true even if AI Observability isn’t fully configured yet, or the dashboards show no data.

To control your trace consumption, you can:

Note that we're currently working on reducing costs for both AI Observability and Dashboards, by moving away from span queries.