Causal AI, used in the latest Dynatrace root cause analysis (RCA), also covers OpenTelemetry services, frontends that call them, and databases used by them.
When directly connected components degrade because of the same issue, Dynatrace merges their alerts into one problem with the same detected root cause, instead of opening one problem per component.
This page explains how RCA covers OpenTelemetry components in new and existing environments, prerequisites needed to run it, and what you need to do to upgrade the root cause engine.
Causal AI follows the Smartscape topology on Grail data and merges alerts along the topology for components that call each other directly:
Causal AI can merge alerts when their start timestamps are within about 15 minutes of each other. The model ranks the candidates by timing and position in the topology to identify a common root cause.
Causal AI only connects components with existing alerts. If any component in the topology doesn't raise an alert, the chain splits on that component.
Causal AI for OpenTelemetry services, frontends, and databases is available starting with SaaS version 1.349+.
For all environments in Group 3 (Enhanced), the causal AI support for OpenTelemetry is available by default.
For environments in Groups 1 and 2, causal AI works as before, but causal AI support for OpenTelemetry isn't enabled by default and there's no manual setting for it. If your environment is in Group 1 or Group 2 and runs only or mostly on OpenTelemetry, you can request causal AI support for OpenTelemetry. To do so, open a support ticket or contact your Dynatrace account team.
After you switch to the latest version of the root cause engine on environments in Groups 1 or 2,
Problems starts showing only problems detected by the latest version of the root cause engine, while
Problems Classic shows only problems detected by the classic version of the root cause engine. As a result,
Problems and
Problems Classic might show different problems for the same situation after the switch.
Latest Dynatrace uses dt.smartscape.* instead of dt.entity.*, so you need to ensure that each OpenTelemetry component uses Smartscape properties.
To ensure that causal AI can identify and merge alerts raised by your OpenTelemetry components, check if your components fulfill the requirements:
traceparent and tracestate, without changing traceparent or the Dynatrace dtr entry.Causal AI in latest Dynatrace operates based on a set rule list when merging alerts. These rules largely depend on the edge type, which defines causal direction.
| Edge type | Alert type | Merged alert type | Causal direction |
|---|---|---|---|
SERVICE calls SERVICE | SLOWDOWN | SLOWDOWN | Downstream (called SERVICE causes the slowdown of the calling SERVICE) |
SERVICE calls SERVICE | ERROR | ERROR | Downstream (called SERVICE causes the failure of the calling SERVICE) |
FRONTEND calls SERVICE | SLOWDOWN | SLOWDOWN | Downstream (called SERVICE causes the slowdown of the calling FRONTEND) |
FRONTEND calls SERVICE | ERROR | ERROR | Downstream (called SERVICE causes the failure of the calling FRONTEND) |
FRONTEND calls SERVICE | AVAILABILITY | ERROR | Downstream (called SERVICE causes the failure of the calling FRONTEND) |
SERVICE runs_on HOST | *1 | *1 | Downstream (HOST causes the failure of the SERVICE that runs on it) |
SERVICE runs_on PROCESS | *1 | *1 | Downstream (PROCESS causes the failure of the SERVICE that runs on it) |
FRONTEND calls SERVICE | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (calling FRONTEND causes the resource contention for the called SERVICE) |
SERVICE calls SERVICE | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (calling SERVICE causes the resource contention for the called SERVICE) |
SERVICE runs_on HOST | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (SERVICE causes the resource contention for the HOST it runs on) |
SERVICE runs_on PROCESS | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (SERVICE causes the resource contention for the PROCESS it runs on) |
SERVICE runs_on OTEL_HOST | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (SERVICE causes the resource contention for the OTEL_HOST it runs on) |
SERVICE runs_on OTEL_PROCESS | RESOURCE_CONTENTION | RESOURCE_CONTENTION | Upstream (SERVICE causes the resource contention for the OTEL_PROCESS it runs on) |
SERVICE runs_on OTEL_HOST or SERVICE runs_on OTEL_PROCESS | *1 | *1 | Downstream (OTEL_HOST or OTEL_PROCESS causes the failure of the SERVICE that runs on it) |
SERVICE calls DB_INSTANCE_{POSTGRES,MSSQL,MYSQL,ORACLE,HANA,MARIADB,DB2} | ERROR | AVAILABILITY | Downstream (called DB_INSTANCE_* causes the failure of the calling SERVICE) |
SERVICE calls AWS_RDS_DBINSTANCE | SLOWDOWN | RESOURCE_CONTENTION | Downstream (called AWS_RDS_DBINSTANCE causes the resource contention for the calling SERVICE) |
SERVICE calls AWS_RDS_DBCLUSTER | SLOWDOWN | RESOURCE_CONTENTION | Downstream (called AWS_RDS_DBCLUSTER causes the resource contention for the calling SERVICE) |
SERVICE calls AWS_ELASTICACHE_CACHECLUSTER | SLOWDOWN | SLOWDOWN | Downstream (called AWS_ELASTICACHE_CACHECLUSTER causes the slowdown of the calling SERVICE) |
SERVICE calls AZURE_MICROSOFT_SQL_SERVERS_DATABASES | SLOWDOWN | RESOURCE_CONTENTION | Downstream (called AZURE_MICROSOFT_SQL_SERVERS_DATABASES causes the resource contention for the calling SERVICE) |
SERVICE calls AZURE_MICROSOFT_CACHE_REDIS | ERROR | ERROR | Downstream (called AZURE_MICROSOFT_CACHE_REDIS causes the failure of the calling SERVICE) |
SERVICE calls GCP_FIRESTORE_GOOGLEAPIS_COM_DATABASE | ERROR | ERROR | Downstream (called GCP_FIRESTORE_GOOGLEAPIS_COM_DATABASE causes the failure of the calling SERVICE) |
Causal AI correlates service problems with infrastructure events on which it runs. For example, suppose there's a new deployment event on the infrastructure and the service failure rate goes up. Then latest RCA diagnoses the problem (ERROR) and concludes that the service fails because of a new deployment.
CUSTOM_ALERT behaviorA custom alert can trigger the root cause analysis only if it's activated on a service (dt.smartscape_source.id should be the ID of that service) and its event type is ERROR or SLOWDOWN. If the event type is CUSTOM_ALERT, no root cause search happens.
Keep in mind that classic custom alerting events use CUSTOM_ALERT by default. If your alert is set to activate on service failures or slowdowns, we recommend setting the event type to ERROR or SLOWDOWN.
A custom alert on a host or, for example, a Kubernetes deployment doesn't start the root cause search itself and can only be merged with an event on a service.
Causal AI in latest Dynatrace is currently subject to the following limitations:
If your environment is in Group 1 or Group 2 and runs only or mostly on OpenTelemetry, you can request causal AI support for OpenTelemetry. To request it, open a support ticket or contact your Dynatrace account team. To make the switch smoother, prepare your environment first.
Before the latest root cause engine is active in your environment, ensure that workflow filters are replaced by their latest Dynatrace equivalent. To do so, follow the Upgrade from Classic problem notification to simple workflows guide.
We recommend using root_cause.smartscape_entity fields as a part of notification content instead of the trigger filters and using tags or smartscape.affected_entities as trigger filters instead.
For more information, see Upgrade from Classic problem notification to simple workflows.
For further upgrade details, see: