Query monitored entities in Grail
This is a guide to effectively using Dynatrace Query Language (DQL) to query monitored entities. This involves using generic DQL features such as expanding arrays, joining data using the lookup command, and using the built-in parse functionality.
Query entity types
To query monitored entities by their respective type, you can use the dt.entity.* views.
To initiate the query, start with a blank query in your notebooks and enter fetch dt.entity. This triggers an auto-complete dialog where you can select the desired entity type.
For example, executing the fetch dt.entity.host query retrieves all host entities. By default, entity records include the ID and entity name.
fetch dt.entity.host
HOST-1
HOST-123
HOST-2
HOST-456
HOST-3
HOST-789
HOST-4
HOST-101
HOST-5
HOST-111
HOST-6
HOST-131
HOST-7
HOST-141
To include additional details, you can use the fieldsAdd command. As you start typing, the auto-complete feature suggests available fields for the entity type.
Another way to get started is to use one of the built-in topology query snippets.
Query relationships
Relationships are exposed as fields and can be added similarly to other fields. For instance, you can add the runs relationship to your host records to obtain a list of all entities running on the host.
fetch dt.entity.host| fieldsAdd runs
HOST-1
HOST-123
runs: Complex record
HOST-2
HOST-456
runs: Complex record
HOST-3
HOST-789
runs: Complex record
The runs field is a nested record that contains a field for each entity type running on a specific host. Depending on the cardinality, these fields are either strings representing a single entity ID or arrays of strings representing a list of entity IDs. In notebooks, select a nested record in the results list to see its contents.
If you only want to see process groups running on that host, you can specify this in the fieldsAdd command. The auto-complete feature will provide a list of possible identifier types.
Selecting a type refines the query to only return an array containing the process group IDs.
fetch dt.entity.host| fieldsAdd runs[dt.entity.process_group]
HOST-1
HOST-123
runs: PROCESS_GROUP-D123, PROCESS_GROUP-567, PROCESS_GROUP-012
HOST-2
HOST-456
runs: PROCESS_GROUP-D234, PROCESS_GROUP-234, PROCESS_GROUP-F10
HOST-3
HOST-789
runs: PROCESS_GROUP-D567, PROCESS_GROUP-789, PROCESS_GROUP-123
The field names for relationships differ from the original relationship names in the previous Dynatrace. Instead of using a single name prefixed with fromRelationship and toRelationship, the fields have different names on both sides.
See the relationship mapping table below to understand how second-generation relationships are represented as DQL fields.
Note that 1:n relationships only return 100 entity IDs per type, per record. In such cases, we recommend using classicEntitySelector() instead.
Entity lookups
You can use the lookup command to join data from related entities.
For example, to include the host tags with your service instances, you can access the host from a service instance by following the runs_on relationship.
fetch dt.entity.service_instance| fieldsAdd runs_on[dt.entity.host]
Mapped Instance for answer_queue on ActiveMQ Artemis
SERVICE_INSTANCE-1AB2
HOST-1
Mapped Instance for Requests executed in the background threads of eT-demo-1-BussinessBackend
SERVICE_INSTANCE-A123
HOST-2
It's important to note that service instances always run on a single host, which means that you obtain a single host ID per service instance record. This allows you to use the lookup command to add the hostname to your records. The hostname is added as the lookup.entity.name field.
fetch dt.entity.service_instance| fieldsAdd runs_on[dt.entity.host]| lookup sourceField:`runs_on[dt.entity.host]`, lookupField:id, [ fetch dt.entity.host ]
Mapped Instance for answer_queue on ActiveMQ Artemis
SERVICE_INSTANCE-2AB1
HOST-1
AB1-abc
HOST-1
Mapped Instance for Requests executed in the background threads of eT-demo-1-BussinessBackend
SERVICE_INSTANCE-B123
HOST-2
BA1-cba
HOST-2
Mapped Instance for :80
SERVICE_INSTANCE-C321
HOST-3
BA1-cba
HOST-2
Expand relationships
Hosts can run multiple service instances, so the runs[dt.entity.service_instance] field is an array of entity IDs.
fetch dt.entity.host| fieldsAdd runs[dt.entity.service_instance]
dw123
HOST-1
SERVICE_INSTANCE-AB123, SERVICE_INSTANCE-CB123, SERVICE_INSTANCE-DB123
abc/123
HOST-2
SERVICE_INSTANCE-AB902
The lookup command doesn't apply to arrays of IDs, so you need to use the expand command first to retrieve individual records per service instance ID.
fetch dt.entity.host| fieldsAdd runs[dt.entity.service_instance]| expand runs[dt.entity.service_instance]
dw123
HOST-1
SERVICE_INSTANCE-DB123
dw456
HOST-2
SERVICE_INSTANCE-BA987
dw789
HOST-3
SERVICE_INSTANCE-CA687
dw652
HOST-4
SERVICE_INSTANCE-1AB2
In this example, the first record expands into four. Now you can use the lookup command to get the service instance details that you include in the lookup field.
fetch dt.entity.host| fieldsAdd runs[dt.entity.service_instance]| expand runs[dt.entity.service_instance]| lookup sourceField:`runs[dt.entity.service_instance]`, lookupField:id, [ fetch dt.entity.service_instance]
dw321
HOST-1
SERVICE_INSTANCE-DB123
Mapped Instance for easytravelazure-weather-service
SERVICE_INSTANCE-DB123
dw024
HOST-2
SERVICE_INSTANCE-2E986
Mapped Instance for weather-service-restify
SERVICE_INSTANCE-2E986
dw056
HOST-3
SERVICE_INSTANCE-D13A2
Mapped Instance for easytravelazure-weather-express
SERVICE_INSTANCE-D13A2
dw178
HOST-4
SERVICE_INSTANCE-D03A2
Mapped Instance for easytravelazure-weather-express
SERVICE_INSTANCE-D03A2
Entity tags
Entity tags consist of up to three values: context, key, and value. Dynatrace creates a string representation of these values in the following format:
[<context>]<key>:<value>
-
All occurrences of the
[,], and:characters need to be escaped using the\character. -
The
tagsfield returns the string representations of these fields.
The following query example retrieves a list of host entity tags.
fetch dt.entity.host| expand tags, alias:tag| fields tag
AppSec:Node.js
[Azure]tenant:CustomerA
HostName:dw123
AppSec:.NET
[AWS]created_at:2023-07-07T12:20:10Z
[Environment]tema:cpn
You can use the expand command to optimize tag filtering. This example filters hosts based on a specific cluster name.
fetch dt.entity.host| expand tags| filter contains(tags, "[Environment]Cluster.Name:prod-eu-west-6-ireland")
HOST-1
HOST-C2
tags:[Environment]Cluster.Name:prod-eu-west-6-ireland
HOST-2
HOST-C3
tags:[Environment]Cluster.Name:prod-eu-west-6-ireland
HOST-3
HOST-C4
tags:[Environment]Cluster.Name:prod-eu-west-6-ireland
HOST-4
HOST-C5
tags:[Environment]Cluster.Name:prod-eu-west-6-ireland
HOST-5
HOST-C6
tags:[Environment]Cluster.Name:prod-eu-west-6-ireland
If you need structured access to the key, context, or value, you can use the following DPL parse expression to split the string representation into individual fields.
fetch dt.entity.host| expand tags, alias:tag_string| parse tag_string, """(('['LD:tag_context ']' LD:tag_key (!<<'\\' ':') LD:tag_value)|(LD:tag_key (!<<'\\' ':') LD:tag_value)|LD:tag_key)"""
HOST-1
HOST-73
Maxk:WebService2-ABC
undefined
Maxk:WebService2-ABC
undefined
HOST-1
HOST-73
testtests:testspreiser
undefined
testtests:testspreiser
undefined
HOST-1
HOST-73
Maxk:WebService3-ABC
undefined
Maxk:WebService3-ABC
undefined
List fields and relationships
Use the describe command to obtain a list of fields and relationships for each entity view.
For example, to retrieve a list of all fields and relationships for the service_instance entity view, enter describe dt.entity.service_instance:
describe dt.entity.service_instance
Take this information into account when working with different fields:
-
Most entity fields have the same names as in the API v2 environment (for example, gcpZone and oneAgentCustomName).
-
The first and last observation timestamp of an entity is stored in the lifetime field, represented as a timeframe type comprising a start and end timestamp. The lifetime of an entity needs to overlap with the query timeframe for the entity to be included in the query.
-
Several entity names are prefixed with 'entity.' (for example,
entity.conditional_name) -
Relationships are returned as records, to learn more about them, see entity relationships.
The describe command is a valuable tool to explore the Grail data schema.
describe dt.entity.service_instance| filter in(data_types, "record")
belongs_to
RELATIONSHIP
runs_on
RELATIONSHIP
sends_to
RELATIONSHIP
receives_from
RELATIONSHIP
instance_of
RELATIONSHIP
Permissions
You need the storage:entities:read permission to query entities.
Grail doesn't apply management zone filters. Users having the storage:entities:read permission can query all entities.
Entity selectors
You can use the classicEntitySelector() function to simplify starting DQL entity queries. This command takes an entity selector as a string argument and provides a list of entity IDs in return. You can use this list to filter entities based on ID.
For example, you can filter service instances running on hosts with a specific tag.
fetch dt.entity.service| filter in(id, classicEntitySelector("type(service), fromRelationship.runsOnHost(type(host), tag([AWS]Category:ABC))"))
123
123
123
123
123
123
123
123
123
123
You can also obtain this result using native DQL with the following query.
fetch dt.entity.service| fieldsAdd host.id = runs_on[dt.entity.host]| expand host.id| lookup sourceField:host.id, lookupField: id, fields:host.tags=tags, [ fetch dt.entity.host]| expand host.tags| filter host.tags == "[AWS]Category:ABC"
123
123
123
123
123
123
123
123
123
123
123
123
123
123
123
123
This query has limitations, such as returning only 100 hosts per service entity, and is generally more complex than the previous example using the classicEntitySelector function.
Relationship mapping table
Entity relationships in the previous Dynatrace (for example, the environment API v2) are mapped to the new names in DQL records according to the following table.
belongsTo
belongs_to
contains
calls
calls
called_by
candidateTalksWith
called_by
calls
hostsComputeNode
hosts
hosted_by
indirectlySendsToQueue
indirectly_sends_to
indirectly_receives_from
isAccessibleBy
accessible_by
can_access
isApplicationMethodOf
belongs_to
contains
isApplicationMethodOfGroup
belongs_to
contains
isApplicationOfSyntheticTest
monitored_by
monitors
isAzrAppServicePlanOf
contains
belongs_to
isAzrEventHubNamespaceOfEventHub
contains
belongs_to
isAzrMgmtGroupOfAzrTenant
belongs_to
contains
isAzrServiceBusNamespaceOfQueue
contains
belongs_to
isAzrServiceBusNamespaceOfTopic
contains
belongs_to
isAzrSQLDatabaseOfElasticPool
belongs_to
contains
isAzrSqlServerOfDatabase
contains
belongs_to
isAzrSqlServerOfElasticPool
belongs_to
contains
isAzrStorageAccountOfAzrEventHub
contains
belongs_to
isAzrSubscriptionOfAzrMgmtGroup
belongs_to
contains
isAzrSubscriptionOfAzrTenant
belongs_to
contains
isAzrSubscriptionOfCredentials
contains
belongs_to
isBalancedBy
balanced_by
balances
isBoshDeploymentOfHost
contains
belongs_to
isCfFoundationOfHost
contains
belongs_to
isCgiOfCa
belongs_to
contains
isCgiOfCai
belongs_to
contains
isCgiOfCluster
belongs_to
contains
isCgiOfHost
belongs_to
contains
isCgiOfNamespace
belongs_to
contains
isChildOf
child_of
parent_of
isClusterOfCa
cluster_of
clustered_by
isClusterOfCai
cluster_of
clustered_by
isClusterOfCni
cluster_of
clustered_by
isClusterOfHost
cluster_of
clustered_by
isClusterOfKubernetesSvc
cluster_of
clustered_by
isClusterOfNamespace
cluster_of
clustered_by
isClusterOfNode
cluster_of
isClusterOfPg
clustered_by
cluster_of
clustered_by
isCnpOfCai
belongs_to
contains
isDatastoreOf
belongs_to
contains
isDeviceApplicationMethodOf
belongs_to
contains
isDeviceApplicationMethodOfGroup
belongs_to
contains
isDiskOf
belongs_to
contains
isDockerContainerOf
contains
belongs_to
isDockerContainerOfPg
contains
belongs_to
isEbsVolumeOf
belongs_to
contains
isGroupOf
group_of
groups
isHostGroupOf
group_of
groups
isHostOfContainer
hosts
hosted_by
isInstanceOf
instance_of
instantiates
isKubernetesSvcOfCa
balances
balanced_by
isKubernetesSvcOfCai
balances
balanced_by
isLocatedIn
belongs_to
contains
isMainPgiOfCgi
belongs_to
contains
isMemberOf
belongs_to
contains
isMemberOfScalingGroup
belongs_to
contains
isNamespaceOfCa
contains
belongs_to
isNamespaceOfCai
contains
belongs_to
isNamespaceOfCni
contains
belongs_to
isNamespaceOfCnp
contains
belongs_to
isNamespaceOfKubernetesSvc
contains
belongs_to
isNamespaceOfPg
contains
belongs_to
isNamespaceOfService
contains
belongs_to
isNetworkClientOf
calls
called_by
isNetworkClientOfHost
calls
called_by
isNetworkClientOfProcessGroup
calls
called_by
isNetworkInterfaceOf
belongs_to
contains
isNodeOfHost
belongs_to
contains
isOpenstackAvZoneOf
belongs_to
contains
isPartOf
belongs_to
contains
isPgAppOf
belongs_to
contains
isPgiOfCgi
belongs_to
contains
isPgOfCa
belongs_to
contains
isPgOfCai
belongs_to
contains
isPgOfCg
belongs_to
contains
isProcessOf
belongs_to
contains
isProcessRunningOpenstackVm
belongs_to
contains
isRuntimeComponentOf
belongs_to
contains
isSameAs
same_as
same_as
isServedByDcrumService
served_by
serves
isServiceMethodOf
belongs_to
contains
isServiceMethodOfService
belongs_to
contains
isServiceOf
belongs_to
contains
isServiceOfProcessGroup
belongs_to
contains
isSiteOf
contains
belongs_to
isSoftwareComponentOfPgi
belongs_to
contains
isStepOf
belongs_to
contains
isUserActionOf
belongs_to
contains
listensOnQueue
belongs_to
contains
manages
manages
managed_by
monitors
monitors
monitored_by
propagatesTo
propagates_to
propagated_from
receivesFromQueue
receives_from
sends_to
runsOn
runs_on
runs
runsOnHost
runs_on
runs
runsOnProcessGroupInstance
runs_on
runs
runsOnResource
runs_on
runs
sendsToQueue
sends_to
receives_from
talksWithCandidate
calls
called_by
affects
affects
affected_by
isRelatedTo
related_to
related_to
Troubleshooting
- The DQL query returns different or fewer entities than API v2 environment
Verify that you are using the same query timeframe
fetch dt.entity.*. TheclassicEntitySelector()function only returns entities that have a lifetime that overlaps with the query timeframe. By default, DQL queries are executed for the last 2 hours, whereas the default timeframe in the API environment is 72 hours.