Event triggers for workflows

Latest Dynatrace
Reference
6-min read

Event triggers are a group of triggers that you can configure to match events in the ingest pipeline to trigger a workflow.

There are three event trigger types

Event trigger
Problem trigger
Davis event trigger

Event trigger

The Event trigger allows you to specify a custom event filter with the help of a DQL matcher expression to define which events will trigger a workflow execution.

The following configuration parameters are available to define the triggering behavior of the Event trigger.

Event type
- events are created by Dynatrace and your monitoring and platform configuration.
- bizevents are business events. An external application sends the bizevents. For more information, see the /bizevents/ingest endpoint.
- dt.system.events are generated by Dynatrace system services. Not all system events are eligible to trigger workflows. To find the events that can be used, look for those with the dt.openpipeline.pipelines property set using filter isNotNull(dt.openpipeline.pipelines) in a notebook.
- security.events are generated by Dynatrace Application Security, ingested via the v1/security.events endpoint or ingested via custom/security.events/<custom-endpoint-name> endpoints. For more information, see security events in the Semantic Dictionary.
In Filter query, you can define how to narrow down the events. The filter definition is provided in DQL matcher syntax.

Optional Select Query past events to estimate the volume of matching events in your environment.

Single events

All events are evaluated in the ingest pipeline against the DQL matcher expression, event by event. A DQL matcher expression, therefore, doesn't allow the use of any aggregation or querying over a set of events.

Problem trigger

Problems are created by Dynatrace Intelligence based on monitoring data. The Problem trigger allows selective triggering of a workflow in response. By default, the trigger fires once when a problem becomes active (on open and re-open), and optionally once when the problem is closed.

The following configuration parameters are available to define the triggering behavior of the Problem trigger.

Problem state
- active means that the problem is not closed yet.
- active or closed means that the problem can be either active or closed.
Event category definitions are in Event categories.
Severity filters problems by severity. Only problems at or above the selected severity level trigger the workflow.
Select Affected entities based on their tags. For more information on tags, see Define and apply tags.
- Include all entities
- Include entities with all defined tags below
- Include entities with any defined tag below

Under Advanced options, you can further refine the triggering conditions:

Updates allow you to re-trigger the workflow on changes to specific problem fields during the lifecycle of a problem. For details, see the Re-trigger on field changes section.
Define Additional custom filter query by adding any DQL matcher expression to the above definition.

Re-trigger on field changes

Dynatrace Intelligence updates problems during their lifecycle. For example, severity can rise as the situation evolves, or the set of affected entities can grow. By default, the Problem trigger fires only when the problem becomes active or is closed.

The Updates option allows you to opt in to re-triggering the workflow when a selected field changes. An update trigger will only happen once for each value combination of selected fields. It won't re-trigger if the fields return to a combination that has already fired.

Regardless of the Problem state, a change to any selected field re-triggers the workflow, including changes that occur after a problem closes.

The affected entities field tracks the union of two problem fields: affected_entity_ids and smartscape.affected_entities. This means that if you select affected entities for Updates, the workflow re-triggers whenever there's a change in the set of affected entities, regardless of which of the two fields the change is in.

Changing or clearing the list of fields for updates

Be aware that any change to the set of selected fields for Updates might cause one extra trigger on the next change to each affected problem.

Davis event trigger

Davis events are created by Dynatrace Intelligence based on monitoring data. The Davis event trigger allows selectively triggering a workflow in response.

The following configuration parameters are available to define the triggering behavior on Davis problem events

Problem state
- active means that the Davis event is not closed yet.
- active or closed means that the Davis event can be active or closed.
Davis event name
- equals means there's a match if the Davis event name exactly matches the event name string.
- contains means there's a match if the Davis event name contains the event name string.
Select the Affected entities based on their tags. For more information on tags, see Define and apply tags.
- Include all entities
- Include entities with all defined tags below
- Include entities with any defined tag below
Define Additional custom filter query by adding any DQL matcher expression to the above definition.

Permissions

The actor on a workflow with event trigger requires access to the event, thus the following permissions must be granted accordingly.

storage:events:read - in case of generic event trigger of type events, as well as Davis problem or event trigger
storage:security.events:read - in case of event trigger of type security.events
storage:bizevents:read - in case of generic event trigger of type bizevents
storage:system:read - in case of generic event trigger of type dt.system.events
storage:buckets:read - its recommended to restrict the permission to the bucket related to the event only

These permissions must be granted for any kind of user selected as an actor. In the case of an interactive user (nonsystem user), make sure that these permissions are also chosen in the authorization settings by this user. For details, see Manage workflow permissions.

Limits

Because event triggers can occur frequently, each workflow is limited to a maximum of 1,000 event triggers per hour, per workflow.

If an event trigger exceeds this limit, no further event triggers for this workflow will run for up to an hour.
If an event trigger reaches the limit three times within seven days, the event trigger is disabled.

In either case, the limit is flagged in the Workflows overview and can be filtered for.

To remedy this situation, we recommend adjusting the event trigger configuration and reenabling the trigger.

Hints on working with workflows and event triggers

To access the event payload to parameterize tasks in your workflow, use the event() expression.
For an event trigger, Workflows will prompt the event context of the last successful execution for manually triggered workflow runs. The Workflows prompt allows you to adjust the event context for manual runs during workflow development iterations.
With Query past events in the event trigger definition, you can quickly see how many matching events were observed in the past in your environment.
Use Notebooks to explore the events in your environment and determine which events you want to leverage.
Be aware that event filter expressions only support DQL matcher syntax, which is a subset of DQL.