Site Reliability Guardian

Latest Dynatrace

The Site Reliability Guardian is a Dynatrace app that automates change impact analysis to validate service availability, performance, and capacity objectives across various systems. It enables DevOps platform engineers to make the right release decisions and empowers SREs to apply Service-Level Objectives (SLOs) for their critical services.

Site Reliability Guardian concepts

Site Reliability Guardian is based on the following concepts:

Guardian

A guardian is the grouping of objectives. It is built around a set of entities reflecting a service or application you want to safeguard.

A guardian provides you with a default automation workflow that performs the objective validation. As a result, a guardian always represents the latest validation result derived from the objectives.

You can create a maximum of 1000 guardians.

Objective

Objectives are means for measuring the performance, availability, capacity, and security of your services. Objectives are measured by indicators. You can define an objective for your guardian that is validated on demand or automatically.

You can create a maximum of 50 objectives for each guardian.

Indicator

An indicator is a value against which the warning and failure thresholds are checked using a comparison operator. To retrieve an indicator value, use DQL or reference an existing SLO.

Static thresholds

The static warning and failure thresholds determine whether the measured value of the indicator meets the objective, is close to violating the objective, or violates the objective.

Warning and failure are optional; objective validation can vary:

  • If both the warning and failure thresholds are set, the objective validation can return warning, failure, or pass.
  • If just the warning threshold is set, the objective validation can return warning or pass.
  • If no threshold is set, the objective validation does not return a status but is used for informational purposes.

Auto-adaptive thresholds

Auto-adaptive thresholds are dynamic limits that adjust over time based on previous validations. If an objective changes its behavior, the threshold adapts automatically.

  • Auto-adaptive thresholds are only available for fetching data via DQL. The Davis AI threshold analyzer requires a minimum of 5 validations for auto-adaptive thresholds to take effect. During this learning phase, the objective validation returns an info state. All subsequent validations will then use the auto-adapted thresholds, impacting the overall validation.
  • Switching from static to auto-adaptive is supported.

Operator

The comparison operator defines whether the objective is met: the indicator is less than or equal to (A lower value is good for my result), or it is greater than or equal to (A higher value is good for my result), the warning and failure threshold.

Tags

To organize your guardians, you can assign tags to them. Tags use the key:value format, with the value being optional.

To assign a tag to your guardian, either specify it in the Add tags to your guardian section during guardian creation or add the tag later in edit mode.

To filter the list of all guardians by a tag, type the tag in the Search by name or tag field—the page automatically updates to show only guardians with matching tags.

This DQL shows you the first guardian.validation.objective business event with a specific guardian ID and parses the guardian tags field to extract a specific tag value from the event JSON.

fetch bizevents |
filter event.type == "guardian.validation.objective" AND guardian.id == "vu9U3hXa3q0AAAABADFhcHA6ZHluYXRyYWNlLnNpdGUucmVsaWFiaWxpdHkuZ3VhcmRpYW46Z3VhcmRpYW5zAAZ0ZW5hbnQABnRlbmFudAAkMWNiZDVkYWYtZThhNi0zMDkxLWFkOGQtMmU5NDNmNWJmZWJmvu9U3hXa3q0" |
limit 1 |
parse guardian.tags, "JSON:parsed_guardian_tags"

This DQL shows you all guardian.validation.finished business events from guardians tagged as tagkey:my-tagged-guardian.

fetch bizevents
| filter event.type == "guardian.validation.finished"
| expand guardian.tags
| filter contains(guardian.tags, "my-tagged-guardian")

Guardian workflow action

You can automate the execution of a guardian via Workflows, tying guardian execution to an event.

To add a guardian action to an existing workflow

  1. Go to Workflows Workflows and open the required workflow.
    Alternatively, select Automate on the guardian page. This option is available on the overview on the tile itself as well as in the validation details.
  2. In the Choose trigger panel, select the trigger best suited to your needs.
  3. On the trigger node, select Add to browse available actions.
  4. Find Site Reliability Guardian in the Choose action panel.
  5. On the Input tab, you have two options to select the required guardian:
    • Select the guardian from the list.
    • Use an expression to extract the guardian from the triggering event or a previous workflow action.
  6. Configure the validation timeframe. You have two options:
    • Select the required timeframe.
    • Use the event() expression to extract the timeframe from the triggering event.

You can create a new workflow by selecting Automate on the top right of the guardian page. When you create a workflow this way, the following parameters are configured, but be sure to adapt them as needed.

  • If the guardian has tags defined, they are used for the event filter of the trigger. Otherwise, it defaults to tag.service == "carts" AND tag.stage == "production".
  • The first action of the workflow is the respective guardian.

The guardian action generates the following output and passes it to the subsequent actions of the workflow.

Parameter

Description

guardian_id

The ID of the validated guardian

guardian_name

The name of the validated guardian

guardian_tags

An array of tags assigned to the validated guardian

execution_context

The execution context property of the trigger, if it was set

validation_id

The ID of all events generated by the validation

validation_url

The URL with the full validation details

validation_status

The status of the validation, indicating the overall result. The following values are possible:

  • info
  • pass
  • warning
  • fail
  • error

validation_summary

The number of objectives for each status

To learn more about workflows for a guardian, open the help menu in the upper-right corner of a guardian and select Get started with Automation.

Validate guardian and its objectives

If a workflow is created, your guardian is validated automatically. You can also perform the validation manually.

Validation overview

By default, the overview page shows validations for the last seven days. You can view older results by opening a guardian and selecting a different timeframe.

Automated validation

The event subscriptions in the workflow define when the validation of a guardian has triggered automatically.

Manual validation

You can perform a validation of a guardian by selecting the Validate button on the overview screen or within the validation details screen.

  • Select the validation timeframe.
  • Click the Validate button.

Individual objective result

For each objective, the validation returns the derived value and classification. The severity goes from the highest (1) to the lowest (5).

Severity
Name
Description
1
Error
The objective could not be validated due to an error deriving the indicator.
2
Fail
The value violates the failure threshold; the objective is not met.
3
Warning
The value is in the warning range; the objective is met, but close to failure.
4
Pass
The value is within the target range, the objective is met.
5
Info
No classification, but the objective's value can be used for informational purposes.

Overall validation result

After the validation of each objective is done, the guarding uses the most severe of individual validations as the overall validation result. Examples of this result usage include:

  • Making a release decision in your delivery pipeline.
  • Reporting on the current status of your service.