Add Site Reliability Guardian objective

Latest Dynatrace

Objectives

An objective is a means for measuring your services such as performance, availability, and security. You can define an objective for your guardian that is validated on demand or automatically.

Add new objective

You can add a new objective for measuring capacity objectives.

  1. Create a Site Reliability Guardian

  2. On the Site Reliability Guardian overview page, select Add Add new objective.

  3. On the right side, enter a new Objective name.

  4. Enter the Description.

  5. Choose from where you Fetch your data from. It is either Grail using DQL or SLO.

    Option Grail using DQL: Define the DQL query.

    Option SLO: Choose from the Reference SLO list.

  6. Run query.

  7. Choose either Static thresholds,

    1. Define thresholds for your objective.

    2. Set Failure.

    3. Set Warning.

  8. or Auto-adaptive thresholds (only available for fetching data via DQL query).

  9. Choose either A higher value is good for my result or A lower value is good for my result.

Informational-only objectives

It is possible to add an objective that is informational-only and doesn't contribute to the pass / fail decisions of the Site Reliability Guardian. They are useful for new services where you're trying to get an idea for the real-world data values of your metrics.

To set an objective as information-only

  1. Select the objective to open the side panel.
  2. In the side panel, move through until you see the Define thresholds section.
  3. Select the No thresholds option.

Objective Examples

Use these objective examples as an inspiration for creating your own guardians.

Error log entries

Objective

  • We want the number of error log entries to be as low as possible.
  • Starting at 8 errors, we want to receive a warning.
  • Once it reaches 10 errors or more, the objective should fail.

Indicator

For this, we'll create a query that fetches the log events from your environment with the loglevel set to ERROR.

fetch logs
| summarize countIf(loglevel == "ERROR")

Comparison operator

Less or equal than (A lower value is good for my result)

Static warning and failure thresholds

  • Failure: 10
  • Warning: 8

Request success rate - log based

Objective

  • We want the ratio of successful requests to total requests to be as close as possible to 100 percent.
  • Starting at 99 percent, we want to get a warning.
  • Once it drops below 98 percent, the objective should fail.

Indicator

For this, we'll use a query that:

  • Fetches some specific log entries and parses them for events indicating requests.
  • Defines what's the successful response in these entries.
  • Measures the success criteria.
fetch logs
| filter endsWith(log.source,"pgi.log")
| parse content, "LD IPADDR:ip ':' LONG:payload SPACE LD 'HTTP_STATUS' SPACE INT:http_status LD (EOL| EOS)"
| fieldsAdd success =  toLong(http_status < 400)
| summarize successRate = sum(success)/count() * 100

Comparison operator

Greater or equals than (A higher value is good for my result)

Static warning and failure thresholds

  • Failure: 98
  • Warning: 99

Request failure rate - log based

Objective

  • We want the ratio of failed requests to total requests to be as low as possible.
  • When the ratio is at 0.5 percent, we want to get a warning.
  • When the ratio increases above 1 percent, the objective should fail.

Indicator

For this, we'll use a query that:

  • Fetches some specific log entries and parses them for events indicating requests.
  • Defines the failed request in these entries.
  • Measures the failure criteria.
fetch logs
| filter endsWith(log.source,"pgi.log")
| parse content, "LD 'HTTP_STATUS' SPACE INT:http_status LD (EOL| EOS)"
| fieldsAdd failure =  toLong(http_status >= 400)
| summarize failureRate = sum(failure)/count() * 100

Comparison operator

Less or equal than (A lower value is good for my result)

Static warning and failure thresholds

  • Target: 1
  • Warning: 0.5

Variables

Variables are defined at a guardian level and are available in all of its objectives to parameterize DQL queries. For example, they allow parameterization of the filter value in a DQL query, which is set at the validation time of a guardian.

Add a variable

There are several ways to add a variable.

Always remember to prepend a variable name with $ in your queries, followed by alphanumerical characters.

For example, if you create a variable named MyTotal, you need to refer to it as $MyTotal in your query.

To add a variable

  1. Select the More actions ellipse next to the title.

SRG Templates and Variables drop down list

  1. Select Dashboards Variables.
  2. In the Variables dialog, select Add Add variable.
  3. In the Add a variable dialog, enter the Name and the Value.
  4. Select Add. You created your variable.

To add a variable when you choose the Grail using DQL option while configuring your objective

  1. In DQL query, enter $, followed by alphanumerical characters.
  2. Select Add Add variable.

Set a variable value

There are two ways to set the value of a variable after adding it:

  • Access it after triggering the guardian validation.
  • Set it in the Workflow action.

If no variable value is explicitly specified for one validation, then the one set during the guardian creation is used.