Log metrics

Latest Dynatrace
How-to guide
7-min read

Dynatrace Log Management and Analytics gives you the ability not only to view and analyze logs, but also to create metrics based on log data and use them throughout Dynatrace like any other metric. Log metrics are stored in Grail and can be accessed and queried with Dynatrace Query Language. You can add them to your dashboard, include them in an analysis, and even create custom alerts.

Log metric pricing is based on the Dynatrace Platform Subscription (DPS) model. For details, see Metrics powered by Grail overview (DPS).

Depending on the processor you select during log metric creation, the new metric value can represent:

Counter metric: The metric value represents a count of log records that match the query. No attribute extraction is required.
Value metric: The metric value represents a numeric log attribute, recorded as a gauge. Use it for current states or absolute measurements, such as memory usage or connection counts. Supported aggregations: Average, Count, Maximum, Minimum, Sum.
Histogram metric: The metric value represents a numeric log attribute, recorded as a histogram. Use it to track value distributions such as request durations or payload sizes. Supported aggregations: Median, Percentile 10th, Percentile 75th, Percentile 90th.

For details on the available processors and their parameters, see Metric extraction stage in OpenPipeline.

For Value metric and Histogram metric, the specified attribute must be of numeric type. Dynatrace will attempt to convert string type attributes to numbers as long as they match the following pattern:
123
123.
123.456
.456
-.456
+.456
+123.456
-123.456
123.4e+5
-123.4E-5
1234e+5
1234e5

Attributes holding more than one value are not converted to attribute-based metrics.

Creating log metric

All metrics based on a log monitoring query have a metric key with the log. prefix.

Log metric availability in Dynatrace

A created log metric is available only when new log data is ingested and it matches the log query defined during log metric creation. Ensure that new log data has been ingested before utilizing the log metric in other areas of Dynatrace.

The range of a timestamp for accepting data that is used in metric creation is between 1 hour in the past and 10 minutes in the future. If no timestamp is provided, the current timestamp of the server is used.

There are two ways to create a metric based on log data:

Create metric using settings

Go to Settings > Process and contextualize > OpenPipeline > Logs and select the Pipelines tab.
Find the pipeline you want to modify, or add a new pipeline.
Select Edit. The pipeline configuration page is displayed.
Select the Metric extraction tab, add a metric processor, and choose the metric type.
Configure the processor, including the metric name and ID.
In the Matching condition field, enter a DQL matcher to filter the log data for your metric.
Optional Select Add dimension one or more times to split the metric by a specific log data attribute.
Select Save changes to create the log metric.

For a guided walkthrough, see Set up custom alerts based on metrics extracted from logs.

Create metrics from logs without storing the logs

If you want to extract metrics from logs without retaining the original log data, combine metric extraction with a no-storage assignment in the same pipeline.

Create a metric as described in Create a metric.

Configure the Storage stage to skip storage for the matching logs. When you configure No storage assignment, the record continues through all configured pipeline stages and is not stored only at the end of the pipeline. This means you can extract metrics and generate alerts from records that you won't store. For details, see Skip storage for selected logs.

Editing log metric

To list, enable, disable, delete, or modify metrics created from log data, go to Settings > Process and contextualize > OpenPipeline > Logs, select the Pipelines tab, open the relevant pipeline, and select the Metric extraction tab.

Editing a metric key will generate a new metric. As a result, historical data will be accessible only with the old metric key.

Using log metrics

Once you've defined a metric, you can chart it, pin it to a dashboard, or create an alert based on it.

Dashboard: Go to Dashboards add a new tile, and select Metrics to find and add your metric. For details, see Explore metrics.
Alert: Go to Anomaly Detection and create a new custom alert based on your metric. For details, see Set up custom alerts based on metrics extracted from logs.