Metrics powered by Grail

  • Concept
  • 10min

The consumption model for Metrics powered by Grail is based on three dimensions of data usage. These correspond to the respective capabilities on your rate card.

Ingest & Process

Retain

Query

Definition

Ingested data is the number of metric data points sent to Dynatrace. A metric data point is a single measurement of a metric.

Retained data is the amount of data saved to storage after data parsing, enrichment, transformation, and filtering but before compression.

Queried data is the data read during the execution of a DQL query.

Unit of measure

Number of data points

Gibibyte-day (GiB-day)

Included

Ingest & Process

Here's what's included with the Ingest & Process data-usage dimension:

Concept

Explanation

Data delivery

Delivery of metrics via OneAgent, extensions or ingest API

Topology enrichment

Enrichment of metrics with data source and topology metadata

Data transformation

  • Rollup of data to reduced granularity to optimize queries for longer timeframes
  • Use of efficient data structures to derive metrics from high volume spans like service response time metrics

Data-retention control

Manage data retention period of incoming metrics based on bucket assignment rules

This section describes the usage stream for "Metrics - Ingest & Process" consumption.

Billable and non-billable metrics

Metrics are either billable or non-billable. Billable metrics consume your DPS subscription. Non-billable metrics do not consume your DPS subscription.

All metrics are billable unless otherwise noted.

Some monitoring modes include a certain amount of metric data points that can be used without consuming your DPS subscription. This is described further in the respective sections, see Host monitoring.

  • The following metric keys are non-billable:

    • dt.* 1
    • dt.cloud.aws.az.running
    • dt.cloud.azure.region.vms.initializing
    • dt.cloud.azure.region.vms.running
    • dt.cloud.azure.region.vms.stopped
    • dt.cloud.azure.vm_scale_set.vms.initializing
    • dt.cloud.azure.vm_scale_set.vms.running
    • dt.cloud.azure.vm_scale_set.vms.stopped
    • legacy.containers.*
    • legacy.dotnet.perform.*
    • legacy.tomcat.*
  • Metrics stored in the bucket dt_system_metrics are non-billable.

1

The following dt.* metric keys are billable unless otherwise noted in the list above: dt.cloud.aws.*, dt.cloud.azure.*, dt.osservice.*, dt.service.*.

Total platform usage

The total platform usage includes every metric data point ingested by Grail.

If many data points within a given metric interval are stored as one data point in Grail, only one data point is counted for total product usage.

Metric dimensions increase data point consumption

A single metric that is written once per minute will consume 525,600 metric data points annually:

1 metric data point × 60 min × 24 h × 365 days = 525,600 metric data points/year

Note that a single metric can have multiple dimensions. For example, if you have the same metric for 2 instances of your cloud service, you will consume 2 metric data points:

Metric key
dt.entity.dynamo_db_table
Value
cloud.aws.dynamo.requests.latency
DYNAMO_DB_TABLE-41043ED33F90F271
21.78
cloud.aws.dynamo.requests.latency
DYNAMO_DB_TABLE-707BF9DD5C975159
4.47

2 instances × 1 metric data point × 60 min × 24 h × 365 days = 1,051,200 metric data points/year

Metric data points are not billed based on an increase in dimensions, but rather by the increased number of metric data points. If dimensions are added, but the number of metric data points remains the same, then billable metric data points usage does not change:

Metric key
dt.entity.dynamo_db_table
Operation
Value
cloud.aws.dynamo.requests.latency
DYNAMO_DB_TABLE-41043ED33F90F271
DeleteItem
21.78
cloud.aws.dynamo.requests.latency
DYNAMO_DB_TABLE-707BF9DD5C975159
DeleteItem
4.47

Therefore, in this case, the same number of metric data points is consumed as shown in the calculation above.

Retain

15 months (462 days) of 1-minute granularity is included with Metrics powered by Grail. Metrics that you choose to retain beyond that period are billed as Metrics powered by Grail - Retain.

Here's what's included with the Retain data-usage dimension:

Concept

Explanation

Data availability

Retained data is accessible for analysis and querying until the end of the retention period. Metrics retention is defined at the bucket level, ensuring tailored retention periods for specific metrics.

Retention periods

Choose a desired retention period. For the default metrics bucket, the available retention period ranges from 15 months (462 days) to 10 years (3,657 days).

15 months (462 days) of 1-minute granularity is included with Metrics powered by Grail, hence it is the volume in excess of 462 days that affects monitoring consumption.

Apply the following calculation to determine your consumption for the Retain data-usage dimension:
(number of GiB-days) × (retention period in days) × (GiB-day price as per your rate card) × (number of days that data is stored in excess of 462 days) = consumption in your local currency

Query

Querying metrics using the timeseries command is always included, irrespective of the origin of the query (Dashboards, Notebooks, Apps or API).

timeseries avg(dt.host.cpu.usage)

Queries involving other data types generally incur usage at each query, even when they output time series format, for example using the maketimeseries command:

fetch logs | maketimeseries count()

If you frequently use predefined maketimeseries or summarize commands, it might be more cost-effective for you to create a log metric. Log metrics are regular metrics that are billed for Ingest & Process (and Retain above 15 months), but not for Query.

The above example could be turned into a log metric log.all_logs_count, consuming 525,600 metric data points per year (assuming at least one log record per minute), and the query would then become timeseries sum(log.all_logs_count).

Assuming the equivalent query using logs (fetch logs | maketimeseries count()):

  • scans 40 GiB over the last 2 hours,
  • is triggered 10 times per day each day.

The query usage over the logs version would be 40 GiB * 10 * 365 = 146,000 GiB per year. When multiplied by the Log Management & Analytics – Query price on your rate card, the Metrics powered by Grail – Ingest & Process cost of the log metric is two orders of magnitude less than the Metrics powered by Grail - Query cost of the command using the logs, and even more so if the amount of logs scanned increases (due to longer timeframes, for example).

Consumption examples

Following are example calculations which show how each data-usage dimension contributes to the overall usage and consumption.

Step 1 – Ingest & Process

For example, say that you produce 55 million billable metric data points per day which you ingest into Metrics powered by Grail. The consumption for one month (30 days) of Ingest & Process is calculated as follows:

  • Ingest volume per day: 55 million data points
  • Ingest volume per month: 55 million * 30 days = 1,650 million
  • Cost per month: 1,650 million * (Ingest & Process price as per your rate card) = Cost

Step 2 – Retain

Following the Ingest & Process step above, your enriched data is retained on an on-going basis. If you ingest 1.5 billion data points per day, 9 GiB of data might be added daily to your storage. Retain includes 15 months (462 days) of storage in the default retention period.

Assuming you don't change the default retention period, metric data will be retained for 15 months (462 days) and there will be no billable Retain usage.

If you increase the retention period to 5 years (1,832 days), the monthly consumption for retain (after 1,832 days) is calculated as follows:

  • Retain volume per day: 9 GiB
  • Total retain volume (after 1,832 days): 9 GiB * (1,832 days - 462 included days) = 12,330 GiB
  • Consumption per day (after 1,832 days): 12,330 GiB * (Retain price as per your rate card) = Cost
  • Consumption per month (after 1,832 days): 12,330 GiB * 30 days * (Retain price as per your rate card) = Cost

Step 3 – Query

Querying metrics using the timeseries command is included.

Step 4 – Total consumption

The total monthly consumption (30 days), after 5 years of retention, is the sum of the monthly consumption for Ingest & Process and Retain.

  • Ingest & Process: 1,650 million * (Ingest & Process price as per your rate card) = Cost
  • Retain: 12,330 GiB * 30 days * (Retain price as per your rate card) = Cost
Related tags
Dynatrace Platform