Log Management and Analytics (DPS)
Dynatrace SaaS only
The consumption model for Log Management and Analytics is based on three dimensions of data usage (Ingest & Process, Retain, and Query). The unit of measure for consumed data volume is gibibytes (GiB) as further described below.
Ingest & Process | Retain | Query | |
---|---|---|---|
Definition | Ingested data is the amount of raw data in bytes (logs and events) sent to Dynatrace before enrichment and transformation. | Retained data is the amount of data saved to storage after data parsing, enrichment, transformation, and filtering but before compression. | Queried data is the data read during the execution of a DQL query, including sampled data. |
Unit of measure | per gibibyte (GiB) | per gibibyte-day (GiB-day) | per gibibyte scanned (GiB scanned) |
Ingest & Process
What's included with the Ingest & Process data-usage dimension?
Concept | Explanation |
---|---|
Data delivery |
|
Topology enrichment |
|
Data transformation |
|
Data-retention control |
|
Conversion to time series |
|
Apply the following calculation to determine your consumption for the Ingest & Process data-usage dimension:
(number of GiBs ingested) × (GiB price as per your rate card) = consumption in your local currency
Be aware that data enrichment and processing can increase your data volume significantly. Depending on the source of the data, the technology, the attributes, and metadata added during processing, the total data volume after processing can increase by a factor of 2 or more.
Retain
Here's what's included with the Retain data-usage dimension:
Concept | Explanation |
---|---|
Data availability |
|
Retention periods |
|
Apply the following calculation to determine your consumption for the Retain data-usage dimension:
(number of GiB-days) × (retention period in days) × (GiB-day price as per your rate card) × (number of days that data is stored) = consumption in your local currency
Query
Query data usage occurs when:
Accessing the Logs & Events viewer in simple mode with filters.
Submitting custom DQL queries in the Logs & Events viewer in advanced mode.
Unified analysis pages show the log data of a specific entity.
Dashboard tiles that are based on log data trigger the execution of DQL queries on refresh and include sampled data.
Executing DQL queries in Notebooks, Workflows, Custom Apps and via API.
What's included with the Query data-usage dimension?
Concept | Explanation |
---|---|
On-read parsing |
|
Aggregation |
|
Reporting |
|
Context |
|
Apply the following calculation to determine your consumption for the Query data-usage dimension:
(number of GiB of uncompressed data read during query execution) × (GiB scanned price as per your rate card) = consumption in your local currency
Consumption examples
Following are example calculations which show how each data-usage dimension contributes to the overall usage and consumption.
Step 1 – Ingest & Process
For example, say that you produce 500 GiB of log data per day which you ingest into Log Management and Analytics for processing. The monthly consumption for Ingest & Process is calculated as follows:
Ingest volume per day | 500 GiB | |
Ingest volume per month | 15,000 GiB | 500 (GiB data per day) × 30 (days) |
Consumption per month | 15,000 (GiB per month) × ingest price as per your rate card |
Step 2 – Retain
Following the Ingest & Process step, your data is retained and enriched on an on-going basis. If you ingested 500 GiB of raw data in step 1,900 GiB of enriched data (500 GiB × 1.8 for enrichment) is added to your storage daily. In this example, your enriched data is retained for 35 days. The monthly consumption (after a ramp-up period of 35 days) for Retain is calculated as follows:
Retained volume for 1 day | 900 GiB | 500 (GiB data per day) × 1.8 (enrichment) |
Retained volume for 35 days | 31,500 GiB | 900 (GiB data per day) × 35 (days) |
Consumption per day | 31,500 (GiB) × retain price per day as per your rate card | |
Consumption per month | 31,500 (GiB) × retain price per day as per your rate card × 30 (days) |
If the same amount of processed data is to be retained for a year, the monthly consumption (after a ramp-up of 365 days in this case) for Retain is calculated as follows:
Retained volume for 1 day | 900 GB | 500 (GiB data per day) × 1.8 (enrichment) |
Retained volume for 365 days | 328,500 GiB | 900 (GiB data per day) × 365 (days) |
Consumption per day | 328,500 (GiB) × retain price per day as per your rate card | |
Consumption per month | 328,500 (GiB) × retain price per day as per your rate card × 30 (days) |
Step 3 – Query
Let's assume that to resolve incidents and analyze performance issues your team executes DQL queries with a total of 25,000 GiB of data read per day. The monthly the consumption for Query is calculated as follows:
Data volume read per day | 25,000 GiB | |
Data volume read per month | 750,000 GiB | 25,000 (GiB data per day) × 30 (days) |
Consumption per month | 750,000 (GiB per month) × query price as per your rate card |
Step 4 – Total consumption
The total monthly consumption for this example scenario of 35 days of data retention is the sum of the monthly consumption for Ingest & Process, Retain, and Query.
Consumption details
Dynatrace provides built-in metrics that help you understand and analyze your organization's consumption of Ingest & Processing, Retain, and Query for Log Management and Analytics. To use them in the Data Explorer, enter Log Management and Analytics into the Search field. These metrics are also available via the Environment API and are linked in your Account Management portal (Usage summary > Log Management and Analytics – Ingest & Process, Retain, Query > Actions > View details). The table below shows the list of metrics you can use to monitor the consumption details for Log Management and Analytics.
Metric key | Metric name | Metric dimension | Resolution | Description |
---|---|---|---|---|
builtin:billing.log.ingest.usage | Log Management and Analytics usage - Ingest & Process | Byte | 1 hour | Number of raw bytes sent to Dynatrace before enrichment and transformation in hourly intervals |
builtin:billing.log.retain.usage | Log Management and Analytics usage - Retain | Byte | 1 hour | Number of bytes saved to storage after data parsing, enrichment, transformation, and filtering but before compression |
builtin:billing.log.query.usage | Log Management and Analytics usage - Query | Byte | 1 hour | Number of bytes read during the execution of a DQL query, including sampled data |
Ingest & Process
You can monitor the total number of bytes ingested for Ingest & Process in hourly intervals for any selected timeframe using the metric Log Management and Analytics usage - Ingest & Process. The example below shows usage aggregated in 1-hour intervals between 2023-09-04 and 2023-09-11 (Last 7 days).
Retain
You can monitor the total bytes stored for Retain in hourly intervals for any selected timeframe using the metric Log Management and Analytics usage - Retain. The example below shows usage aggregated in 1-hour intervals between 2023-09-04 and 2023-09-11 (Last 7 days).
Query
You can monitor the total scanned bytes for Query in hourly intervals for any selected timeframe using the metric Log Management and Analytics usage - Query. The example below shows usage aggregated in 1-hour intervals between 2023-09-04 and 2023-09-11 (Last 7 days).