Take control of log query costs using Retain with Included Queries

  • Tutorial
  • 6-min read

Introduction

Logs are a powerful resource for understanding system behavior, diagnosing issues, supporting migration scenarios, etc. Across many teams, logs play a central role in daily operations and decision-making. However, this widespread use often leads to a high volume of queries–sometimes resulting in unexpected and significant costs.

Scenario

This guide explains how to use the DPS capability Retain with Included Queries to control and predict log consumption:

  • Ensure predictable log query costs by using the retention-based model, which is ideal for teams that need upfront cost visibility and budget control.

  • Promote cost-efficient usage by showing how users and groups can run included queries, supporting responsible exploration without the need to monitor every query.

Prerequisites

Before you begin, make sure the following requirements are met:

Steps

required

Go to Settings > Storage Management, select Bucket storage management, and select > Edit for the log bucket you want to configure.

required

In Edit bucket, set Included query retention period to the number of days (minimum 10 and maximum 35). Queries in this timeframe will not be billable.

Example: When Included query retention period is set to 12 days and you run query: fetch logs, from: now()-2d the query won’t be billable. If you run the query fetch logs, from: now()-30d, you will be charged for scanned bytes for data older than 13 days.

Included query retention period setting can be changed once every 24 hours and takes effect daily at midnight UTC. Changes to this setting will affect your billing. Plan adjustments carefully to avoid unexpected costs.

optional

By default, users can access all log data. You can restrict specific user groups to access only log data in the included timeframe.

To restrict access to only included log data, see Permissions in Grail.

  1. Go to Account Management > Identity & access management > Policy management

  2. Select Create policy or edit existing policy by clicking > Edit policy

  3. Define a policy that restricts access to only included queries by adding following Policy statement:

    ALLOW storage:buckets:read WHERE storage:query-consumption="INCLUDED";

    For details, see Permissions in Grail.

  4. Select Save.

  5. Assign the policy.

For details on managing group permissions with IAM, see Working with policies.

Users restricted to included queries will receive a warning if they attempt to scan data outside the allowed timeframe.

optional

The data volume stored in a bucket configured for Retain with Included Queries defines the query volume that is included.

Included query usage per day = (GiB of logs retained) × 15

Example: if you retain 1 GiB/day and set the included queries retention period to 10 days, your quota is 150 GiB.

Be aware of your quota. If you exceed your quota, Dynatrace reserves the right to throttle query throughput. You’ll still have access to logs, and our team will assist with optimization.

optional

Go to the Dashboards Dashboards, select Ready-made dashboards, and display the Log query usage and costs dashboard to track your Daily query volume and stay within your quota.

Conclusion

By configuring Retain with Included Queries, you’ve taken a key step toward predictable and efficient log management. This setup empowers your teams to explore log data confidently within defined cost boundaries, while giving administrators the tools to govern usage without added complexity. Whether you’re enabling business units or managing platform-wide cost governance, this approach helps you strike the right balance between flexibility and control to find the right model for your use cases.

Frequently asked questions

If your usage goes beyond the included queries quota, Dynatrace may throttle your queries to maintain system performance. However, your access to logs will not be restricted. Our team will proactively reach out to help you optimize your usage.

You can change it once every 24 hours. The change takes effect daily.

The quota is calculated as 15× the volume of retained data within the included queries period. For example, if you retain 1 GiB/day and set the period to 10 days, your quota is:

15 × 1 GiB/day × 10 days = 150 GiB

No. Users who are restricted to included queries will receive a warning if they attempt to scan data outside the allowed timeframe, but they won’t be blocked from accessing logs within the included time-period for the scanned bucket.

Yes. The hybrid model allows you to use included queries for data up to 35 days old (minimum 10 days) and switch to on-demand billing for data retained beyond that period - without needing to manage separate storage tiers or rehydrate archived data.