Take control of log query costs using Retain with Included Queries

  • Tutorial
  • 6min

Introduction

Organizations often struggle with the enablement of log query best practices. When multiple teams access and analyze log data, especially after migration, they can experience unexpected query costs.

Using Retain with Included Queries helps you to:

  • Ensure predictable log query costs by shifting from usage-based billing to a retention-based model - ideal for teams that need upfront cost visibility and budget control.

  • Promote cost-efficient usage by guiding users and groups to run included queries, supporting responsible exploration without the need to monitor every query.

  • Streamline data access and operations by eliminating the need to reload data from archives or manage different storage tiers - users get instant access to all retained log data for up to 10 years.

Scenario

This guide shows you how to use the Retain with Included Queries capability to control and predict log consumption. By leveraging this feature, you can shift from a usage-based model (based on scanned bytes for every query) to a retention-based model - ideal for central teams supporting business units with strict budget constraints.

The provided flexibility allows you to assign billing models at the bucket level. You can choose from:

  • Usage-based: Pay for every query – ideal for not frequently accessed data

  • Retention-based: Use only included queries based on retained volume

  • Or hybrid (combining both): included queries for data up to 35 days old and on-demand billing when querying older data

Prerequisites

Before you begin, make sure the following requirements are met:

  • The Log Management and Analytics – Retain with Included Queries DPS capability must be enabled for your environment. Please contact Dynatrace Support or your Customer Success Manager (CSM).

  • You need the necessary permissions to configure Buckets and manage Permissions within your Dynatrace environment.

Steps

Configure the bucket for included queries

  1. Go to Settings > Storage Management and edit the log bucket you want to configure.

  2. Set the Included query retention period.

    In the Edit bucket dialog, define the number of days (minimum 10 and maximum 35). Queries in this timeframe will not be billable.

    Example: When included queries period is set to 12 days and you run query: fetch logs, from: now()-2d the query won’t be billable. If you run a query: fetch logs, from: now()-30d you will be charged for scanned bytes for data older than 13 days.

    Included query retention period setting can be changed once every 24 hours and takes effect daily at midnight UTC. Changes to this setting will affect your billing. Plan adjustments carefully to avoid unexpected costs.

  3. Understand your quota

    The data volume stored in a bucket configured for Retain with Included Queries defines the query volume that is included.

    Included query usage per day = (GiB of logs retained) × 15

    Example: if you retain 1 GB/day and set the included queries retention period to 10 days, your quota is 150 GB.

    Be aware of your quota. If you exceed your quota, Dynatrace reserves the right to throttle query throughput. You’ll still have access to logs, and our team will assist with optimization.

  4. Monitor usage

    Go to Dashboards > Ready-made > Log query usage and costs dashboard to track your Daily query volume and stay within your quota.

Set permissions for included queries only (optional)

  1. Go to Account Management > Identity & access management > Policies

  2. Select Create policy or edit existing policy by clicking > Edit policy

  3. Define a policy that restricts access to only included queries by adding following Policy statement:

    ALLOW storage:buckets:read WHERE storage:query-consumption="INCLUDED";

    For details, see Permissions in Grail.

  4. Select Save.

  5. Assign the policy

    Apply the policy to the relevant groups. For details on managing group permissions with IAM, see Manage group permissions with IAM policies.

    Users restricted to included queries will receive a warning if they attempt to scan data outside the allowed timeframe.

    Limited results warning

Conclusion

By configuring Retain with Included Queries, you’ve taken a key step toward predictable and efficient log management. This setup empowers your teams to explore log data confidently within defined cost boundaries, while giving administrators the tools to govern usage without added complexity. Whether you’re enabling business units or managing platform-wide cost governance, this approach helps you strike the right balance between flexibility and control to find the right model for your use cases.

Frequently asked questions

If your usage goes beyond the included queries quota, Dynatrace may throttle your queries to maintain system performance. However, your access to logs will not be restricted. Our team will proactively reach out to help you optimize your usage.

You can change it once every 24 hours. The change takes effect daily.

The quota is calculated as 15× the volume of retained data within the included queries period. For example, if you retain 1 GB/day and set the period to 10 days, your quota is:

15 × 1GB/day × 10 days = 150GB

No. Users who are restricted to included queries will receive a warning if they attempt to scan data outside the allowed timeframe, but they won’t be blocked from accessing logs within the included time-period for the scanned bucket.

Yes. The hybrid model allows you to use included queries for data up to 35 days old (minimum 10 days) and switch to on-demand billing for data retained beyond that period - without needing to manage separate storage tiers or rehydrate archived data.