Grail is the Dynatrace database designed explicitly for observability and security data. It acts as single unified storage solution for logs, metrics, traces, events, and more. All data stored in Grail is interconnected within a real-time model that reflects the topology and dependencies within a monitored environment.
Grail is generally available for Dynatrace SaaS customers deployed on AWS or Azure.
Grail is schema-on-read and indexless, built with scaling in mind. There's no need to think about schema and indexes, re-hydration, or hot/cold storage. This architecture also means you are not required to determine your log data use cases beforehand (for example, at ingest). On-read parsing enables you to solve all analytics tasks on historical data stored in Grail.
Benefits of Grail:
Grail is built and optimized for Dynatrace Davis® AI, which processes billions of dependencies in a time-efficient manner to provide precise anomaly detection, root cause analysis, or business impact analysis. The Dynatrace Query Language (DQL) serves as the single interface to explore, query, combine, and process all data persisted in Grail. Queries are executed in parallel and retrieve results in real time without undermining execution performance.
The cost-effective licensing model supports your storage and analytics needs. You can filter, enrich, aggregate, and preserve your data without limiting analytic capabilities.
The Grail-based data management and analytics process has four stages.
Ingest
Any data ingested into Dynatrace eventually ends up in Grail.
Dynatrace forwards logs and business events into the Grail data lakehouse. The ingest endpoints receive the data and channel it as records into the processing pipeline .
Process
The processing pipeline transforms and enriches records with additional fields and determines target buckets.
Store
Records are stored in buckets. The retention period and the record type are set separately for each bucket. Each bucket is assigned to a DQL database table. You need to assign permissions to user groups or single users to provide them with access to buckets and tables.
Query
Dynatrace provides a single interface to query all kinds of data using Dynatrace Query Language (DQL), which offers a set of commands that you can use to query logs, events, and more.
The DQL fetch command loads records from a table. Tables serve to form sets of stored records into logical groups. The predefined tables for Grail are logs and business events.
Log Management and Analytics for Dynatrace SaaS leverages the unique architecture of the Grail™ storage engine. Grail allows you to benefit from your log data with these innovations:
Scalability
Your log data—no matter the volume, rate of growth, or configured retention period—is always accessible for detailed analysis. Storage is managed for you automatically, without the need to manage indexes or archives.
Adjustable consumption
A three-tier log data-usage model tracks your DDU consumption across three dimensions: Ingest & Process, Retain, and Query. This approach ensures that your environment's DDU consumption is closely aligned with your usage and the value you receive. For details, see DDUs for Log Management and Analytics.
Advanced analytics
Advanced analytics gives you direct access to the log content of all your system's critical processes. You can filter your log data using the log viewer, as well as retrieve and organize your data using DQL.
Flexible retention
Choose the period of retention to match specific use-cases of log data.
Dynatrace Business Analytics is a powerful tool for ingesting, exploring, and analyzing your organization's critical business data.
Business event processing uses the capabilities of Grail and Dynatrace Query Language (DQL) to refine your data. Within the business event ingest pipeline, you can define how your data will be further processed.