Grail stores security events in dedicated buckets in the security.events table. Starting with Dynatrace version 1.353, one bucket named default_security replaces both previous default buckets, and the default retention period becomes 35 days.
Your events keep flowing into Grail without any action on your part. Work through this guide if you filter queries by bucket name, or if you need to keep security events for longer than 35 days.
storage:security.events:read to query security events.storage:bucket-definitions:read and storage:bucket-definitions:write to change a bucket's retention period.storage:bucket-definitions:delete to delete a bucket.The following table compares the storage of each kind of security event before and after the change.
| Security events | Previous bucket | Previous retention | New bucket | New retention |
|---|---|---|---|---|
Generated by Dynatrace |
| Three years, fixed |
| 35 days, configurable |
Ingested from third-party tools |
| One year, configurable |
| 35 days, configurable |
Two further changes come with the new bucket:
You can configure security events in OpenPipeline like any other data, including processing rules, metric extraction, dt.security_context for record-level access control, dt.cost.costcenter for cost allocation, and routing to custom buckets.
Retention beyond 35 days is billable as Events powered by Grail - Retain for every security.events bucket, whichever source the events come from. For details, see Calculate your consumption of Events - Retain (DPS).
VULNERABILITY_STATE_REPORT_EVENT, VULNERABILITY_ASSESSMENT_CHANGE_EVENT, VULNERABILITY_TRACKING_LINK_CHANGE_EVENT, and VULNERABILITY_STATUS_CHANGE_EVENT remain in the default_security bucket, because a future release will deprecate them in favor of native concepts. You can't configure these event types in OpenPipeline or route them to another bucket. Adjusting the retention period of default_security still applies to them.
A query that names a bucket explicitly returns no new events once Dynatrace writes to default_security instead. Don't swap in the new bucket name, though: because you can route security events to any bucket through OpenPipeline, a bucket filter holds only for as long as the routing stays unchanged.
Filter on dt.openpipeline.source instead. Every event Dynatrace generates has the dt.openpipeline.source field set to built-in-security-events, which separates the two kinds of security event wherever each one is stored.
| Previous filter | New filter |
|---|---|
|
|
|
|
The previous filters match only a default configuration. If you already route security events to custom buckets, your queries name those buckets instead, and the same advice applies: filter on dt.openpipeline.source rather than on the bucket.
To update your queries
Review your notebooks, dashboards, and workflows for DQL queries that filter on a security events bucket.
Replace each bucket filter as shown in the preceding table. For example, a query for the events you ingest from third-party tools becomes:
fetch security.events| filter dt.openpipeline.source != "built-in-security-events"
Because the query no longer names a bucket, it returns matching events from every security.events bucket, including the events stored before the change.
Run each updated query and confirm that it returns the events you expect.
A query without a bucket filter scans every security.events bucket. In large environments, this increases your query cost and slows down results. To narrow the scan, add a dt.system.bucket filter that lists the buckets your OpenPipeline rules currently target, and update it whenever you change your routing. This is a performance optimization only; keep the dt.openpipeline.source filter.
If 35 days is too short for your use case, either extend the retention period of default_security or route your events to a bucket that keeps them longer. Extending default_security is quicker to set up and applies to every security event in your environment. Routing to a custom bucket takes more configuration, and lets you keep one group of events longer than the rest.
default_securityGo to
Storage Management > Grail Buckets.
Select the default_security bucket.
Enter a retention period between 1 and 3,657 days, then save your change.
The new retention period applies to every security event in the bucket, whether Dynatrace generated it or you ingested it.
Shortening a retention period deletes any data older than the new period. For details, see How to organize your data stored in Grail.
In
Storage Management > Grail Buckets, create a custom bucket for the security.events table with the retention period you need, as described in How to organize your data stored in Grail.
Go to
Settings > Process and contextualize > OpenPipeline > Security events.
Add a dynamic route that targets your new bucket, as described in Route data.
To route only the events you ingest from third-party tools, use this routing condition:
dt.openpipeline.source != "built-in-security-events"
To route only the events Dynatrace generates, use == instead of !=.
Place your route before the built-in security.events pipelines so that it runs first.
Wait for new events to arrive, then confirm that they reach the custom bucket:
fetch security.events| filter dt.system.bucket == "my_custom_bucket"| summarize count()
Dynatrace stops writing to default_securityevents_builtin and default_securityevents, but keeps both buckets so that you can still query the events already in them.
Dynatrace deletes default_securityevents_builtin for you once its original three-year retention period ends and the bucket is empty. default_securityevents stays in your environment until you delete it yourself.
To delete either bucket ahead of time
Confirm that the bucket is empty. The following query returns the record count for default_securityevents. Replace the name to check the other bucket:
fetch dt.system.buckets| filter dt.system.table == "security.events"| filter name == "default_securityevents"| fields name, dt.system.table, records, estimated_uncompressed_bytes
A records value of 0 means the bucket holds no events.
Confirm that none of your OpenPipeline routing rules still send data to the bucket. Go to
Settings > Process and contextualize > OpenPipeline > Security events and review your routes.
Go to
Storage Management, select the bucket, and delete it.