Record deletion in Grail via API

  • 4min

You can delete records from Grail using the API by leveraging DQL (Dynatrace Query Language), which allows you to specify criteria for selecting records to be deleted.

The record deletion API is primarily designed to help users remove selected records containing privacy data. Alternatively, you can use Privacy Rights to review and delete data stored in Grail. Privacy Rights utilizes the record deletion API and provides a user-friendly interface on top of it.

With the record deletion API, you can delete records stored in the following tables from both built-in and custom buckets:

  • logs
  • events
  • bizevents
  • spans
  • user.events
  • user.sessions
  • application.snapshots

To delete spans, you need a “trace query entitlement” to be able to execute a fetch spans query. This must be enabled as a capability in your Dynatrace Platform Subscription (DPS).

Record deletion is final and can't be undone.

Access API documentation

To access the Swagger API documentation for the record deletion API and to start making API requests using Swagger:

  1. Search for and select Dynatrace API.
  2. In the Select a definition field, select Grail - Storage Record Deletion.
  3. optional Authenticate with your API token if you want to use Swagger to execute your requests. For details, see Authentication. Select the Try it out button to interact with the API directly from the documentation.
  4. Perform one of the following actions.

To do this

Go to Record Delete and select this

Cancel running process

POST/delete:cancel

Start new deletion process

POST/delete:execute

Get status of currently running process

POST/delete:status

To delete your records, you need to use the /delete:execute endpoint and a DQL query which can contain the following commands and functions.

Available DQL commands

Available DQL functions

fetch, fields, fieldsAdd, fieldsRemove, filter, parse

concat, contains, timestamp, timeframe, asNumber, asDouble, asLong, asString, asBoolean, toDouble, toLong, toString, toBoolean, now, matchesPhrase, matchesValue, matchesPattern

The fetch command parameters scanLimitGBytes and samplingRatio are not available in the Record deletion API.

Grant access with IAM permissions

To delete the records, the policy bound to your user group must contain the following permissions:

  • storage:records:delete
  • storage:events:read
  • storage:logs:read
  • storage:bizevents:read
  • storage:spans:read
  • storage:buckets:read

If you don't have these permissions, you'll get a 403 error. To learn more about permissions, see IAM policies.

When creating an OAuth token or platform token to make API calls from an API client, ensure these permissions are also configured for the token. The user linked to that OAuth token or platform token must have these permissions assigned.

Example API requests

The following examples show curl commands for interacting with the Record Deletion API using a platform token.

Start a new deletion process to delete log records:

curl -X POST "https://<environment>.apps.dynatrace.com/platform/storage/record/v1/delete:execute" \
-H "Authorization: Bearer <platformtoken>" \
-H "Content-Type: application/json" \
-H 'accept: application/json' \
-d '{
        "query": "fetch logs\n | filter contains(content, \"confidential data to be removed\")",
        "timeFrame": {
        "start": "2025-02-04T00:00:00Z",
        "end": "2025-02-05T00:00:00Z"
   },
   "timezone": "UTC"
}' \
-w "\nHTTP Status Code: %{http_code}\n"


> {"taskId":"d2ee1222-2775-4e94-b422-5f948a4349e0"}
> HTTP Status Code: 202

Get status of a currently running process:

curl -X 'POST' \
  'https://<environment>.apps.dynatrace.com/platform/storage/record/v1/delete:status' \
  -H "Authorization: Bearer <platformtoken>" \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "taskId": "d2ee1222-2775-4e94-b422-5f948a4349e0"
  }' \
-w "\nHTTP Status Code: %{http_code}\n"


> {"status":"finished","message":"Deletion of records has been completed. 11502351 byte(s) have been rewritten."}
> HTTP Status Code: 200

Limitations

To avoid excessive deletions on a large volume of records, we introduced the following limitations.

  • Only one deletion request can be in progress per tenant at any given time. Additional requests will be accepted and queued for execution.
  • The maximum timeframe for a record deletion request is 24 hours. If you need to delete data spanning multiple days, you must submit separate requests for each day.
  • The end of the timeframe must be at least 4 hours prior to the time of record deletion.

Deleting data using the record deletion API is a hard delete, but it won't reduce the bucket size.

To manage bucket size, go to Settings > Storage management or use the Storage Management API.