Snowflake for Workflows (Preview) - Best practices for Grail

By default, business events stored via the Store statement result Action of Snowflake for Workflows (Preview) are stored in Grail bucket default_bizevents with a retention period of 35 days. For more details, see Built-in Grail buckets.

To manage retention and costs, and protect data, we recommend creating a dedicated bucket and setting up policies that allow data access from this bucket.

Store data in a separate bucket

  1. Create a dedicated bucket for business events with a custom retention period. We will refer to this bucket as <bucket-name>.
  2. Configure a business event bucket assignment with a matcher-specific DQL query, as shown in the example below.
matchesPhrase(event.type, "snowflake_connector") and matchesValue(event.provider, "data_connector")

Policies

To allow access to this bucket, create the following policies via Account Management:

  • Read Snowflake-data policy (needed for any user that wants to read data from this bucket): 
    ALLOW storage:buckets:read
      WHERE storage:bucket-name = "<bucket-name>";
    ALLOW storage:bizevents:read;
  • Write Snowflake-data policy (needed for the user that executes the workflow action): 
    ALLOW storage:buckets:read
      WHERE storage:bucket-name = "<bucket-name>";
    ALLOW storage:events:write;

For more details, see Permissions in Grail.

To roll out policies on a large scale, we recommend using Dynatrace Configuration as Code (Monaco).