This page describes the Dynatrace workflow for applying daily quotas for log query consumption to users.
You will learn how to set up a workflow to check, each minute, whether a user has exceeded their allocated daily query quota. All steps in this workflow are fully customizable to your needs.
The default daily quota is 10 TB, equivalent to 35 USD of query volume at the rate card list price.
This workflow checks every minute whether a user has exceeded its allocated query quota per day.
If the quota is exceeded
This can be used for auditing which users are hitting the quota limit. It’s also being used to determine whether a user has already been locked for the day.
A Dynatrace account administrator can subsequently unlock the user again by removing the “Quota exceeded” group again. The workflow will not lock the user anymore for this day. At midnight UTC time, the workflow will automatically remove all users from the “Quota exceeded” group again, so that they can start fresh on the next day.
Go to Account Management > Identity & access management > Policy Management.
Create a new policy called "Deny logs (quota exceeded)." Set the following Policy statement:
DENY storage:logs:read;
Create a new policy called "Quota exceeded check." Set the following Policy statement:
//Basic permissions to run workflowsALLOW environment:roles:viewer;ALLOW automation:workflows:read;ALLOW app-engine:functions:run;ALLOW app-engine:apps:run;//Allow to read billing usage dataALLOW storage:bucket-definitions:read;ALLOW storage:system:read WHERE storage:event.kind = "BILLING_USAGE_EVENT";//Allow to read whether the Quota exceeded event was sent beforeALLOW storage:bizevents:read WHERE storage:event.type = "log.quota.exceeded";//Allow to write a Quota exceeded eventALLOW storage:events:write;//Allow to read usersALLOW iam:users:read, iam:groups:read;//for sending “Quota exceeded” emailsALLOW email:emails:send;
Go to Account Management > Identity & access management > Group Management.
Create a new group.
a. Set the Group name to Quota exceeded. b. Select Permissions to add a new permission. c. Add the "Deny logs (quota exceeded)" permission that you created in the previous step. d. Set the Permission Type to POLICY. e. Set the Scope to Account.
Go to Account Management > Identity & access management > Service Users.
Create a new service user.
a. Set the Service user name to Quota Exceeded Check User. b. Select Permissions to add a new permission. c. Add the “Quota exceeded check” policy to the user. d. Set the Permission Type to POLICY. e. Set the Scope to Account. d. Copy the Service user email and save it somewhere safe.
For more information about OAuth clients, see OAuth clients.
Go to Account Management > Identity & Access Management > OAuth clients.
#TODO SOMEWHERE SELECT SOMETHING TO Create a new OAuth client.
a. In subject user email, select a user that has the Account manager permission. Note that you can't select the service user here. b. In Permissions, select Account > View users and groups and Account > Manage users and groups. c. #TODO SOMEWHERE SELECT SOMETHING to Create the client.
Copy the Client ID, Client secret, and Dynatrace account URN and save them somewhere safe.
For more information about Credential Vault, see Credential vault.
urn:dtaccount:
)Create a new text file and copy the contents of the code block below.
example codewftpl_log_quota_check.yaml
In Workflows, create a new workflow.
Select and set Actor to Quote Exceeded Check User SERVICE_IDENTITY.
Select to save the workflow.
You can customize the limits by modifying the DQL query. The full query is shown below, and sections below indicate how to change the query to achieve certain use cases.
/*Get all users that consumed > 10 TB of log query todayand whom haven't been locked out before today*/fetch dt.system.events, from: -1d@d| filter event.kind == "BILLING_USAGE_EVENT"| filter event.type == "Log Management & Analytics - Query"| filter event.version == "1.0"| dedup event.id| summarize {query_volume = sum(billed_bytes)}, by:{user.email}| filter query_volume > 10 * 1024 * 1024 * 1024 * 1024 //Modify `10 TB` to the desired quota| filterOut user.email in [fetch bizevents, from: -1d@d| filter event.type=="log.quota.exceeded"| fields user.email]| fields email = user.email
By default the quota per user is 10 TB = 35 USD at list price.
Modify the filter query_volume > 10 * 1024 * 1024 * 1024 * 1024
.
Change the value of 10
to the desired quota that can be scanned, in TB.
You can exclude certain users from the quota check by changing the above query.
The workflow can be customized to also take weekly or monthly quotas into account.
In Workflows, open the Workflow and select the quota_reset_at_midnight action.
Under Conditions, modify And custom condition was met.
You can modify the timezone and the specific time.
a. To modify the timezone, change UTC
to the desired timezone abbreviation.
b. To modify the time, change 00:00
to the desired time, in hours and minutes.
You can customize this to a different point in time and different timezone. If you adjust this, make sure you do one of the following:
By default, the workflow sends an e-mail to the user who is exceeding the quota. Ideas for customization here might be:
Similar to log management & analytics, this workflow can work for any querying capability in Dynatrace (such as Events powered by Grail – Query or Traces powered by Grail – Query). Adjust the query above and the policy to also include those capabilities.
For ingest-based capabilities (like Traces powered by Grail – Ingest, Log powered by Grail – Ingest) a similar workflow could be created which automatically deactivates API tokens once a certain quota is reached.