Configure a segment to function as a convenient filter for logs by Grail buckets to optimize query performance and license consumption.
This article is intended for administrators controlling data partitioning in Grail buckets as well as power users aiming for more efficient log queries.
In this article, you'll learn how to create a new segment to function as a log bucket filter for fetching logs queries. You'll also learn the difference between one-off segments using static conditions and dynamic segments leveraging variables.
storage:filter-segments:write
and storage:filter-segments:read
permissions. To learn how to set up the permissions, see Permissions in Grail.Logs powered by Grail can be stored in different buckets. Buckets can improve query performance by reducing query execution time and the scope of data read.
Segment configured with static conditions for a one-off scenario.
Segment configured with dynamic conditions using variables.
Create a segment for a single log bucket
Add a variable to filter for any log bucket
Use segment to analyze logs by bucket
Having a segment for a single bucket might be desired in some situations. The following example shows how to do that by filtering for logs of bucket logs_default
.
dt.system.bucket
dt.system.bucket = logs_default
)Successfully configured segments are displayed in the segments list. Select > Edit to modify a segment.
Adding a variable to the segment to dynamically filter for many log buckets instead of one makes the segment universally applicable.
Select Variable
Query the list of log buckets, sorted alphabetically
fetch dt.system.buckets| filter dt.system.table == "logs"| fields bucket=name| sort bucket
Select Run query
Select Done to finish variable configuration
Adjust the condition of the include for logs to use $bucket
variable (dt.system.bucket = $bucket
)
Select Save
Successfully configured variables are displayed as on top of existing include blocks of a segment. Select Edit variable(s) to modify variables.
You can analyze logs in different apps. To query for logs in Notebooks
You’ve configured a segment for a single bucket statically. You’ve learned how variables help to make segments more dynamic and cover broader use cases. Lastly, you’ve seen how to analyze logs of certain buckets, allowing optimization of query performance and license consumption.