Set up permissions for Distributed Tracing

Latest Dynatrace
How-to guide
1-min read

Dynatrace has a permission model for Grail. This applies to all telemetry data, such as metrics, events, spans, and logs.

We recommend setting up permissions along organizational lines and deployment scopes. Suitable concepts include host groups, Kubernetes clusters, and Kubernetes namespaces. These attributes are typically available for all telemetry data ingested via Dynatrace collection methods like OneAgent, OpenTelemetry, Kubernetes operator, etc. Hence, these attributes can be used to enable record-level permissions.

If you only require a basic permission concept, setting up bucket-level permissions is the best option. You can then route your data to the correct bucket in OpenPipeline by matching one of the mentioned deployment-relevant primary Grail fields.

For more control in Dynatrace, you can set up policy boundaries with more granular restrictions on a data level. By default, you can use the following attributes:

dt.host_group.id
host.name
k8s.cluster.name
k8s.namespace.name
Any other attribute listed in the permission model

Dynatrace provides a comprehensive permission model for Grail that applies to all telemetry data-including metrics, logs, spans, and events.

Set up the security context

If the permissions on deployment-level attributes or the bucket level are insufficient, Dynatrace allows you to set up fine-grained permissions by adding a dt.security_context attribute to your data. This context can represent your security architecture and could even be hierarchical by encoding this into a string such as: department-A/department-AB/team-C.

Leverage existing tags at the source

You can either define the security context already at the source via OneAgent, OpenTelemetry resource attributes, or Kubernetes labels or annotations. This enables you to use your existing labels and tags to facilitate permissions in Dynatrace.

Set up the security context in OpenPipeline

Alternatively, you can define a security context based on existing resource attributes to your span data within OpenPipeline:

Filter the records that should get the dt.security_context attribute added to them. To do so, open a new notebook and create a filtering DQL query, such as:

fetch spans
| filter matchesPhrase(deployment.release_stage, "prod-")

This query allows you to filter the span records to which you'll want to add the dt.security_context attribute. Once you're satisfied with the query result, copy the span processing function of the DQL query, in this case: matchesPhrase(deployment.release_stage, "prod-").

Define the spans security context rule using the resulting function and specify the value of the dt.security_context attribute. The value of the dt.security_context attribute can be a literal value that you provide, or the name of another attribute; the value will be used as the value of dt.security_context.

Recommendations for Distributed Tracing permissions

Permissions are typically configured for distributed tracing so users can see the complete end-to-end trace. Traces often span multiple services, hosts, or clusters, and cutting across traces with permission boundaries can result in incomplete or fragmented data. While service-level trace analytics will be less affected, and the distributed tracing app will work just fine, the potential lack of visibility impacts analytics and troubleshooting.

When setting up permissions for Distributed Tracing, consider these recommendations:

Avoid cutting through traces—make sure users can access all spans within a trace relevant to their role or deployment stage, while restricting access to sensitive services. Therefore, set graceful permissions and avoid boundaries restricting access to whole spans within a trace, as this can prevent a comprehensive analysis. For example, provide access to all spans in the relevant deployment stage (such as staging or production) or within organizational units (such as department or geographic region), and just restrict access to sensitive services (for example, the SSO).
Use field-level security for sensitive data—instead of restricting access to entire spans or traces, use field-level security to protect sensitive information.
- Dynatrace automatically identifies selected span attributes—defined in the Global Field Reference—and requests attributes marked as confidential.
- Only users with the builtin-sensitive-spans and builtin-request-attributes-spans field permissions can see these sensitive fields.
- Custom fieldsets can also be defined to specify sensitive attributes and the scope in which they apply. For example, see Masking at display.
Leverage the security context to define permissions on individual span records—Dynatrace allows you to tweak your ingested span data by adding a dt.security_context attribute to specific span records. This allows you to set additional options, such as permissions for individual records. To create a security context to your ingested span data, you need to create a pipeline rule.

User permissions for Distributed Tracing

When working with the Distributed Tracing Distributed Tracing, make sure that you've read and set all the necessary permissions:

Policy scope

Table permission

Read buckets data

storage:buckets:read

Read span data

storage:spans:read

Read entities data

storage:entities:read

Read log data

storage:logs:read

Read filter-segments data

storage:filter-segments:read

View sensitive fields trace data ¹ ²

storage:fieldsets:read WHERE storage:fieldset-name="builtin-sensitive-spans

storage:fieldsets:read WHERE storage:fieldset-name="builtin-request-attributes-spans

Read user app states

state:user-app-states:read

Write user app states

state:user-app-states:write

Delete user app states

state:user-app-states:delete

Sensitive attributes for spans are tagged as sensitive-spans in Global field reference.

To learn more about restricted view access to personal data and confidential request attributes, see Masking at display.