OpenPipeline pipeline groups

  • Latest Dynatrace
  • Explanation
  • 4-min read
  • Published Feb 11, 2026

Pipeline groups provide centralized control and compliance in large-scale pipeline ecosystems. Administrators can enforce critical security and operational standards on the global level, such as security and cost allocation, while enabling development teams to build their own pipelines. Pipeline groups prevent misconfiguration, support regulatory requirements, and simplify operational oversight.

Overview

Pipeline groups let central platform and SRE teams manage ingest configuration for multiple pipelines at once. A pipeline group defines a shared configuration—composition—for a set of pipelines—member pipelines. The configuration can restrict or mandate stages.

  • Restrict: Prevent stages of member pipelines from being execute.
  • Mandate: Composition stages are executed before or after a certain member stage, improving consistent processing and reusability.

Administrators own and manage the group, as well as its compositions and which stages are enabled or disabled for the member pipelines. Development teams own their member pipelines which operate within the group's boundaries.

  • A pipeline belongs to only one pipeline group at a time.
  • Pipeline groups can't be nested.

Pipeline group

Composition

A composition enables administrators to enforce consistency without blocking team autonomy. It defines what must happen across all pipelines in the group and how execution flows between composition and member stages.

A composition:

  • Exists only inside a pipeline group.
  • Mandates stages.
  • Defines the execution order between composition and member stages.

Compositions are configured via one or multiple composition pipelines owned by administrators. Composition pipelines can be custom (created by administrators) or ready‑made (provided by extensions).

Member pipeline

Member pipelines implement team-specific logic within the group boundaries, allowing teamas to focus on domain‑specific transformations without compromising global standards.

Member pipelines inherit restrictions and mandatory stages from the pipeline group. Teams can add optional stages after mandated ones but can't override restricted stages.

A member pipeline:

  • Can be custom (created by a user) or ready‑made (provided by an extension).
  • Can exist standalone or inside a group.
  • When part of a group, data is processed by the stages that the group enables.

Mandate

Mandating a stage ensures it's executed and in the correct order between composition and member stages. This guarantes data is consistently processed.

Example: An administrator mandates the Permissions stage to run before any departmental definition. Global rules are applied first, preventing premature or conflicting permission changes.

"isPipelinePlaceholder": false,
"pipelineId": "<pipeline-id>",
"stages": {
  "type": "include",
  "include": ["securityContext"]
}

Restrict

Restricting a stage prevents modification of critical stages. This is essential to protect sensitive or compliance-relevant configurations, such as permissions or bucket assignment.

It ensures that member pipelines can't override or redefine a stage as restricted stages are unavailable to member pipelines. If the composition pipeline contains the stage, records are processed only by the composition stage.

Example: An administrator restricts the Permissions stage for all pipelines in the group. Teams can't modify dt.security_context, which ensures consistent access control across the organization.

"memberStages": {
  "type": "exclude",
  "exclude": ["securityContext"]
}

Settings API

Pipeline groups are available exclusively through the Settings API.

Changes to pipeline schemas

Pipeline groups introduce two new parameters in your existing pipeline schemas:

  • groupRole: Identifies whether a pipeline can be a memberPipeline or a compositionPipeline. This field doesn't indicate whether the pipeline is currently part of a group. The group role affects routing.

  • routing: Specifies if the pipeline supports routing. Composition pipelines are typically notRoutable, except for standalone ready‑made pipelines from extensions. The following table summarizes which pipelines support routing, depending on their group role and pipeline type.

    Pipeline typeGroup roleIs in a pipeline groupSupports routing
    CustomMemberNoYes
    Ready-madeCompositionNoYes
    CustomMemberYesYes
    CustomCompositionYesNo
    Ready-madeCompositionYesNo

Important parameters in pipeline group schemas

Pipeline group schemas include the following parameters:

  • isPipelinePlaceholder
    • true: Identifies the execution of the member pipeline.
    • false: Identifies the execution as a composition pipeline.
  • The type parameter (memberStages) referes to which member stages are executed. It supports the follwoing values:
    • include: Executes specified member stages.
    • exclude: Doesn't execute specified member stages.
    • includeAll: Executes all member stages, without specifing them.

Use cases

  • Enforce global security, routing, and operational standards across multiple pipelines.
  • Provide development teams with autonomy while preventing misconfiguration.
  • Centralize governance for large organizations with many pipelines.
  • Ensure consistent execution order between global and team-specific stages.
Related tags
Dynatrace Platform