Migrate OpenPipeline configurations to Settings API

  • Latest Dynatrace
  • How-to guide
  • 3-min read

OpenPipeline configuration with Settings API facilitates configuration management on a larger scale. This is achieved through a fine-grained approach to permissions. It introduces a hierarchical split of configurations for enterprises with many teams, giving teams flexibility in ingesting and processing data according to their needs while maintaining adaptable control by monitoring and SRE teams.

This migration is a permanent change and cannot be undone. It introduces breaking changes to your policies and API integrations.

Why migrate

With configuration via Settings API, you get:

  • Separate objects for routing, individual pipelines, and individual ingest sources of each configuration scope.

    Instead of a single, complex, and large JSON file, Dynatrace Configuration as Code users can now focus on files specific to the object and the configuration scope, and distributed them in their repositories.

  • Fine-grained permissions for routing, pipelines, and ingest sources of each configuration scope.

    Administrators can create separate policies for routing, pipelines, and ingest souces of each configuration scope, providing flexibility for teams responsible for data ingestion and processing, as well as restricted routing management.

  • Owner-based access control for individual pipelines and ingest sources.

    Users can create, delete, and share access to pipelines and ingest sources they own.

What's in scope of the migration?

The OpenPipeline Configurations API is replaced by the Settings API and the schemas for each configuration scope

  • Routing (builtin:openpipeline.<configuration.scope>.routing)
  • Pipelines (builtin:openpipeline.<configuration.scope>.pipelines)
  • Ingest sources (builtin:openpipeline.<configuration.scope>.ingest-sources)

The content of the top level keys remains unchanged, except for David events, which are now extracted in a separate stage. The migration affects endpoint URLs, query parameters, and the response/request body parameters structure, as well as the scope of the token for the request.

Base URL

new Settings 2.0old OpenPipeline Configurations API
/api/v2/settings/api/platform/openpipeline/v1/configurations

Authentication token scope

new Settings 2.0old OpenPipeline Configurations API
Read settings (settings.read)
Write settings (settings.write)		Read configuration (openpipeline:configurations:read)
Write configuration (openpipeline:configurations:write)

Parameters

In the Settings 2.0 framework, routing for each configuration scope, individual pipelines, and individual ingest sources of your OpenPipeline configuration are represented by a settings object. An object contains some metadata (like the scope or creation timestamp) and its configuration, encapsulated in the value object. To view the parameters of routing, pipelines, and ingest sources, you can query the related schema with a GET request. To learn about new query/body parameters, see the documentation of individual requests.

The new Settings API has dedicated permissions for routing, pipelines, and ingest sources. They substitute the previous permissions as described in the table below.

new Settings 2.0old OpenPipeline Configurations API
settings:objects:readopenpipeline:configurations:read
settings:objects:writeopenpipeline:configurations:write

Pipelines and ingest source schemas have object owners and accessors.

Owner

The first user to store the object. Owners have read and write access to the objects they create and can delete it or share it with other users. It can be set to all users or a single user.

Accessor

One or multiple users who can access an object shared by an owner. It can be set to all users, a single user, or multiple users.

Therefore a pipeline or ingest source can be accessed by a user that is one of the following:

  • Administrator
  • The owner of the object
  • An accessor of the object with sufficient permissions (settings:objects:read or settings:objects:write)

Administrators with settings:objects:admin permissions have unlimited read and write access to all objects. Administrators can also share access to objects that have an owner and change the owner.

What's not in scope of the migration

The migration won't affect

  • Other OpenPipeline API endpoints and their permissions
    • The OpenPipeline Preview API, Matcher API, Processor API, and Technology API remain unchanged.
    • The existing permissions (openpipeline:configurations:read) remain valid for the above endpoints.
  • OpenPipeline Ingest API

Before you migrate

Warning Breaking change

  • You must update policies and integrations based on the OpenPipeline Configurations API.
  • If you are using Dynatrace Configuration as Code, you must download the new configuration before or immediately after the migration.

We recommend that you back up your existing configuration before starting the migration.

Prerequisites

  • When you go to Settings Settings > Process and contextualize > OpenPipeline and select any configuration scope, you see the migration banner.

    Ready to migrate

    This migration is rolled out gradually until September 25, 2025. If you don't see the migration banner, your environment might have already been migrated or will be migrated soon. If your environment has already been migrated, ingest sources and pipelines have a new Owners column. If you don't see either, check again in the upcoming days.

  • You have the following permissions: openpipeline:configurations:write and settings:objects:admin.

  • For Terraform users, make sure to use Dynatrace Terraform version 1.85+.

How to migrate?

By default, after the migration all users have settings:objects:read (read access) and become owners of all pipelines and ingest sources. However, to enable owners to modify existing pipelines or users to create new pipeline, you must grant settings:objects:write (write access).

  1. Go to Account Management > Identity and Access Management

  2. Create new policies to grant settings:objects:write (write access) to specific users.

    • Example 1: Grant write access to a single object of a configuration scope

      The following policy grants write access only to the pipeline object for the logs configuration scope. Users who are listed in the policy can modify all log pipelines.

      ALLOW settings:object:write WHERE settings:schemaId = "builtin:openpipeline.logs.pipelines"

    • Example 2: Grant write access to multiple objects of multiple configuration scopes

      The following policy grants write access to the pipeline and ingest source objects for both the user event and span configuration scopes. Users who are listed in the policy can modify all user event and span ingest sources and pipelines.

      ALLOW settings:object:write WHERE settings:schemaId IN ("builtin:openpipeline.user.events.pipelines", "builtin:openpipeline.user.events.ingest-sources", "builtin:openpipeline.spans.pipelines", "builtin:openpipeline.spans.ingest-sources")

    • Example 3: Grant write access to all objects of a single configuration scope

      The following policy grants write access to the routing, pipeline, and ingest source objects for business event configuration scopes. Users who are listed in the policy can modify event routing and all ingest sources and pipelines.

      ALLOW settings:object:write WHERE settings:schemaId startsWith "builtin:openpipeline.bizevents"

Review the integrations that use the OpenPipeline Configurations API endpoints, such as API calls, scripts, configurations-as-code files, or third-party integrations. Make sure to use the new Settings API schemas instead.

To migrate your OpenPipeline configuration to the new API

  1. Go to Settings Settings > Process and contextualize > OpenPipeline and select any configuration scope.

    This process will migrate all configuration scopes, regardless of which one you select.

  2. In the upper part of the configuration scope page, select Migrate in the dedicated banner.

    The migration process will start automatically. It will take a few minutes to complete.

    When them migration completes successfully, a Migration successful message appears. Configurations for all configuration scopes have been migrated to the new API.

    Migration success

    The following endpoints can no longer be used:

    • GET /platform/openpipeline/v1/configurations

    • GET /platform/openpipeline/v1/configurations/{id}

    • PUT /platform/openpipeline/v1/configurations/{id}

If you are not using configuration as code, your migration is completed.

Dynatrace has migrated the OpenPipeline Configurations API to the Settings API schema automatically for you, therefore you don't need to manually look into the configuration files.

To complete the migration,

  1. Download and create new configuration-as-code files containing the new configuration.

  2. Modify all necessary configuration files containing the old schema ID with the new schema ID.

  3. Delete the old configurations files.

What to do after the migration?

  • Set up Owner-based access control to start handing-over the configuration reponsibility for pipelines to the individual teams who own the data.

  • Preview You can enroll in the Preview program with higher configuration limits, up to 2,000 pipelines. To enroll in the program, contact a Dynatrace product expert via live chat.