Langchain

LangChain is a framework designed for building applications that utilize large language models (LLMs). It provides a set of tools and components that make it easier to integrate LLMs into various applications, enabling developers to create complex workflows and functionalities.

With Dynatrace, you can get visibility into each step of the workflows monitoring prompt and completion messages, error tracking, performance metrics, and more.

Configuration

Follow Ingest OpenTelemetry metrics to see how OpenTelemetry metrics are mapped to Dynatrace metric types.

Create a Dynatrace token

To create a Dynatrace token

In Dynatrace, go to Access Tokens.
To find Access Tokens, press Ctrl/Cmd+K to search for and select Access Tokens.
In Access Tokens, select Generate new token.
Enter a Token name for your new token.
Give your new token the following permissions:
Search for and select all of the following scopes.
- Ingest metrics (metrics.ingest)
- Ingest logs (logs.ingest)
- Ingest events (events.ingest)
- Ingest OpenTelemetry traces (openTelemetryTrace.ingest)
- Read metrics (metrics.read)
- Write settings (settings.write)
Select Generate token.
Copy the generated token to the clipboard. Store the token in a password manager for future use.
You can only access your token once upon creation. You can't reveal it afterward.

Instrumentation

The Dynatrace backend exclusively works with delta values and requires the respective aggregation temporality. Make sure your metrics exporter is configured accordingly, or set the OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE environment variable to DELTA.

We can leverage OpenTelemetry to provide autoinstrumentation that collects traces and metrics of your AI workloads, particularly OpenLLMetry that can be installed with the following command:

pip install traceloop-sdk

Afterward, add the following code at the beggining of your main file.

from traceloop.sdk import Traceloop

headers = { "Authorization": "Api-Token <YOUR_DT_API_TOKEN>" }
Traceloop.init(
    app_name="<your-service>",
    api_endpoint="https://<YOUR_ENV>.live.dynatrace.com/api/v2/otlp",
    headers=headers
)

We can leverage OpenTelemetry to provide autoinstrumentation that collects traces and metrics of your AI workloads, particularly OpenLLMetry that can be installed with the following command:

npm i @opentelemetry/exporter-trace-otlp-proto

Afterward, add the following code at the beggining of your main file.

import {OTLPTraceExporter} from "@opentelemetry/exporter-trace-otlp-proto";
import * as traceloop from "@traceloop/node-server-sdk";

const exporter = new OTLPTraceExporter({
    url: "https://<YOUR_ENV>.live.dynatrace.com/api/v2/otlp",
    headers: { Authorization: "Api-Token <YOUR_DT_API_TOKEN>" },
});

traceloop.initialize({
  appName: "<your-service>",
  exporter: exporter
});

Currently, OpenLLMetry for Node.js doesn't support Metrics.

Spans

The following attributes are available for GenAI Spans.

Attribute

Type

Description

gen_ai.completion.0.content

string

The full response received from the GenAI model.

gen_ai.completion.0.role

string

The role used by the GenAI model.

gen_ai.prompt.0.content

string

The full prompt sent to the GenAI model.

gen_ai.prompt.0.role

string

The role setting for the GenAI request.

gen_ai.request.model

string

The name of the GenAI model a request is being made to.

gen_ai.response.model

string

The name of the model that generated the response.

gen_ai.system

string

The GenAI product as identified by the client or server instrumentation.

llm.request.type

string

The type of the operation being performed.

traceloop.entity.name

string

The name of the action in the chain.

traceloop.span.kind

string

The type of action in the chain.

traceloop.workflow.name

string

The name of the chain.

Metrics

Metric

Type

Unit

Description

http.client.duration

histogram

ms

The duration of the outbound HTTP request.

Langchain

Configuration

Create a Dynatrace token

Instrumentation

Spans

Metrics

Related topics