Try it free

Traces version 1.325

  • Latest Dynatrace
  • Reference
  • Published Sep 22, 2025

Distributed traces are used to capture transactions flowing through a system. Traces are made of spans, which represent the units of work within a distributed trace.

CICS Transaction Gateway Spans

Semantic conventions for CTG request and response spans captured on a CTG server or CTG client. The span.kind is client for spans captured on a CTG client and is server for spans captured on a CTG server. This applies to both the request and the response.

CTG supports different request types, such as ECI, ESI, or EPI. The call type and response codes have different semantics depending on the request type. Furthermore, some fields are only populated for some request types.

Request

Request spans have attributes conforming to the following table:

AttributeTypeDescriptionExamples

cics.transaction.user_id

string

experimentalThe user ID of the user who triggered this transaction.

USER1; anon

ctg.request.call_type

long

experimentalInteger representing the call type of the CTG GatewayRequest. The set of possible values varies per request type. 1

2

ctg.request.commarea_length

long

experimentalLength of the COMMAREA. Only set when the request type is ECI.

0

ctg.request.extend_mode

long

experimentalInteger representing the extended mode of the CTG GatewayRequest. Only set when the request type is ECI. 2

11

ctg.request.flow_type

long

experimentalInteger representing the flow type of the CTG GatewayRequest. 3

5

ctg.request.gateway_url

string

experimentalURL of the gateway. Only set on client-side spans.

tcp://1.2.3.4:5678/

ctg.request.object_name

string

experimentalName of the request object. Only set when the request type is ADMIN.

ctg.request.server_id

string

experimentalID of the server. Not set for all request types.

IPICTEST

ctg.request.term_id

string

experimentalName of the terminal resource. Only set when the request type is EPI.

CN02

ctg.request.type

string

experimentalType of the CTG GatewayRequest.

BASE

ibm.cics.program

string

resource experimentalThe name of the CICS program.

EDUCHAN

network.transport

string

stableOSI Transport Layer or Inter-process Communication method

tcp; udp

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known.

example.com

server.port

long

stableLogical server port number.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

zos.transaction.id

string

experimentalThe ID of this transaction.

CEMT; DTAX; IVTNO

1

https://www.ibm.com/docs/api/v1/content/SSZHFX_9.3.0/basejavadoc/constant-values.html

2

https://www.ibm.com/docs/api/v1/content/SSZHFX_9.3.0/basejavadoc/constant-values.html

3

The values are defined in the IBM CTG API source code.

ctg.request.type MUST be one of the following:

ValueDescription

ADMIN

Admin request.

AUTH

Authentication request.

BASE

Base. A base GatewayRequest without a further subtype. 1

ECI

External Call Interface. Enables a client application to call a CICS program synchronously or asynchronously. 2

EPI

External Presentation Interface. Enables a user application to install a virtual IBM 3270 terminal into a CICS server. 3

ESI

External Security Interface. Enables user applications to perform security-related tasks. 4

XA

CICS Request Exit. It can be used for request retry, dynamic server selection, and rejecting non-valid requests. 5

1

https://www.ibm.com/docs/en/cics-tg-multi/8.1?topic=classes-gatewayrequest

2

https://www.ibm.com/docs/en/cics-tg-zos/9.1.0?topic=applications-external-call-interface-eci

3

https://www.ibm.com/docs/en/cics-tg-multi/8.1?topic=guide-external-presentation-interface-epi

4

https://www.ibm.com/docs/en/cics-tg-zos/9.3.0?topic=applications-external-security-interface-esi

5

https://www.ibm.com/docs/en/cics-tg-zos/9.3.0?topic=applications-creating-cics-request-exit

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

inproc

In-process communication. 1

other

Something else (non-IP-based).

pipe

Named or anonymous pipe.

tcp

TCP

udp

UDP

unix

Unix domain socket.

1

Signals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.

Response

Response spans have attributes conforming to the following table:

AttributeTypeDescriptionExamples

ctg.request.type

string

experimentalType of the CTG GatewayRequest.

BASE

ctg.response.code

long

experimentalCTG response code. The set of possible values varies per request type. 1

-23

1

https://www.ibm.com/docs/api/v1/content/SSZHFX_9.3.0/basejavadoc/constant-values.html

ctg.request.type MUST be one of the following:

ValueDescription

ADMIN

Admin request.

AUTH

Authentication request.

BASE

Base. A base GatewayRequest without a further subtype. 1

ECI

External Call Interface. Enables a client application to call a CICS program synchronously or asynchronously. 2

EPI

External Presentation Interface. Enables a user application to install a virtual IBM 3270 terminal into a CICS server. 3

ESI

External Security Interface. Enables user applications to perform security-related tasks. 4

XA

CICS Request Exit. It can be used for request retry, dynamic server selection, and rejecting non-valid requests. 5

1

https://www.ibm.com/docs/en/cics-tg-multi/8.1?topic=classes-gatewayrequest

2

https://www.ibm.com/docs/en/cics-tg-zos/9.1.0?topic=applications-external-call-interface-eci

3

https://www.ibm.com/docs/en/cics-tg-multi/8.1?topic=guide-external-presentation-interface-epi

4

https://www.ibm.com/docs/en/cics-tg-zos/9.3.0?topic=applications-external-security-interface-esi

5

https://www.ibm.com/docs/en/cics-tg-zos/9.3.0?topic=applications-creating-cics-request-exit

Custom Services

Custom service spans represent entry points into a service or entry points to specific components within a larger application. OneAgent Code Modules can create custom service spans via automatic instrumentation rules or directly by OneAgent SDK APIs.

AttributeTypeDescriptionExamples

custom_service.method

string

experimentalThe service method of a custom service. This field only exists if a custom service was created via Dynatrace OneAgent SDK.

startTask; run; authenticate

custom_service.name

string

experimentalThe name of a custom service. This field only exists if a custom service was created via Dynatrace OneAgent SDK.

MyCustomService; AuthenticationComponent

supportability.custom_service.rule_id

uid

experimentalThe ID of a custom service configuration rule. This field is only present if a custom service was configured as an automatic instrumentation rule in Dynatrace.

4d76194c11a9426197a9062548f9e66e

Database Client Spans

Semantic conventions for database client spans. The span.kind for database client spans is client.

A span representing a database client operation might include the operation itself and the following result processing (for example, fetch from SQL result set, MongoDB cursor operations, etc.). Besides, several similar db operations might be aggregated into a single span for efficiency.

The aggregation attributes provide information on how many db operations have been aggregated.

The db.result.* attributes represent the details of the result processing.

AttributeTypeDescriptionExamples

db.affected_item_count

long

experimentalThe number of items (rows, documents,…) affected.

32

db.collection.name

string

stableThe name of a collection (table, container) within the database.

customers; public.users

db.cosmosdb.request_charge

double

experimentalThe cost of the request in Azure Cosmos DB request units (RU).

4.95; 2.0

db.dynamodb.table_names

string[]

experimentalThe list of tables the request targets.

[Cats, Dogs]

db.namespace

string

stableThe name of the database, fully qualified within the server address and port.

customers; test.users

db.operation.name

string

stableThe name of the operation or command executed, for example the MongoDB command name, SQL keyword, Redis command name,… 1

drop; findAndModify; SELECT; PREPARE; GetItem; set; LPUSH; mutateIn; ReadItems

db.query.parameters

record[]

experimentalThe query parameters used in db.query.text represented as a key and value map. For database systems without named keys, the map key is the string representation of the index starting with 0. Several database requests may get aggregated into a single span. Each entry in the array holds the bind parameters for one database request.Tags: sensitive-spans

[{'name': 'paul', 'age': '23'}, {'name': 'mary', 'age': '32'}]; [{'0': 'paul', '1': '23'}, {'0': 'mary', '1': '32'}]

db.query.text

string

stableThe database query being executed. 2

SELECT * FROM wuser_table; SET mykey "WuValue"

db.result.duration_max

duration

experimentalThe maximum duration in nanoseconds used for fetching the result.

345

db.result.duration_min

duration

experimentalThe minimum duration in nanoseconds used for fetching the result.

123

db.result.duration_sum

duration

experimentalThe total duration in nanoseconds used for fetching the result.

234

db.result.exception_count

long

experimentalThe number of exceptions encountered while fetching the result.

2

db.result.execution_count

long

experimentalThe number of operations executed on the result (for example, fetches from SQL result set, MongoDB cursor operations).

12

db.result.roundtrip_count

long

experimentalThe number of round-trips triggered by fetching the result.

2

db.system

string

experimentalAn identifier for the database management system (DBMS) product being used. See below for a list of well-known identifiers.

mongodb; mysql

network.transport

string

stableOSI Transport Layer or Inter-process Communication method

tcp; udp

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known.

example.com

server.port

long

stableLogical server port number.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

1

Depending on the data provided on ingest, this attribute may be derived by e.g., parsing db.query.text. Parsing might fail, or the result might be inaccurate.

2

The value may be sanitized to exclude sensitive information.

db.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

adabas

Adabas (Adaptable Database System)

cache

InterSystems Caché

cassandra

Apache Cassandra

clickhouse

ClickHouse

cloudscape

Cloudscape

cockroachdb

CockroachDB

coldfusion

ColdFusion IMQ

cosmosdb

Microsoft Azure Cosmos DB

couchbase

Couchbase

couchdb

CouchDB

db2

IBM Db2

derby

Apache Derby

dl/i

IBM DL/I

dynamodb

Amazon DynamoDB

edb

EnterpriseDB

elasticsearch

Elasticsearch

filemaker

FileMaker

firebird

Firebird

firstsql

FirstSQL

geode

Apache Geode

h2

H2

hanadb

SAP HANA

hbase

Apache HBase

hive

Apache Hive

hsqldb

HyperSQL DataBase

informix

Informix

ingres

Ingres

instantdb

InstantDB

interbase

InterBase

mariadb

MariaDB

maxdb

SAP MaxDB

memcached

Memcached

mongodb

MongoDB

mssql

Microsoft SQL Server

mssqlcompact

Microsoft SQL Server Compact

mysql

MySQL

neo4j

Neo4j

netezza

Netezza

opensearch

OpenSearch

oracle

Oracle Database

other_sql

Some other SQL database. Fallback only. See notes.

pervasive

Pervasive PSQL

pointbase

PointBase

postgresql

PostgreSQL

progress

Progress Database

redis

Redis

redshift

Amazon Redshift

spanner

Cloud Spanner

sqlite

SQLite

sybase

Sybase

teradata

Teradata

vertica

Vertica

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

inproc

In-process communication. 1

other

Something else (non-IP-based).

pipe

Named or anonymous pipe.

tcp

TCP

udp

UDP

unix

Unix domain socket.

1

Signals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.

Fields for dli database

AttributeTypeDescriptionExamples

db.dli.pcb

string

experimentalThe name of the program communication block associated with this DL/I method.

3; MYPCBNAM

db.dli.segment_name

string

experimentalThe name of the last segment that was matched or returned.

PARTROOT

db.dli.segment_level

string

experimentalThe hierarchical level of the segment that was matched or returned.

3; 24

db.dli.processing_options

string

experimentalThe PCB processing options.

GR

db.dli.terminal_name

string

experimentalThe DL/I database or logical terminal name associated with this DL/I method.

HWSAM5ZD; 10505

db.dli.status_code

string

experimentalThe DL/I status code.

QC

db.dli.pcb_type

string

experimentalThe PCB type.

DC; DL/I; F/P

db.dli.pcb_type MUST be one of the following:

ValueDescription

DC

Data communications.

DL/I

DL/I db.

F/P

Fast Path.

Dynatrace Span

The semantic conventions for the Dynatrace span and the fields the user can expect.

Hierarchical Attributes

The following hierarchical attributes are mandatory.

AttributeTypeDescriptionExamples

span.alternate_parent_id

uid

experimentalThe alternative span.id of this span's parent span. If a trace is monitored by more tracing systems (for example, OneAgent and OpenTelemetry), there might be two parent spans. If the two parent spans differ, span.parent_id holds the ID of the parent span originating from same tenant of the span while span.alternate_parent_id holds the other parent span ID. The span.alternate_parent_id is an 8-byte ID and hex-encoded if shown as a string.

f76281848bd8288c

span.id

uid

stableA unique identifier for a span within a trace. The span.id is an 8-byte ID and hex-encoded if shown as a string.

f76281848bd8288c

span.is_subroutine

boolean

experimentalIf set to true, it indicates that this span is a subroutine of its parent span. The spans represent functions running on the same thread on the same call stack.

span.kind

string

stableDistinguishes between spans generated in a particular context.

server

span.parent_id

uid

stableThe span.id of this span's parent span. The span.parent_id is an 8-byte ID and hex-encoded if shown as a string.

f76281848bd8288c

trace.id

uid

stableA unique identifier for a trace. The trace.id is a 16-byte ID and hex-encoded if shown as a string.

357bf70f3c617cb34584b31bd4616af8

Timing Attributes

Attributes start_time, end_time and duration are mandatory for all spans. The attributes in the span.timing namespace are optional and represent measurements provided by the OneAgent.

AttributeTypeDescriptionExamples

duration

duration

stableThe difference between start_time and end_time in nanoseconds.

42

end_time

timestamp

stableEnd time of a data point. Value is a UNIX Epoch time in nanoseconds and greater or equal to the start_time.

1649822520123123165

span.timing.cpu

duration

stableThe overall CPU time spent executing the span, including the CPU times of child spans that are running on the same thread on the same call stack.

span.timing.cpu_self

duration

stableThe CPU time spent exclusively on executing this span, not including the CPU times of any children.

start_time

timestamp

stableStart time of a data point. Value is a UNIX Epoch time in nanoseconds and less or equal to the end_time.

1649822520123123123

Aggregation Attributes

OneAgent might aggregate spans having the same parent span into a single one. The aggregated span contains attributes to indicated that aggregation happened and to allow to reconstruct details.

For aggregated spans the start_time holds the earliest start_time and end_time holds the latest end_time of all aggregated spans. Like for non aggreated spans duration is the difference between start_time and end_time which might differ from aggregation.duration_sum because aggregated spans were executed in parallel or there were gaps between the spans.

AttributeTypeDescriptionExamples

aggregation.count

long

stableThe number of spans aggregated into this span. Because this span represents multiple spans, the value is >1.

3

aggregation.duration_max

duration

stableThe duration in nanoseconds for the longest aggregated span.

482

aggregation.duration_min

duration

stableThe duration in nanoseconds for the shortest aggregated span.

42

aggregation.duration_samples

duration[]

stableArray of reservoir sampled span durations of the aggregated spans. The duration samples can be used to estimate a more accurate duration distribution of aggregated spans rather than the average value.

[42, 482, 301]

aggregation.duration_sum

duration

stableThe duration sum in nanoseconds for all aggregated spans.

123

aggregation.exception_count

long

stableThe number of aggregated spans that included an exception.

0; 6

aggregation.parallel_execution

boolean

stabletrue indicates that aggregated spans may have been executed in parallel. Therefore, start_time + duration_sum may exceed end_time.

Sampling Attributes

If the span does not represent a single span, it can have attributes to support extrapolation of its values.

AttributeTypeDescriptionExamples

sampling.threshold

long

experimentalThe sampling probability is encoded as sampling.threshold with respect to a 56-bit random integer rv. A span is sampled if rv >= sampling.threshold; the sampling threshold acts as a rejection threshold and can be interpreted as the number of spans discarded out of 2^56. The attribute is only available if the sampling.threshold is not 0, and therefore sampling happened. The relationship between sampling probability and threshold is sampling probability = (2^56-sampling.threshold) * 2^(-56). Hence, sampling.threshold=0 means 100% sampling probability (collect all data), sampling.threshold=2^55 corresponds to a sampling probability of 50%, sampling.threshold=2^54 corresponds to a sampling probability of 75%.

36028797018963968

supportability.alr_sampling_ratio

long

experimentalThe denominator of the sampling ratio of the Dynatrace cluster, the attribute is only set if Adaptive Load Redution (ALR) is active on the Dynatrace cluster. A numerator is not specified, as it's always 1. If, for example, the Dynatrace cluster samples with a probability of 1/8 (12,5%), the value of supportability.alr_sampling_ratio would be 8 and the numerator is 1.

8

supportability.atm_sampling_ratio

long

experimentalThe denominator of the sampling ratio of an Adaptive Traffic Management (ATM) aware sampler. The attribute is always present if an ATM-aware sampler is active (this applies, for example, to Dynatrace OneAgent). A numerator is not specified, as it is always 1. If, for example, Dynatrace OneAgent samples with a probability of 1/16 (6,25%), the value of supportability.atm_sampling_ratio would be 16 and the numerator is 1.

16

trace.capture.reasons

string[]

experimentalExplains why this trace was captured, multiple reasons can apply simultaneously. Note: 'atm' and 'fixed' are mutually exclusive sampling approaches, though 'fixed' may appear with other capture triggers. Values: 'atm' (Dynatrace's intelligent sampling automatically adjusted trace capture based on traffic volume and system load), 'fixed' (trace captured due to configured percentage rules - either global settings or specific endpoint rules), 'custom' (trace captured because of custom correlation headers propagated between services or systems), 'mainframe' (trace originated from or includes IBM mainframe/z/OS components), 'serverless' (trace captured from serverless functions like AWS Lambda, Azure Functions, or similar platforms), 'rum' (trace initiated by user interactions in web browsers or mobile apps monitored by Dynatrace RUM agents).

[['atm'], ['fixed'], ['fixed', 'custom'], ['fixed', 'rum']]

Currently sampling can happen two stages in the data processing. Independend where sampling happens the span has the sampling.threshold for calculation of the combined (effective) sample rate. Supportability attributes help the understand the sampling on the different stages.

  • OneAgent: If the OneAgent has enabled adaptive traffic management (ATM), the agent samples PurePaths and the attribute supportability.atm_sampling_ratio is added to all effected spans.
  • Dynatrace Cluster: If the Dynatrace cluster is overloaded, it starts adaptive load reduction (ALR) and samples PurePaths. The attribute supportability.alr_sampling_ratio is added to all effected spans.

If for example OneAgents samples with a probability of 25% the spans would contain the attributes sampling.threshold=54043195528445952 and supportability.atm_sampling_ratio=4.

Details about adaptive traffic management for distributed tracing can be found in the documentation.

Code Attributes

When a span logically represents the execution of a function, it will have code.* attributes, describing that function.

invoked.code.* attributes describe the function in which a span has been started, but not necessarily ended. It often represents the function that has been instrumented in order to start a span. These attributes are only populated, if they mismatch with code.*.

AttributeTypeDescriptionExamples

code.function

string

experimentalThe method or function name, or equivalent (usually the rightmost part of the code unit's name). Represents the name of the function that is represented by this span.

serveRequest

code.namespace

string

experimentalThe namespace within which code.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit.

com.example.MyHttpService

code.filepath

string

experimentalThe source code file name that identifies the code unit as uniquely as possible.

/usr/local/MyApplication/content_root/app/index.php

code.line.number

long

experimentalThe line number within the source code file.

1337

code.invoked.function

string

experimentalLike code.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.

invoke

code.invoked.namespace

string

experimentalLike code.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.

com.sun.xml.ws.server.InvokerTube$2

code.invoked.filepath

string

experimentalLike code.filepath, only it represents the file path of the function that was active when a span has been started. Typically, it is the function that has been instrumented. It should only be set if it differs from code.filepath.

/usr/local/MyApplication/content_root/app/index.php

code.call_stack

string

experimentalThe call stack of the code.function. The call stack starts with the code.function, and the stack frames are separated by a line feed.

com.example.SampleClass.doProcessing(SampleClass.java) com.example.SampleClass.doSomeWork(SampleClass.java) com.example.SampleClass.main(SampleClass.java)

Events

AttributeTypeDescriptionExamples

span.events

record[]

stableA collection of events. An event is an optional time-stamped annotation of the span and consists of a name and key-value pairs.

supportability.dropped_events_count

long

experimentalThe number of span events that were discarded on the source.

1

Span events have their own semantics defined here.

Exception Events

If the span was exited by an exception or contains other exception events, the following fields are available to provide a reference to the correct exception in the list of the span.events.

AttributeTypeDescriptionExamples

span.exit_by_exception_id

uid

stableThe exception.id of the exception the its span.events with the current span exited. The referenced exception has set the attribute exception.escaped to true.

span.is_exit_by_exception

boolean

stableSet to true if an exception exited the span. If set to false, the span has exception events, but none exited the span.

Links

AttributeTypeDescriptionExamples

span.links

record[]

stableA collection of links. A link is a reference from this span to a whole trace or a span in the same or different trace.

supportability.dropped_links_count

long

experimentalThe number of span links that were discarded on the source.

1

Span links have their own semantics defined here.

Failure Detection

Fields that can be expected for a failure detection on a Dynatrace span. Failure detection will be applied to spans that represent requests on endpoints and incoming Istio services mesh proxies. A request is considered failed if at least one failure reason is detected and no success forcing rule matches. The combined result (failure or success) will be stored in the attribute request.is_failed (see also Request). To modify failure detection behavior, modify its configuration.

AttributeTypeDescriptionExamples

dt.failure_detection.ruleset_id

uid

experimentalThe id of the failure detection rule set (failure detection v2) that was applied to that span (uid128).

4d76194c11a9426197a9062548f9e66e

dt.failure_detection.general_parameters_id

uid

experimentalThe id of the failure detection general parameters (failure detection v1) that were applied to that span (uid128).

4d76194c11a9426197a9062548f9e66f

dt.failure_detection.http_parameters_id

uid

experimentalThe id of the failure detection HTTP parameters (failure detection v1) that were applied to that span (uid128).

4d76194c11a9426197a9062548f9e66a

dt.failure_detection.global_rule_id

uid

experimentalThe id of the global failure detection rule (failure detection v1) that was applied to that span (uid128).This is always used in conjunction with the dt.failure_detection.global_parameters_id.

4d76194c11a9426197a9062548f9e66b

dt.failure_detection.global_parameters_id

uid

experimentalThe id of the global failure detection parameters (failure detection v1) that were applied to that span (uid128).This is always used in conjunction with the dt.failure_detection.global_rule_id.

4d76194c11a9426197a9062548f9e66c

dt.failure_detection.results

record[]

experimentalA collection of individual failure detection reasons and verdicts for each applied matching rule. If no entries exist, no rules matched, and the attribute does not exist.

Failure detection has its own semantics defined here.

Server and client attributes

These attributes may be used to describe the client and server in a connection-based network interaction where there is one side (the client) that initiates the connection. This covers all TCP network interactions since TCP is connection-based and one side initiates the connection (an exception is made for peer-to-peer communication over TCP where the "user-facing" surface of the protocol / API does not expose a clear notion of client and server). This also covers UDP network interactions where one side initiates the interaction, e.g. QUIC (HTTP/3) and DNS.

In an ideal situation, not accounting for proxies, multiple IP addresses or host names, the server.* attributes are the same on the client and server span.

Server attributes

AttributeTypeDescriptionExamples

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known.

example.com

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

server.port

long

stableLogical server port number.

65123; 80

server.address

For IP-based communication, the name should be a DNS host name of the service. On client side it matches remote service name, on server side, it represents local service name as seen externally on clients.

When connecting to a URL https://example.com/foo, server.address matches "example.com" on both client and server side.

On client side, it's usually passed in form of a URL, connection string, host name, etc. Sometimes host name is only available as a string which may contain DNS name or IP address.

If network.transport is pipe, the absolute path to the file representing it is used as server.address.

For Unix domain socket, server.address attribute represents the remote endpoint address on client side and local endpoint address on server side.

Client attributes

AttributeTypeDescriptionExamples

client.ip

ipAddress

experimentalThe IP address of the client that makes the request. This may be IPv4 or IPv6.Tags: sensitive-spans

194.232.104.141; 2a01:468:1000:9::140

client.port

long

stableClient port number.

65123; 80

client.isp

string

experimentalThe name of the Internet Service Provider (ISP) associated with the client's IP address.

Internet Service Provider Name

client.ip.is_public

boolean

experimentalIndicates whether IP is a public IP.

true

client.app.name

string

experimentalThe name of the client application used to perform the request.

MS Outlook

client.address

string

experimentalClient address - domain name if available without reverse DNS lookup; otherwise, IP address or Unix domain socket name.

client.example.com; 10.1.2.80; [local]

Supportability Attributes

Supportability attributes help to understand the characteristics of the span.

AttributeTypeDescriptionExamples

supportability.dropped_attributes_count

long

experimentalThe number of attributes that were discarded on the source. Attributes can be discarded because their keys are too long or because there are too many attributes.

1

supportability.non_persisted_attribute_keys

string[]

experimentalA string array of attribute keys that were not stored as they were not allow-listed or were removed during the pipeline steps.

["my_span_attribute", "db.name"]

trace.alternate_id

uid

experimentalThe preserved trace ID when OneAgent and other tracing systems monitor the same process and the trace ID from the other tracing system was replaced by the OneAgent trace ID. The trace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.

357bf70f3c617cb34584b31bd4616af8

trace.state

string

experimentalThe trace state in the w3c-trace-context format.

f4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=bar

Requests

Requests and Endpoints

Requests are an important hierarchical grouping of spans within a trace. A request spawns all spans from the detected request root span until the service is left again. The grouping of spans into requests allows for very powerful queries and advanced analysis. Every request always has a request root span which is special as it is the trigger for detecting endpoints and hence defining the service API.

AttributeTypeDescriptionExamples

dt.endpoint_detection.rule_id

uid

experimentalThe ID of the endpoint detection rule that was applied to that span.

4d76194c11a9426197a9062548f9e66e

endpoint.name

string

stableThe endpoint name is derived from endpoint detection rules and uniquely identifies one endpoint of a particular service. Endpoint names are usually technology-specific and should be defined by attributes with low cardinality, like http.route or rpc.method. Endpoints are exclusively detected on request root spans.

GET /; PUT /users/:userID?; GET /productpage; Reviews.GetReviews

request.is_failed

boolean

experimentalIndicates that the request is considered failed according to the failure detection rules. Only present on the request root span.

request.is_root_span

boolean

experimentalMarks the root of a request. It's the first span and starts the request within a service.

Request Attributes

Request attributes allow you to enrich spans collected by OneAgents with deep-insight data which is not captured on trace data by default. They are modelled as:

  • Captured attributes, which represent the raw value as reported by the OneAgent.
  • Request attributes, which represent the normalized value along a complete request

The names of request and captured attributes are composed of the prefixes "captured_attribute" and "request_attribute" and the name given in the configuration by the user:

AttributeTypeDescriptionExamples

captured_attribute.__attribute_name__

array

stableContains the span scoped raw values that were captured under the name __attribute_name__ defined by the request attribute configuration. The values are mapped as an array according to the type of the captured attributes, so either boolean, double, long, or string. If the captured attributes have mixed types (e.g. long and string, or double and long, etc.), all attributes are converted to string and stored as string array.

[42]; ['Platinum']; [32, 16, 8]; ['Special Offer', '1702']; ['18.35', '16']

request_attribute.__attribute_name__

array

stableContains the request scoped reconciled values of the attribute named __attribute_name__ defined by the request attribute configuration. The data type of the value depends on the request attribute definition.Tags: sensitive-spans

42; Platinum; ['Product A', 'Product B']; ['Special Offer', '1702']

Size of a span

The calculated sizes of a span in bytes. The dt.ingest.size is calculated when the span is ingested while the dt.retain.size is calculated before the span gets stored.

AttributeTypeDescriptionExamples

dt.ingest.size

long

stableThe size of the ingested data point in bytes.

2005

dt.retain.size

long

stableThe size of the retained data point in bytes.

2005

Status

A span contains a status consisting of a code and an optional descriptive message. The status is especially relevant if there is a known error in the application code, such as an exception in which case the span status can be set to error. The span status is only present if it is explicitly set to error or ok.

AttributeTypeDescriptionExamples

span.status_code

string

stableDefines the status of a span, predominantly used to indicate a processing error. This field is absent if the reported span status is unset.

error

span.status_message

string

experimentalAn optional text that can provide a descriptive error message in case the span.status_code is error.

Connection closed before message completed; Error sending request for url

Error status reasons

The following reasons cause the span.status_code to be error:

  • If the span was exited by an exception, i.e. the attribute span.is_exit_by_exception is set to true.
  • HTTP spans:
    • General: For http.response.status_code values in the 5xx range.
    • If span.kind is client: For http.response.status_code values in the 4xx range.
  • gRPC spans:
    • If span.kind is client: For all rpc.grpc.status_code values except OK (0).
    • If span.kind is server: For rpc.grpc.status_code values UNKNOWN (2), DEADLINE_EXCEEDED (4), UNIMPLEMENTED (12), INTERNAL (13), UNAVAILABLE (14), DATA_LOSS (15).

Additional Attributes

Beside the attributes listed above, arbitrary other attributes are allowed on a span.

Dynatrace Span Events

Semantic conventions for span events on Dynatrace spans.

Common

In general, a span event does not need to follow specific semantics, but typically, span events have the following common attributes.

AttributeTypeDescriptionExamples

span_event.name

string

stableSome span events have a defined semantics based on the name of the span event.

exception

supportability.dropped_attributes_count

long

experimentalThe number of attributes that were discarded on the source. Attributes can be discarded because their keys are too long or because there are too many attributes.

1

supportability.non_persisted_attribute_keys

string[]

experimentalA string array of attribute keys that were not stored as they were not allow-listed or were removed during the pipeline steps.

["my_span_attribute", "db.name"]

timestamp

timestamp

stableThe time (UNIX Epoch time in nanoseconds) when the event originated, typically when the source created it. If no original timestamp is available, it will be populated at ingest time and required for all events. In the case of a correlated event (for example, ITIL events), this time could be different from the event.start time, as this time represents the actual timestamp when the "update" for the event was created.

1649822520123123123

Besides the common attributes, any arbitrary attributes are allowed for span events.

Exception

If exceptions happen and are captured during a span, they're available as span events. Exceptions events have the span_event.name set to exception. Besides the following exception event-specific attributes, all semantics of the common section apply.

AttributeTypeDescriptionExamples

exception.caused_by_id

uid

stableThe exception.id of the exception the current exception was caused by.

exception.escaped

boolean

stabletrue indicates that the exception was recorded at a point where it is known that the exception escaped the scope of the span.

exception.id

uid

stableThe identifier of an exception. It should be unique within a list of exceptions of a span. The identifier is used to reference the exception.

exception.is_caused_by_root

boolean

stableIs set to true if the exception is the first exception caused by the chain.

exception.message

string

stableA message that describes the exception.

Division by zero

exception.stack_trace

string

experimentalThe stack trace of the exception. The format depends on the technology and source. While OneAgent formats stack traces to unify them across technologies, stack traces from an OpenTelemetry source are in the format they were sent to Dynatrace.

@https://www.foo.bar/path/main.js:59:26 e@https://www.foo.bar/path/lib/1.1/lib.js:2:30315

exception.type

string

stableThe type of the exception, for example its fully-qualified class name.

java.net.ConnectException; OSError

span_event.name

string

stableIs set to exception for exception events.

exception

Feature Flag Evaluation

A flag evaluation SHOULD be recorded as an event on the span during which it occurred. Feature flag events have the span_event.name set to feature_flag.evaluation.

AttributeTypeDescriptionExamples

feature_flag.context.id

string

experimentalThe unique identifier for the flag evaluation context. For example, the targeting key.

5157782b-2203-4c80-a857-dbbd5e7761db

feature_flag.key

string

experimentalThe unique identifier of the feature flag.

logo-color

feature_flag.provider.name

string

experimentalThe name of the service provider that performs the flag evaluation.

Flag Manager

feature_flag.result.reason

string

experimentalThe reason code, which shows how a feature flag value was determined.

static; targeting_match; error; default

feature_flag.result.variant

string

experimentalA semantic identifier for an evaluated flag value. 1

red; true; on

feature_flag.set.id

string

experimentalThe identifier of the flag set to that the feature flag belongs.

proj-1; ab98sgs; service1/dev

feature_flag.version

string

experimentalThe version of the ruleset used during the evaluation. This can be any stable value that uniquely identifies the ruleset.

1; 01ABCDEF

span_event.name

string

stableIs set to feature_flag.evaluation for feature flag events.

feature_flag.evaluation

1

A semantic identifier, commonly referred to as a variant, provides a means for referring to a value without including the value itself. This can provide additional context for understanding the meaning behind a value. For example, the variant red maybe be used for the value #c05543.

Business Events

If a span is related to a business event, the IDs of the business events are available as span events. The span events have the span_event.name set to bizevent.

AttributeTypeDescriptionExamples

event.id

string

stableUnique identifier string of an event; is stable across multiple refreshes and updates.

5547782627070661074_1647601320000

span_event.name

string

stableIs set to bizevent for bizevent events.

bizevent

Dynatrace Span Links

Semantic conventions for well known span links.

By Span and Trace Id

A span link by span.id and trace.id references to a target span on a different trace.

AttributeTypeDescriptionExamples

span.id

uid

stableA unique identifier for a span within a trace. The span.id is an 8-byte ID and hex-encoded if shown as a string.

f76281848bd8288c

supportability.dropped_attributes_count

long

experimentalThe number of attributes that were discarded on the source. Attributes can be discarded because their keys are too long or because there are too many attributes.

1

supportability.non_persisted_attribute_keys

string[]

experimentalA string array of attribute keys that were not stored as they were not allow-listed or were removed during the pipeline steps.

["my_span_attribute", "db.name"]

trace.alternate_id

uid

experimentalThe preserved trace ID when OneAgent and other tracing systems monitor the same process and the trace ID from the other tracing system was replaced by the OneAgent trace ID. The trace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.

357bf70f3c617cb34584b31bd4616af8

trace.id

uid

stableA unique identifier for a trace. The trace.id is a 16-byte ID and hex-encoded if shown as a string.

357bf70f3c617cb34584b31bd4616af8

trace.state

string

experimentalThe trace state in the w3c-trace-context format.

f4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=bar

Besides the attributes listed above, arbitrary other attributes are allowed on a generic span link.

By Custom Link Id

A span link by dt.tracing.custom_link.id references another span with a span link to the same dt.tracing.custom_link.id. This span link is used for technologies where full context propagation with span ID and trace ID is not possible, and only a limited amount of information can be propagated between tiers.

The dt.tracing.link.direction defines the hierarchy between two spans having span links with the same dt.tracing.custom_link.id.

AttributeTypeDescriptionExamples

dt.tracing.custom_link.id

uid

experimentalThe custom link ID to identify spans calling each other. The ID is derived from the custom link bytes.

736bd2684696c4a8

dt.tracing.custom_link.original_bytes

binary

experimentalThe original binary data of the custom link.

ycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==

dt.tracing.custom_link.transformed_bytes

binary

experimentalThe transformed binary data of the custom link. Only available if a mapping was applied.

ycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==

dt.tracing.custom_link.type

string

experimentalThe type of the custom link defines if a mapping of the dt.tracing.custom_link.original_bytes to the dt.tracing.custom_link.transformed_bytes was applied.

generic

dt.tracing.link.direction

string

experimentalThe direction of the span link to define the correct order between spans.

outgoing

dt.tracing.link.is_sync

boolean

experimentaltrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.

timestamp

timestamp

stableThe time (UNIX Epoch time in nanoseconds) when the span was propagated. Only available on span links with dt.tracing.link.direction set to outgoing.

1649822520123123123

By Dynatrace Link Id

A Dynatrace link provides additional details regarding the parent and child timing on top of span.parent_id. This data is optional and not provided by all span data sources.

AttributeTypeDescriptionExamples

dt.tracing.link.direction

string

experimentalThe direction of the span link to define the correct order between spans.

outgoing

dt.tracing.link.id

uid

experimentalUnique identifier for a Dynatrace link.

dt.tracing.link.is_sync

boolean

experimentaltrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.

timestamp

timestamp

stableThe time (UNIX Epoch time in nanoseconds) when the span was propagated. Only available on span links with dt.tracing.link.direction set to outgoing.

1649822520123123123

By Foreign Link

A span link by dt.tracing.foreign_link refers to an upstream transaction. This could be a cross-environment link or a cross-product link to a distributed trace in the legacy AppMon product. Depending on whether the link information was received in binary or text, either dt.tracing.foreign_link.bytes or dt.tracing.foreign_link.text will be set, respectively.

AttributeTypeDescriptionExamples

dt.tracing.foreign_link.bytes

binary

experimentalAn incoming foreign link (cross-environment or cross-product).

00000004000000010000000200000003000000040000002300000001

dt.tracing.foreign_link.text

string

experimentalAn incoming foreign link (cross-environment or cross-product).

FW4;129;12;-2023406815;4539717;0;17;66;c511;2h02;3h12345678;4h676767; FW1;129;4711;59959450;-1859959450;3;17;0

By Response Headers

A span link by dt.tracing.response.headers refers to a downstream transaction. This could be a cross-environment link.

AttributeTypeDescriptionExamples

dt.tracing.response.headers

record

experimentalA collection of key-value pairs containing received response headers related to tracing from an outgoing call. There may be multiple values for each header. Used for cross-environment linking.

{'traceresponse': ['00-7b9e3e4068167838398f50017bfad358-d4ffc7e33530967a-01'], 'x-dt-tracestate': ['9651e1a8-19506b7c@dt']}

ESB Spans

Semantic conventions for ESB (Enterprise Service Bus) spans. An ESB span holds information about the realm in which the span is produced. This metadata includes, for example, the workflow in which the span is placed and the application or library to which the workflow belongs.

Attributes

The workflow name is mandatory information and contributes the most to identifying where this span comes from. The application and library provide a high-level view of which deployment the span belongs to. It's also worth mentioning that application and library are typically seen as mutually exclusive, although this is not enforced in any way."

AttributeTypeDescriptionExamples

esb.application.name

string

resource experimentalThe name of the application that owns the current workflow.

myBusinessApp; YourServiceApp; any_work

esb.library.name

string

resource experimentalThe name of the library that owns the current workflow.

myWebServicesLib; YourMessagingLibrary; any_tools

esb.vendor

string

resource experimentalThe name of vendor of the ESB technology of the current workflow.

ibm; tibco

esb.workflow.is_subprocess

boolean

experimentalDefines whether the provided workflow is a subprocess or not.

false

esb.workflow.name

string

resource experimentalThe label of the current workflow.

myMessageFlow; YourBusinessWorkflow; any_flow

Failure Detection Result

Record containing fields that can be expected for a failure detection result. These results are part (as a record array) of a set of fields used by the failure detection feature and are referenced there via failure_detection.results. This record relates to failure detection v1 and v2.

  • Failure detection v1 will produce only one result in the array.
  • Failure detection v2 might produce multiple failure reasons. Each matching rule of a failure detection rule set will produce a single result.

Fields

AttributeTypeDescriptionExamples

custom_rule_name

string

experimentalName of the custom rule that caused this result. Uniquely identifies the rule within a failure detection v2 rule set. Related to the custom_rule reason.

Fail on my.failure.attribute==failed

exception_ids

uid

experimentalIDs of the exceptions that caused this rule result (more on exceptions).

[123423523456, 523463467234]

reason

string

experimentalAll possible failure detection reasons that caused a verdict.

exception

request_attribute_name

string

experimentalName of the request attribute that caused this result. Related to a custom error rule in a failure detection v1 configuration and the custom_error_rule reason.

my special method count RA

verdict

string

experimentalAll possible failure detection verdicts.

failure

reason MUST be one of the following:

ValueDescription

custom_error_rule

Verdict is caused by a custom error rule (request attribute). Applicable in failure detection v1.This reason always comes together with the request_attribute_name field.

custom_rule

Verdict is caused by a custom rule. Applicable in failure detection v2.This reason always comes together with the custom_rule_name field.

exception

Verdict is caused by an exception. Applicable in failure detection v1 and v2.

grpc_code

Verdict is caused by the GRPC response code. Applicable in failure detection v2.

http_code

Verdict is caused by the HTTP response code. Applicable in failure detection v1 and v2.

span_status

Verdict is caused by the span status attribute. Applicable in failure detection v1 and v2.

verdict MUST be one of the following:

ValueDescription

failure

Indicates that the rule failed.

success

Indicates that the rule was successful.

Function as a Service (FaaS)

Fields that can be expected from serverless functions or Function as a Service (FaaS) on various cloud platforms. There are general attributes and attributes specific to incoming FaaS invocations (server side) and outgoing invocations (FaaS function as client calling out).

General Attributes

AttributeTypeDescriptionExamples

azure.class_name

string

resource experimentalThe fully qualified name of the class executing an Azure function.

Host.Functions

azure.site_name

string

resource experimentalGlobally unique deployment information about an Azure function.

dt-function-scripted

cloud.account.id

string

resource stableThe cloud account ID to which the resource is assigned.

111111111111; opentelemetry

cloud.platform

string

resource stableThe cloud platform in use. 1

alibaba_cloud_ecs

cloud.provider

string

resource experimentalName of the cloud provider.

alibaba_cloud

cloud.region

string

resource stableIdentifier of the cloud vendor's data center geographic region.

us-east-1

cloud.resource_id

string

resource stableCloud provider-specific native identifier of the monitored cloud resource (for example, an ARN on AWS, a fully qualified resource ID on Azure, or a complete resource name on GCP).

arn:aws:lambda:REGION:ACCOUNT_ID:function:my-function; //run.googleapis.com/projects/PROJECT_ID/locations/LOCATION_ID/services/SERVICE_ID; /subscriptions/<SUBSCIPTION_GUID>/resourceGroups/<RG>/providers/Microsoft.Web/sites/<FUNCAPP>/functions/<FUNC>

faas.max_memory

long

resource experimentalThe amount of memory available to the serverless function in Bytes.

faas.name

string

resource experimentalThe name of the single function that this runtime instance executes. 2

my-function; myazurefunctionapp/some-function-name; test_function

faas.version

string

resource experimentalThe immutable version of the function being executed. 3

14; 254

1

The prefix of the service matches the one specified in cloud.provider.

2

This is the name of the function as configured/deployed on the FaaS platform and is usually different from the name of the callback function (which may be stored in the code.namespace/code.function span attributes).

3

Value of the field depends on a cloud provider. This field is not set for Azure.

Incoming Invocations

This section describes incoming FaaS invocations as they are reported by the FaaS instance itself. For incoming FaaS spans, the span.kind is either server or consumer.

AttributeTypeDescriptionExamples

aws.request_id

string

experimentalThe AWS request ID (e.g., value of x-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).

0e7bc729-a468-57e8-8143-98f2eec5c925

aws.xray.trace_id

string

experimentalContains the AWS X-Ray trace id (e.g., value of the x-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)

Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678

faas.coldstart

boolean

experimentalA boolean that is true if the serverless function is executed for the first time (aka cold-start).

faas.document.collection

string

experimentalThe table/collection name on which the operation above was executed. 1

my-coll-name

faas.document.name

string

experimentalThe identifier for the specific item that changed after executing the operation above. 2

my-file.jpg; 63eeb6e7d418cd98afb1c1d7

faas.document.operation

string

experimentalRelevant only for "datasource" trigger. The operation type which triggered the function invocation.

delete

faas.document.time

string

experimentalThe UTC ISO-8601 timestamp of the operation above. 3

2020-03-08T00:30:12.456Z

faas.event.__key__

string

stableFaas event attribute, the __key__ attribute in a Faas event represents the precise attribute name as received in the event. For example, it might be "faas.event.StackId" for the "StackId" attribute in an AWS CloudFormation event or "faas.event.IdentityPoolId" for the "IdentityPoolId" attribute in an AWS Cognito event. The value of this attribute is identical to the value received in the event.

arn:aws:cloudformation:us-west-2:123456789012:stack/MyStack/1a2b3c4d-5678-90ab-cdef-EXAMPLE11111; eu-west-1:12345678-1234-1234-1234-123456789012

faas.event_name

string

experimentalThe API action that triggered the faas event. 4

ObjectCreated:Put (aws:s3); INSERT (aws:dynamodb)

faas.event_source

string

experimentalThe cloud service that originated the event.

aws:cloudwatch; aws:cloudformation

faas.trigger

string

experimentalType of the trigger which caused this function invocation.

datasource

1

Relevant only for faas.trigger=datasource trigger

2

Relevant only for faas.trigger=datasource trigger

3

Relevant only for faas.trigger=datasource trigger

4

The value of this attribute is specific to the service that generated the event.

Outgoing Invocations

This section describes outgoing FaaS invocations as they are reported by a client calling a FaaS instance. For outgoing FaaS spans, the span.kind is either client or producer.

AttributeTypeDescriptionExamples

aws.request_id

string

experimentalThe AWS request ID (e.g., value of x-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).

0e7bc729-a468-57e8-8143-98f2eec5c925

aws.xray.trace_id

string

experimentalContains the AWS X-Ray trace id (e.g., value of the x-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)

Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678

faas.invoked_name

string

experimentalThe name of the invoked function.

my-function

faas.invoked_provider

string

experimentalThe cloud provider of the invoked function. Will be equal to the invoked function's cloud.provider resource attribute.

alibaba_cloud

faas.invoked_region

string

experimentalThe cloud region of the invoked function. 1

eu-central-1

1

Will be equal to the invoked function's cloud.region resource attribute.

Generative AI (GenAI) Spans

Semantic convention for spans related with GenAI applications.

AttributeTypeDescriptionExamples

gen_ai.guardrail.id

string

experimentalIdentifier of the guardrail that has been activated for the request.

sensitive_data_guardrail

gen_ai.guardrail.version

string

experimentalVersion of the guardrail that has been activated.

DRAFT; 5; 12345678

gen_ai.operation.name

string

experimentalName of operation being performed.

chat; generate_content; text_completion

gen_ai.prompt_caching

string

experimentalIndicates how prompt cache has been used when handling the request.

read; write

gen_ai.provider.name

string

experimentalName of GenAI product being used.

aws_bedrock; openai

gen_ai.request.frequency_penalty

double

experimentalFrequency penalty setting for GenAI request.

0.4

gen_ai.request.max_tokens

long

experimentalMaximum number of tokens that the model can generate for a request.

50

gen_ai.request.model

string

experimentalModel chosen to handle the request.

amazon.nova-micro-v1:0; anthropic.claude-3-7-sonnet-20250219-v1:0

gen_ai.request.presence_penalty

double

experimentalPresence penalty setting for GenAI request.

0.4

gen_ai.request.stop_sequences

string[]

experimentalList of sequences that will stop the model from generating further tokens.

[forest, lived]

gen_ai.request.temperature

double

experimentalTemperature setting for GenAI request.

0.8

gen_ai.request.top_k

long

experimentalTemperature setting for GenAI request.

300

gen_ai.request.top_p

double

experimentalTemperature setting for GenAI request.

0.6

gen_ai.response.finish_reasons

string[]

experimentalList of reasons why the model stopped generating tokens, corresponding to each generation received.

[["stop_sequence"], ["stop_sequence", "max_tokens"]]

gen_ai.response.model

string

experimentalModel that handled the request.

amazon.nova-micro-v1:0; anthropic.claude-3-7-sonnet-20250219-v1:0

gen_ai.usage.input_tokens

long

experimentalNumber of tokens sent to the model in the request.

42

gen_ai.usage.output_tokens

long

experimentalNumber of tokens generated by the model while handling the request.

42

gen_ai.prompt_caching has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

read

Cache hit. Reading from cache.

write

Cache miss. Creating cache checkpoint.

HTTP Spans

Semantic conventions for HTTP client and server spans. They can be used for HTTP and HTTPS schemes and various HTTP versions like 1.1, 2, and SPDY.

Common fields

The common fields listed in this section apply to both HTTP clients and servers in addition to the specific fields listed in the HTTP client and HTTP server sections below.

AttributeTypeDescriptionExamples

http.request.body.size

long

stableThe size of the request payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the Content-Length header. For requests using transport encoding, this should be the compressed size.

3495

http.request.header.__key__

string

stableHTTP request headers, __key__ being the lowercase HTTP header name, for example "http.request.header.accept-encoding". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.Tags: sensitive-spans

https://www.foo.bar/; gzip, deflate, br; 1.2.3.4, 1.2.3.5

http.request.method

string

stableHTTP request method.

GET; POST; HEAD

http.response.body.size

long

stableThe size of the response payload body in bytes. This is the number of bytes transferred excluding headers and is often, but not always, present as the Content-Length header. For requests using transport encoding, this should be the compressed size.

3495

http.response.header.__key__

string

stableHTTP response headers, __key__ being the lowercase HTTP header name, for example, "http.response.header.content-type". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.

909; text/html; charset=utf-8; abc, def

http.response.status_code

long

stableHTTP response status code.

200

network.protocol.name

string

stableOSI Application Layer or non-OSI equivalent.

amqp; http; mqtt

network.protocol.version

string

experimentalVersion of the application layer protocol used.

1.1; 3.1.1

HTTP client

This span type represents an outbound HTTP request.

For an HTTP client span, span.kind is client.

AttributeTypeDescriptionExamples

server.address

string

stableHost identifier of the "URI origin" HTTP request is sent to.

example.com

server.port

long

stablePort identifier of the "URI origin" HTTP request is sent to.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

url.full

string

stableAbsolute URL describing a network resource according to RFC3986.Tags: sensitive-spans

https://www.foo.bar/docs/search?q=OpenTelemetry#SemConv

HTTP server

This span type represents an inbound HTTP request.

For an HTTP server span, span.kind MUST be server.

AttributeTypeDescriptionExamples

client.ip

ipAddress

experimentalIP address of the original client (IPv4 or IPv6) making the request. This request might have passed several proxies or load balancers. The client IP is the result of resolving the socket connection, X-Forward-For, and other headers.Tags: sensitive-spans

194.232.104.141; 2a01:468:1000:9::140

http.route

string

stableThe matched route (path template in the format used by the respective server framework).

/users/:userID?; Home/Index/{id?}

server.address

string

stableName of the local HTTP server that received the request.

example.com

server.port

long

stableLogical server port number.

65123; 80

url.path

string

stableThe URI path component.

/docs/search

url.query

string

stableThe URI query component.Tags: sensitive-spans

q=OpenTelemetry

url.scheme

string

stableThe URI scheme component identifying the used protocol.

https; ftp; telnet

Messaging Spans

Common attributes

AttributeTypeDescriptionExamples

messaging.batch.failed_count

long

experimentalThe number of messages in the batch for which publishing failed.

1; 3; 15

messaging.batch.failure_codes

string[]

experimentalThe vendor-provided error codes explaining why an operation on the message broker failed. To limit attribute size, not all error codes might be included.

[['MalformedDetail', 'InvalidArgument']]

messaging.batch.message_count

long

experimentalThe number of messages sent, received, or processed in the scope of the batching operation.

1; 2; 3

messaging.client.id

string

experimentalA unique identifier for the client that consumes or produces a message.

aclient; myhost@68d46b89c9-c29qc

messaging.is_failed

boolean

experimentalIndicates that the messaging operation is considered failed according to the failure detection rules. Only present if the messaging.operation.type is process.

messaging.message.body.size

long

experimentalThe (uncompressed) size of the message payload in bytes.

2738

messaging.message.conversation_id

string

experimentalThe conversation ID identifying the conversation to which the message belongs, represented as a string. Sometimes called "Correlation ID".

MyConversationId

messaging.message.header.__key__

record

experimentalThe message headers, __key__ being the message header/attribute name, for example, "messaging.message.header.ExtendedPayloadSize". The data type of the value depends on the attribute.

1024, "my-eu-bucket-3", ["a", "b"]

messaging.message.id

string

experimentalA value used by the messaging system as an identifier for the message, represented as a string.

452a7c7c7c7048c2f887f61572b18fc2

messaging.operation.type

string

experimentalA string identifying the kind of messaging operation.

peek

messaging.system

string

experimentalAn identifier for the messaging system. See below for a list of well-known identifiers.

kafka; rabbitmq

network.protocol.name

string

stableOSI Application Layer or non-OSI equivalent.

amqp; http; mqtt

network.transport

string

stableOSI Transport Layer or Inter-process Communication method 1

inproc

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known. 2

example.com

server.port

long

stableLogical server port number.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

1

Only necessary when the server.* attributes don't apply.

2

This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from.

messaging.operation.type has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

peek

A message is received from a destination by a message consumer/server, but left there (span.kind is "consumer").

process

A message previously received from a destination is processed by a message consumer (span.kind is "consumer").

publish

A message is sent to a destination by a message producer (span.kind is "producer").

receive

A message is received from a destination by a message consumer (span.kind is "consumer").

messaging.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

activemq

ActiveMQ

artemis

ActiveMQ Artemis

aws_eventbridge

Amazon EventBridge

aws_sns

Amazon Simple Notification Service (SNS)

aws_sqs

Amazon Simple Queue Service (SQS)

azure_eventgrid

Azure Event Grid

azure_eventhubs

Azure Event Hubs

azure_servicebus

Azure Service Bus

gcp_pubsub

Google Cloud Pub/Sub

hornetq

HornetQ

jms

Java Message Service

kafka

Apache Kafka

mqseries

IBM MQ

msmq

MSMQ

rabbitmq

RabbitMQ

rocketmq

Apache RocketMQ

sag_webmethods_is

Software AG, webMethods Integration Server

tibco_ems

Tibco EMS

weblogic

Oracle WebLogic

websphere

IBM WebSphere Application Server

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

inproc

In-process communication. 1

other

Something else (non-IP-based).

pipe

Named or anonymous pipe.

tcp

TCP

udp

UDP

unix

Unix domain socket.

1

Signals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.

Messaging Destination

A destination represents the component within a messaging system where messages are sent to and consumed from. A destination is usually uniquely identified by its name within the messaging system instance. Examples of a destination name would be a URL or any other identifier of a specific queue, topic, or other entity within the broker. Messaging producer and consumer spans will have the following messaging destination attributes defined:

AttributeTypeDescriptionExamples

messaging.destination.manager_name

string

experimentalThe destination's manager name 1

MyBroker

messaging.destination.name

string

experimentalThe message destination name 2

MyQueue; MyTopic

messaging.destination.temporary

boolean

experimentalA boolean that is true if the message destination is temporary and might not exist anymore after messages are processed.

1

Manager name uniquely identifies the broker.

2

Destination name uniquely identifies a specific queue, topic or other entity within the broker.

Akka Messaging

Akka Producer

Sender side via ActorRef.tell() or ActorSelection.tell() is represented by a span, with span.kind set to producer.

AttributeTypeDescriptionExamples

messaging.akka.actor.path

string

experimentalPath to actor inside actor system.

/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$a

messaging.akka.actor.system

string

experimentalName of the actor system.

RequesterSystem; ResponseSystem

messaging.akka.message.type

string

experimentalFully qualified type name of the message.

java.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessage

messaging.message.body.size

long

experimentalThe (uncompressed) size of the message payload in bytes.

2738

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known.

example.com

server.port

long

stableLogical server port number.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

Akka Consumer

The receiver side via ActorCell.invoke() (inproc) or Actor.receive() (tcp) is represented by a span, with span.kind set to consumer.

AttributeTypeDescriptionExamples

messaging.akka.actor.kind

string

experimentalName of the top-level actor. See The Akka actor hierarchy 1

system; user

messaging.akka.actor.path

string

experimentalPath to actor inside actor system. 2

/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$a

messaging.akka.actor.system

string

experimentalName of the actor system. 3

RequesterSystem; ResponseSystem

messaging.akka.actor.type

string

experimentalFully qualified type name of actor. 4

com.acme.RespondingActor

messaging.akka.message.type

string

experimentalFully qualified type name of the message.

java.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessage

messaging.message.body.size

long

experimentalThe (uncompressed) size of the message payload in bytes.

2738

network.transport

string

stableFor Akka local, network.transport is set to inproc; for Akka remoting, it's set to tcp.

inproc; tcp

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known. 5

example.com

server.port

long

stableLogical server port number. 6

65123; 80

1

Unavailable in case of network.transport = tcp

2

Unavailable in case of network.transport = inproc

3

Unavailable in case of network.transport = inproc

4

Unavailable in case of network.transport = tcp

5

Unavailable in case of network.transport = inproc

6

Unavailable in case of network.transport = inproc

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription

inproc

In-process communication. 1

other

Something else (non-IP-based).

pipe

Named or anonymous pipe.

tcp

TCP

udp

UDP

unix

Unix domain socket.

1

Signals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.

Kafka Messaging

This convention extends the default semantic convention for messaging systems.

Kafka Producer

AttributeTypeDescriptionExamples

messaging.destination.partition.id

string

experimentalString representation of the partition ID the message is sent to or received from.

1

messaging.kafka.message.key

string

experimentalThe key property of the message.

mykey

messaging.kafka.message.tombstone

boolean

experimentalA boolean that is true if the message is a tombstone. 1

true

messaging.kafka.offset

long

experimentalThe offset of the message.

42

1

If the message is a tombstone, the value is true. When missing, the value is assumed to be false.

Kafka Consumer

AttributeTypeDescriptionExamples

messaging.consumer.group.name

string

experimentalThe name of the consumer group with which a consumer is associated. 1

my-group; indexer

messaging.destination.partition.id

string

experimentalString representation of the partition ID the message is sent to or received from.

1

messaging.kafka.message.key

string

experimentalThe key property of the message.

mykey

messaging.kafka.message.tombstone

boolean

experimentalA boolean that is true if the message is a tombstone. 2

true

messaging.kafka.offset

long

experimentalThe offset of the message.

42

1

Kafka consumer group ID

2

If the message is a tombstone, the value is true. When missing, the value is assumed to be false.

RPC Spans

Fields that describe remote procedure calls (also called "remote method invocations" / "RMI") with spans.

A remote procedure call is described by two separate spans, one on the client and one on the server side.

For outgoing requests, the SpanKind MUST be set to client and for incoming requests to server.

Common Fields

AttributeTypeDescriptionExamples

network.protocol.name

string

stableThe protocol that is used in the remote procedure call or web service. It can be omitted if it matches with rpc.system. See below for a list of well-known identifiers.

grpc; rest_http; soap; dotnet_remoting; hessian; java_rmi; json_rpc

rpc.method

string

experimentalThe name of the (logical) method being called 1

exampleMethod

rpc.namespace

string

experimentalThe namespace of the method being called. In SOAP, it would be the XML namespace.

tempuri.org

rpc.service

string

experimentalThe full (logical) name of the service being called, including its package name, if applicable. 2

myservice.EchoService

rpc.system

string

experimentalA string identifying the remoting system or framework. See below for a list of well-known identifiers.

apache_cxf; dotnet_wcf; grpc; jax_ws

server.address

string

stableLogical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known.

example.com

server.port

long

stableLogical server port number.

65123; 80

server.resolved_ips

ipAddress[]

stableA list of IP addresses that are the result of DNS resolution of server.address.

[194.232.104.141, 2a01:468:1000:9::140]

1

This is the logical name of the method from the RPC interface perspective, which can be different from the name of any implementing method/function. The code.function attribute may be used to store the latter (e.g., method executing the call on the server side, RPC client stub method on the client side).

2

This is the logical name of the service from the RPC interface perspective, which can be different from the name of any implementing class. The code.namespace attribute may be used to store the latter (despite the attribute name, it may include a class name, e.g., class with method executing actually executing the call on the server side, RPC client stub class on the client side).

RPC server

This span type represents an inbound RPC request.

For an RPC server span, SpanKind is Server.

AttributeTypeDescriptionExamples

code.function

string

experimentalThe method or function name, or equivalent (usually the rightmost part of the code unit's name). Represents the name of the function that is represented by this span. 1

serveRequest

code.invoked.function

string

experimentalLike code.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.

invoke

code.invoked.namespace

string

experimentalLike code.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.

com.sun.xml.ws.server.InvokerTube$2

code.namespace

string

experimentalThe namespace within which code.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit. 2

com.example.MyHttpService

network.transport

string

stableOSI Transport Layer or Inter-process Communication method

tcp; udp

1

In the case of RPC, code.function represents the handler function that processes the RPC.

2

In the case of RPC, code.namespace represents the namespace of the handler function that processes the RPC.

gRPC

Additional conventions for remote procedure calls via [gRPC][].

rpc.framework and rpc.protocol MUST be set to "grpc".

gRPC Attributes

AttributeTypeDescriptionExamples

rpc.grpc.status_code

long

experimentalThe numeric status code of the gRPC request.

gRPC

RMI

Additional conventions for remote procedure calls via [RMI][].

rpc.framework and rpc.protocol MUST be set to "java-rmi".

RMI Fields

AttributeTypeDescriptionExamples

rpc.rmi.registry

string

experimentalThe URL of a rmi endpoint.

Calculator

RMI

z/OS Connect EE Spans

z/OS Connect EE (Enterprise Edition) is an IBM product that exposes applications and data in z/OS subsystems such as CICS, IMS, or MQ via a RESTful API. The z/OS Connect EE (v3.0) implementation is built on the WebSphere Application Server for the z/OS Liberty profile.

AttributeTypeDescriptionExamples

ibm.cics.program

string

resource experimentalThe name of the CICS program. 1

EDUCHAN

zosconnect.api.description

string

experimentalThe z/OS Connect API description.

The API for the CICS catalog manager sample application.

zosconnect.api.name

string

experimentalThe z/OS Connect API name.

catalog

zosconnect.api.version

string

experimentalThe z/OS Connect API version.

1.0.0

zosconnect.request.body.size

long

experimentalThe size of the request payload in bytes.

234

zosconnect.request.id

long

experimentalThe z/OS Connect request ID.

2215

zosconnect.request.type

string

experimentalThe type of the REST request. 2

ADMIN

zosconnect.response.body.size

long

experimentalThe size of the response payload in bytes.

125

zosconnect.service.description

string

experimentalThe z/OS Connect service description.

EDUCHAN service using the CICS Service Provider

zosconnect.service.name

string

experimentalThe z/OS Connect service name.

placeOrder

zosconnect.service.provider.name

string

experimentalThe service provider name.

CICS-1.0

zosconnect.service.version

string

experimentalThe z/OS Connect service version.

2.0

zosconnect.sor.identifier

string

experimentalThe system of record identifier. The format differs depending on the SOR type. 3

localhost:8080

zosconnect.sor.reference

string

experimentalThe system of record reference.

cicsConn

zosconnect.sor.resource

string

experimentalIdentifier for the resource invoked on the system of record. The format differs depending on the SOR type. 4

01,DFH0XCMN

zosconnect.sor.type

string

experimentalThe system of record type.

CICS

1

Only applicable if zosconnect.sor.type is CICS

2

See https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=spi-datarequesttype

3

See https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=spi-data#SOR_IDENTIFIER

4

See https://www.ibm.com/docs/en/zos-connect/zosconnect/3.0?topic=spi-data#SOR_RESOURCE

zosconnect.request.type MUST be one of the following:

ValueDescription

ADMIN

admin

API

api

SERVICE

service

UNKNOWN

unknown

zosconnect.sor.type MUST be one of the following:

ValueDescription

CICS

cics

IMS

ims

MQ

mq

REST

rest

WOLA

wola

z/OS Spans

The semantic conventions for z/OS client and server spans. You can use them for CICS and IMS schemes.

Common Attributes

The common attributes listed in this section apply to both CICS and IMS clients and servers, in addition to the specific attributes listed in the CICS common, CICS client, and CICS server sections below.

AttributeTypeDescriptionExamples

zos.transaction.call_type

string

experimentalThe type of transaction call that was invoked.

CTG

zos.transaction.job_name

string

resource experimentalThe jobname of the z/OS address space that the transaction executed in.

CICSAOR0; CTGATM00; IMSCR15

zos.transaction.lpar_name

string

resource experimentalThe name of the LPAR that the transaction executed on.

S0W1; ABCD

CICS common

AttributeTypeDescriptionExamples

cics.transaction.system_id

string

resource experimentalThe system ID of the CICS region that this transaction executed on.

C259; CICS

cics.transaction.task_id

long

experimentalThe CICS task ID of this transaction.

1234

CICS client

This span type represents an outbound CICS request.

For a CICS client span, span.kind is client.

AttributeTypeDescriptionExamples

cics.transaction.system_id

string

resource experimentalThe system ID of the server CICS region that will contain the started server transaction.

C259; CICS

zos.transaction.lpar_name

string

resource experimentalThe LPAR name that hosts the CICS region that will contain the started server transaction.

S0W1; ABCD

CICS server

This span type represents an inbound CICS request.

For a CICS server span, span.kind MUST be server.

AttributeTypeDescriptionExamples

cics.transaction.system_id

string

resource experimentalThe system ID of the client CICS region that triggered this transaction.

C259; CICS

zos.transaction.lpar_name

string

resource experimentalThe LPAR name that hosts the client CICS region.

S0W1; ABCD

Related tags
Application Observability