Distributed traces are used to capture transactions flowing through a system. Traces are made of spans, which represent the units of work within a distributed trace.
Semantic conventions for CTG request and response spans captured on a CTG server or CTG client.
The span.kind is client for spans captured on a CTG client and is server for spans captured on a CTG server. This applies to both the request and the response.
CTG supports different request types, such as ECI, ESI, or EPI. The call type and response codes have different semantics depending on the request type. Furthermore, some fields are only populated for some request types.
Request spans have attributes conforming to the following table:
cics.transaction.user_idUSER1; anonctg.request.call_type2ctg.request.commarea_length0ctg.request.extend_mode11ctg.request.flow_type5ctg.request.gateway_urltcp://1.2.3.4:5678/ctg.request.object_namectg.request.server_idIPICTESTctg.request.term_idCN02ctg.request.typeBASEibm.cics.programEDUCHANserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]zos.transaction.idCEMT; DTAX; IVTNOThe values are defined in the IBM CTG API source code.
ctg.request.type MUST be one of the following:
ADMINAUTHBASEECIEPIESIXAnetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
Response spans have attributes conforming to the following table:
ctg.request.typeBASEctg.response.code-23ctg.request.type MUST be one of the following:
ADMINAUTHBASEECIEPIESIXACustom service spans represent entry points into a service or entry points to specific components within a larger application. OneAgent Code Modules can create custom service spans via automatic instrumentation rules or directly by OneAgent SDK APIs.
custom_service.methodstartTask; run; authenticatecustom_service.nameMyCustomService; AuthenticationComponentsupportability.custom_service.rule_id4d76194c11a9426197a9062548f9e66eSemantic conventions for database client spans. The span.kind for database client spans is client.
A span representing a database client operation might include the operation itself and the following result processing (for example, fetch from SQL result set, MongoDB cursor operations, etc.). Besides, several similar db operations might be aggregated into a single span for efficiency.
The aggregation attributes provide information on how many db operations have been aggregated.
The db.result.* attributes represent the details of the result processing.
db.affected_item_count32db.collection.namecustomers; public.usersdb.cosmosdb.request_charge4.95; 2.0db.dynamodb.table_names[Cats, Dogs]db.namespacecustomers; test.usersdb.operation.namedrop; findAndModify; SELECT; PREPARE; GetItem; set; LPUSH; mutateIn; ReadItemsdb.query.parameterssensitive-spans[{'name': 'paul', 'age': '23'}, {'name': 'mary', 'age': '32'}]; [{'0': 'paul', '1': '23'}, {'0': 'mary', '1': '32'}]db.query.textSELECT * FROM wuser_table; SET mykey "WuValue"db.result.duration_max345db.result.duration_min123db.result.duration_sum234db.result.exception_count2db.result.execution_count12db.result.roundtrip_count2db.systemmongodb; mysqlDepending on the data provided on ingest, this attribute may be derived by e.g., parsing db.query.text. Parsing might fail, or the result might be inaccurate.
The value may be sanitized to exclude sensitive information.
db.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
adabascachecassandraclickhousecloudscapecockroachdbcoldfusioncosmosdbcouchbasecouchdbdb2derbydl/idynamodbedbelasticsearchfilemakerfirebirdfirstsqlgeodeh2hanadbhbasehivehsqldbinformixingresinstantdbinterbasemariadbmaxdbmemcachedmongodbmssqlmssqlcompactmysqlneo4jnetezzaopensearchoracleother_sqlpervasivepointbasepostgresqlprogressredisredshiftspannersqlitesybaseteradataverticanetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
db.dli.pcb3; MYPCBNAMdb.dli.segment_namePARTROOTdb.dli.segment_level3; 24db.dli.processing_optionsGRdb.dli.terminal_nameHWSAM5ZD; 10505db.dli.status_codeQCdb.dli.pcb_typeDC; DL/I; F/Pdb.dli.pcb_type MUST be one of the following:
DCDL/IF/PThe semantic conventions for the Dynatrace span and the fields the user can expect.
The following hierarchical attributes are mandatory.
span.alternate_parent_idspan.id of this span's parent span. If a trace is monitored by more tracing systems (for example, OneAgent and OpenTelemetry), there might be two parent spans. If the two parent spans differ, span.parent_id holds the ID of the parent span originating from same tenant of the span while span.alternate_parent_id holds the other parent span ID. The span.alternate_parent_id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288cspan.idspan.id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288cspan.is_subroutinetrue, it indicates that this span is a subroutine of its parent span. The spans represent functions running on the same thread on the same call stack.span.kindserverspan.parent_idspan.id of this span's parent span. The span.parent_id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288ctrace.idtrace.id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8Attributes start_time, end_time, and duration are mandatory for all spans.
The attributes in the span.timing namespace are optional and represent measurements provided by OneAgent.
durationstart_time and end_time in nanoseconds.42end_timestart_time.1649822520123123165span.timing.cpuspan.timing.cpu_selfstart_timeend_time.1649822520123123123OneAgent might aggregate spans with the same parent span into a single one. The aggregated span contains attributes to indicate that aggregation happened and to allow the reconstruction of details.
For aggregated spans, start_time holds the earliest start_time and end_time holds the latest end_time of all aggregated spans. For non-aggregated spans, duration is the difference between start_time and end_time, which might differ from aggregation.duration_sum because aggregated spans were executed in parallel or there were gaps between the spans.
aggregation.count3aggregation.duration_max482aggregation.duration_min42aggregation.duration_samples[42, 482, 301]aggregation.duration_sum123aggregation.exception_count0; 6aggregation.parallel_executiontrue indicates that aggregated spans may have been executed in parallel. Therefore, start_time + duration_sum may exceed end_time.If the span does not represent a single span, it can have attributes to support the extrapolation of its values.
sampling.thresholdsampling.threshold with respect to a 56-bit random integer rv. A span is sampled if rv >= sampling.threshold; the sampling threshold acts as a rejection threshold and can be interpreted as the number of spans discarded out of 2^56. The attribute is only available if the sampling.threshold is not 0, and therefore sampling happened. The relationship between sampling probability and threshold is sampling probability = (2^56-sampling.threshold) * 2^(-56). Hence, sampling.threshold=0 means 100% sampling probability (collect all data), sampling.threshold=2^55 corresponds to a sampling probability of 50%, sampling.threshold=2^54 corresponds to a sampling probability of 75%.36028797018963968supportability.alr_sampling_ratiosupportability.alr_sampling_ratio would be 8 and the numerator is 1.8supportability.atm_sampling_ratiosupportability.atm_sampling_ratio would be 16 and the numerator is 1.16Sampling can occur in two stages of data processing. The span has the sampling.threshold for calculating the combined (effective) sample rate in both cases. Supportability attributes help in understanding the sampling at the different stages.
supportability.atm_sampling_ratio is added to all affected spans.supportability.alr_sampling_ratio is added to all affected spans.If, for example, OneAgent samples with a probability of 25%, the spans would contain the attributes sampling.threshold=54043195528445952 and supportability.atm_sampling_ratio=4.
To learn about Adaptive Traffic Management for distributed tracing see the documentation.
When a span logically represents the execution of a function, it will have code.* attributes describing that function.
invoked.code.* attributes describe the function in which a span started but not necessarily ended. It often represents the function that has been instrumented to start a span. These attributes are populated only if they mismatch with code.*.
code.functionserveRequestcode.namespacecode.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit.com.example.MyHttpServicecode.filepath/usr/local/MyApplication/content_root/app/index.phpcode.invoked.functioncode.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.invokecode.invoked.namespacecode.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.com.sun.xml.ws.server.InvokerTube$2code.invoked.filepathcode.filepath, only it represents the file path of the function that was active when a span has been started. Typically, it is the function that has been instrumented. It should only be set if it differs from code.filepath./usr/local/MyApplication/content_root/app/index.phpcode.call_stackcode.function. The call stack starts with the code.function, and the stack frames are separated by a line feed.com.example.SampleClass.doProcessing(SampleClass.java) com.example.SampleClass.doSomeWork(SampleClass.java) com.example.SampleClass.main(SampleClass.java)span.eventssupportability.dropped_events_count1Span events semantics are defined here.
If an exception exited the span or contains other exception events, the following fields are available to reference the correct exception in the list of the span.events.
span.exit_by_exception_idexception.id of the exception the its span.events with the current span exited. The referenced exception has set the attribute exception.escaped to true.span.is_exit_by_exceptiontrue if an exception exited the span. If set to false, the span has exception events, but none exited the span.span.linkssupportability.dropped_links_count1Span links have their own semantics defined here.
These attributes may be used to describe the client and server in a connection-based network interaction, where one side (the client) initiates the connection. This covers all TCP network interactions since TCP is connection-based, and one side initiates the connection (an exception is made for peer-to-peer communication over TCP where the user-facing surface of the protocol or API doesn't expose a clear notion of client and server). This also covers UDP network interactions where one side initiates the interaction, for example, QUIC (HTTP/3) and DNS.
In an ideal situation, not accounting for proxies, multiple IP addresses, or host names, the server.* attributes are the same on the client and server span.
server.addressexample.comserver.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]server.port65123; 80server.addressFor IP-based communication, the name should be one of the service's DNS hostnames. On the client side, the name matches the remote service name; on the server side, it represents the local service name as seen externally on clients.
When connecting to a URL https://example.com/foo, server.address matches example.com on both the client and server sides.
On the client side, it's usually passed as a URL, connection string, host name, etc. Sometimes, the hostname is only available as a string that may contain a DNS name or IP address.
If network.transport is pipe, the absolute path to the file representing it is used as server.address.
For a Unix domain socket, the server.address attribute represents the remote endpoint address on the client side and the local endpoint address on the server side.
client.ipsensitive-spans194.232.104.141; 2a01:468:1000:9::140client.port65123; 80client.ispInternet Service Provider Nameclient.ip.is_publictrueclient.app.nameMS OutlookSupportability attributes help to understand the characteristics of the span.
supportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]trace.alternate_idtrace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.statef4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=barSpans on which an endpoint is detected represent requests. They are marked as request roots and are evaluated by failure detection. Request root spans are central to monitoring the health and performance of services and their endpoints.
endpoint.namehttp.route or rpc.method. Endpoints are exclusively detected on request root spans.GET /; PUT /users/:userID?; GET /productpage; Reviews.GetReviewsrequest.is_failedrequest.is_root_spanRequest attributes allow you to enrich spans collected by OneAgents with deep-insight data, which is not captured on trace data by default. They're modelled as:
The names of request and captured attributes are composed of the prefixes captured_attribute and request_attribute and the name given in the configuration by the user:
captured_attribute.__attribute_name____attribute_name__ defined by the request attribute configuration. The values are mapped as an array according to the type of the captured attributes, so either boolean, double, long, or string. If the captured attributes have mixed types (e.g. long and string, or double and long, etc.), all attributes are converted to string and stored as string array.[42]; ['Platinum']; [32, 16, 8]; ['Special Offer', '1702']; ['18.35', '16']request_attribute.__attribute_name____attribute_name__ defined by the request attribute configuration. The data type of the value depends on the request attribute definition.sensitive-spans42; Platinum; ['Product A', 'Product B']; ['Special Offer', '1702']Subtraces are an important hierarchical grouping of spans within a trace. A subtrace spawns, for example, all spans created during a single service entry. The grouping of spans into subtraces allows for very powerful queries and advanced analysis that require correlation across spans, for example, to count the number of database calls per request. Every subtrace always has a subtrace root span, which triggers the detection of endpoints and hence defines the service API. Subtraces are only available for OneAgent spans.
subtrace.id95efd70fcdb5b7b3; 96835e1d65490b48subtrace.is_root_spanThe calculated sizes of a span in bytes. The dt.ingest.size is calculated when the span is ingested, while the dt.retain.size is calculated before the span gets stored.
dt.ingest.size2005dt.retain.size2005A span contains a status consisting of a code and an optional descriptive message. The status is especially relevant if there is a known error in the application code, such as an exception, in which case the span status can be set to error. The span status is only present if set to error or ok.
span.status_codeunset.errorspan.status_messagespan.status_code is error.Connection closed before message completed; Error sending request for urlThe following reasons cause the span.status_code to be error:
span.is_exit_by_exception is set to true.http.response.status_code values in the 5xx range.span.kind is client: For http.response.status_code values in the 4xx range.span.kind is client: For all rpc.grpc.status_code values except OK (0).span.kind is server: For rpc.grpc.status_code values UNKNOWN (2), DEADLINE_EXCEEDED (4), UNIMPLEMENTED (12), INTERNAL (13), UNAVAILABLE (14), DATA_LOSS (15).span.nameprepareOrderItemsAndShippingQuoteFromCart; org.example.CheckoutService/PlaceOrder; orders process; GET /products/{product_id}; HTTP POSTThe technology-specific attributes defining the span name are described here.
Besides the attributes listed above, other arbitrary attributes are allowed on a span.
Semantic conventions for span events on Dynatrace spans.
In general, a span event does not need to follow specific semantics, but typically, span events have the following common attributes.
span_event.nameexceptionsupportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]timestamp1649822520123123123Besides the common attributes, any arbitrary attributes are allowed for span events.
If exceptions happen and are captured during a span, they're available as span events. Exceptions events have the span_event.name set to exception. Besides the following exception event-specific attributes, all semantics of the common section apply.
exception.caused_by_idexception.id of the exception the current exception was caused by.exception.escapedtrue indicates that the exception was recorded at a point where it is known that the exception escaped the scope of the span.exception.idexception.is_caused_by_roottrue if the exception is the first exception caused by the chain.exception.messageDivision by zeroexception.stack_trace@https://www.foo.bar/path/main.js:59:26 e@https://www.foo.bar/path/lib/1.1/lib.js:2:30315exception.typejava.net.ConnectException; OSErrorspan_event.nameexception for exception events.exceptionA flag evaluation SHOULD be recorded as an event on the span during which it occurred. Feature flag events have the span_event.name set to feature_flag.
feature_flag.keylogo-colorfeature_flag.provider_nameFlag Managerfeature_flag.variantred; true; onspan_event.namefeature_flag for feature flag events.feature_flagA semantic identifier, commonly referred to as a variant, provides a means
for referring to a value without including the value itself. This can
provide additional context for understanding the meaning behind a value.
For example, the variant red maybe be used for the value #c05543.
A stringified version of the value can be used in situations where a semantic identifier is unavailable. The implementer should determine the string representation of the value.
If a span is related to a business event, the IDs of the business events are available as span events. The span events have the span_event.name set to bizevent.
event.id5547782627070661074_1647601320000span_event.namebizevent for bizevent events.bizeventSemantic conventions for well known span links.
A span link by span.id and trace.id references to a target span on a different trace.
span.idspan.id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288csupportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]trace.alternate_idtrace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.idtrace.id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.statef4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=barBesides the attributes listed above, arbitrary other attributes are allowed on a generic span link.
A span link by dt.tracing.custom_link.id references another span with a span link to the same dt.tracing.custom_link.id. This span link is used for technologies where full context propagation with span ID and trace ID is not possible, and only a limited amount of information can be propagated between tiers.
The dt.tracing.link.direction defines the hierarchy between two spans having span links with the same dt.tracing.custom_link.id.
dt.tracing.custom_link.id736bd2684696c4a8dt.tracing.custom_link.original_bytesycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==dt.tracing.custom_link.transformed_bytesycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==dt.tracing.custom_link.typedt.tracing.custom_link.original_bytes to the dt.tracing.custom_link.transformed_bytes was applied.genericdt.tracing.link.directionoutgoingdt.tracing.link.is_synctrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.timestampdt.tracing.link.direction set to outgoing.1649822520123123123A Dynatrace link provides additional details regarding the parent and child timing on top of span.parent_id.
This data is optional and not provided by all span data sources.
dt.tracing.link.directionoutgoingdt.tracing.link.iddt.tracing.link.is_synctrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.timestampdt.tracing.link.direction set to outgoing.1649822520123123123A span link by dt.tracing.foreign_link refers to an upstream transaction. This could be a cross-environment link or a cross-product link to a distributed trace in the legacy AppMon product.
Depending on whether the link information was received in binary or text, either dt.tracing.foreign_link.bytes or dt.tracing.foreign_link.text will be set, respectively.
dt.tracing.foreign_link.bytes00000004000000010000000200000003000000040000002300000001dt.tracing.foreign_link.textFW4;129;12;-2023406815;4539717;0;17;66;c511;2h02;3h12345678;4h676767; FW1;129;4711;59959450;-1859959450;3;17;0A span link by dt.tracing.response.headers refers to a downstream transaction. This could be a cross-environment link.
dt.tracing.response.headers{'traceresponse': ['00-7b9e3e4068167838398f50017bfad358-d4ffc7e33530967a-01'], 'x-dt-tracestate': ['9651e1a8-19506b7c@dt']}Semantic conventions for ESB (Enterprise Service Bus) spans. An ESB span holds information about the realm in which the span is produced. This metadata includes, for example, the workflow in which the span is placed and the application or library to which the workflow belongs.
The workflow name is mandatory information and contributes the most to identifying where this span comes from.
The application and library provide a high-level view of which deployment the span belongs to.
It's also worth mentioning that application and library are typically seen as mutually exclusive, although this is not enforced in any way."
esb.application.namemyBusinessApp; YourServiceApp; any_workesb.library.namemyWebServicesLib; YourMessagingLibrary; any_toolsesb.vendoribm; tibcoesb.workflow.is_subprocessfalseesb.workflow.namemyMessageFlow; YourBusinessWorkflow; any_flowFields that can be expected from serverless functions or Function as a Service (FaaS) on various cloud platforms. There are general attributes and attributes specific to incoming FaaS invocations (server side) and outgoing invocations (FaaS function as client calling out).
azure.class_nameHost.Functionsazure.site_namedt-function-scriptedcloud.account.id111111111111; opentelemetrycloud.provideralibaba_cloudcloud.regionus-east-1cloud.resource_idarn:aws:lambda:REGION:ACCOUNT_ID:function:my-function; //run.googleapis.com/projects/PROJECT_ID/locations/LOCATION_ID/services/SERVICE_ID; /subscriptions/<SUBSCIPTION_GUID>/resourceGroups/<RG>/providers/Microsoft.Web/sites/<FUNCAPP>/functions/<FUNC>faas.max_memoryfaas.namemy-function; myazurefunctionapp/some-function-name; test_functionThe prefix of the service matches the one specified in cloud.provider.
This is the name of the function as configured/deployed on the FaaS platform and is usually different from the name of the callback
function (which may be stored in the code.namespace/code.function span attributes).
Value of the field depends on a cloud provider. This field is not set for Azure.
This section describes incoming FaaS invocations as they are reported by the FaaS instance itself.
For incoming FaaS spans, the span.kind is either server or consumer.
aws.request_idx-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).0e7bc729-a468-57e8-8143-98f2eec5c925aws.xray.trace_idx-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678faas.coldstartfaas.triggerdatasourceThis section describes outgoing FaaS invocations as they are reported by a client calling a FaaS instance.
For outgoing FaaS spans, the span.kind is either client or producer.
aws.request_idx-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).0e7bc729-a468-57e8-8143-98f2eec5c925aws.xray.trace_idx-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678faas.invoked_namemy-functionfaas.invoked_providercloud.provider resource attribute.alibaba_cloudWill be equal to the invoked function's cloud.region resource attribute.
Semantic conventions for HTTP client and server spans. They can be used for HTTP and HTTPS schemes and various HTTP versions like 1.1, 2, and SPDY.
The common fields listed in this section apply to both HTTP clients and servers in addition to the specific fields listed in the HTTP client and HTTP server sections below.
http.request.body.size3495http.request.header.__key____key__ being the lowercase HTTP header name, for example "http.request.header.accept-encoding". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.sensitive-spanshttps://www.foo.bar/; gzip, deflate, br; 1.2.3.4, 1.2.3.5http.request.methodGET; POST; HEADhttp.response.body.size3495http.response.header.__key____key__ being the lowercase HTTP header name, for example, "http.response.header.content-type". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.909; text/html; charset=utf-8; abc, defnetwork.protocol.version1.1; 3.1.1This span type represents an outbound HTTP request.
For an HTTP client span, span.kind is client.
server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]url.fullsensitive-spanshttps://www.foo.bar/docs/search?q=OpenTelemetry#SemConvThis span type represents an inbound HTTP request.
For an HTTP server span, span.kind MUST be server.
client.ipsensitive-spans194.232.104.141; 2a01:468:1000:9::140http.route/users/:userID?; Home/Index/{id?}server.addressexample.comserver.port65123; 80url.path/docs/searchurl.querysensitive-spansq=OpenTelemetryurl.schemehttps; ftp; telnetmessaging.client.idaclient; myhost@68d46b89c9-c29qcmessaging.message.body.size2738messaging.message.conversation_idMyConversationIdmessaging.message.header.__key____key__ being the message header/attribute name, for example, "messaging.message.header.extendedPayloadSize". The data type of the value depends on the attribute.1024, "my-eu-bucket-3", ["a", "b"]messaging.message.id452a7c7c7c7048c2f887f61572b18fc2messaging.operation.typepeekmessaging.systemkafka; rabbitmqserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]If a custom value is used for messaging.operation.type, it MUST be of low cardinality.
Only necessary when the server.* attributes don't apply.
This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from.
messaging.operation.type has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
peekprocesspublishreceivemessaging.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
activemqartemisaws_eventbridgeaws_snsaws_sqsazure_eventgridazure_eventhubsazure_servicebusgcp_pubsubhornetqjmskafkamqseriesmsmqrabbitmqrocketmqsag_webmethods_istibco_emsweblogicwebspherenetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
A producer of a message should set the span kind to producer unless it synchronously waits for a response; then it should use client.
The following additional attributes describe message producer operations.
messaging.destination.kindqueue; topicmessaging.destination.temporaryManager name SHOULD uniquely identify the broker.
Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker.
messaging.destination.kind MUST be one of the following:
queuetopicThe consumer of a message should set the span kind to consumer unless it consistently sends back a reply that is directed to the producer of the message (as opposed to e.g., a queue on which the producer happens to listen): then it should use server.
The following additional attributes describe message consumer operations.
messaging.source.kindqueue; topicmessaging.source.temporaryManager name SHOULD uniquely identify the broker.
Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker.
messaging.source.kind MUST be one of the following:
queuetopicSender side via ActorRef.tell() or ActorSelection.tell() is represented by a span, with span.kind set to producer.
messaging.akka.actor.path/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$amessaging.akka.actor.systemRequesterSystem; ResponseSystemmessaging.akka.message.typejava.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessagemessaging.message.body.size2738server.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]The receiver side via ActorCell.invoke() (inproc) or Actor.receive() (tcp) is represented by a span, with span.kind set to consumer.
messaging.akka.actor.kindsystem; usermessaging.akka.actor.path/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$amessaging.akka.actor.systemRequesterSystem; ResponseSystemmessaging.akka.actor.typecom.acme.RespondingActormessaging.akka.message.typejava.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessagemessaging.message.body.size2738network.transportnetwork.transport is set to inproc; for Akka remoting, it's set to tcp.inproc; tcpserver.addressexample.comUnavailable in case of network.transport = tcp
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = tcp
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = inproc
network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
This convention extends the default semantic convention for messaging systems.
messaging.destination.partition.id1messaging.kafka.message.keykey property of the message.mykeymessaging.kafka.message.tombstonetruemessaging.kafka.offset42If the message is a tombstone, the value is true. When missing, the value is assumed to be false.
messaging.consumer.group.namemy-group; indexermessaging.destination.partition.id1messaging.kafka.message.keykey property of the message.mykeymessaging.kafka.message.tombstonetruemessaging.kafka.offset42Kafka consumer group ID
If the message is a tombstone, the value is true. When missing, the value is assumed to be false.
Fields that describe remote procedure calls (also called "remote method invocations" / "RMI") with spans.
A remote procedure call is described by two separate spans, one on the client and one on the server side.
For outgoing requests, the SpanKind MUST be set to client and for incoming requests to server.
network.protocol.namerpc.system. See below for a list of well-known identifiers.grpc; rest_http; soap; dotnet_remoting; hessian; java_rmi; json_rpcrpc.namespacetempuri.orgrpc.servicemyservice.EchoServicerpc.systemapache_cxf; dotnet_wcf; grpc; jax_wsserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]This is the logical name of the method from the RPC interface perspective, which can be different from the name of any implementing method/function. The code.function attribute may be used to store the latter (e.g., method executing the call on the server side, RPC client stub method on the client side).
This is the logical name of the service from the RPC interface perspective, which can be different from the name of any implementing class. The code.namespace attribute may be used to store the latter (despite the attribute name, it may include a class name, e.g., class with method executing actually executing the call on the server side, RPC client stub class on the client side).
This span type represents an inbound RPC request.
For an RPC server span, SpanKind is Server.
code.functionserveRequestcode.invoked.functioncode.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.invokecode.invoked.namespacecode.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.com.sun.xml.ws.server.InvokerTube$2code.namespacecode.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit. 2com.example.MyHttpServiceIn the case of RPC, code.function represents the handler function that processes the RPC.
In the case of RPC, code.namespace represents the namespace of the handler function that processes the RPC.
Additional conventions for remote procedure calls via [gRPC][].
rpc.framework and rpc.protocol MUST be set to "grpc".
Additional conventions for remote procedure calls via [RMI][].
rpc.framework and rpc.protocol MUST be set to "java-rmi".
rpc.rmi.registryCalculatorz/OS Connect EE (Enterprise Edition) is an IBM product that exposes applications and data in z/OS subsystems such as CICS, IMS, or MQ via a RESTful API. The z/OS Connect EE (v3.0) implementation is built on the WebSphere Application Server for the z/OS Liberty profile.
zosconnect.api.descriptionThe API for the CICS catalog manager sample application.zosconnect.api.namecatalogzosconnect.api.version1.0.0zosconnect.request.body.size234zosconnect.request.id2215zosconnect.response.body.size125zosconnect.service.descriptionEDUCHAN service using the CICS Service Providerzosconnect.service.nameplaceOrderzosconnect.service.provider.nameCICS-1.0zosconnect.service.version2.0zosconnect.sor.identifierlocalhost:8080zosconnect.sor.referencecicsConnzosconnect.sor.resource01,DFH0XCMNzosconnect.sor.typeCICSOnly applicable if zosconnect.sor.type is CICS
zosconnect.request.type MUST be one of the following:
ADMINAPISERVICEUNKNOWNzosconnect.sor.type MUST be one of the following:
CICSIMSMQRESTWOLAThe semantic conventions for z/OS client and server spans. You can use them for CICS and IMS schemes.
The common attributes listed in this section apply to both CICS and IMS clients and servers, in addition to the specific attributes listed in the CICS common, CICS client, and CICS server sections below.
zos.transaction.call_typeCTGzos.transaction.job_nameCICSAOR0; CTGATM00; IMSCR15zos.transaction.lpar_nameS0W1; ABCDcics.transaction.system_idC259; CICScics.transaction.task_id1234This span type represents an outbound CICS request.
For a CICS client span, span.kind is client.
cics.transaction.system_idC259; CICSzos.transaction.lpar_nameS0W1; ABCDThis span type represents an inbound CICS request.
For a CICS server span, span.kind MUST be server.
cics.transaction.system_idC259; CICSzos.transaction.lpar_nameS0W1; ABCD