Distributed traces are used to capture transactions flowing through a system. Traces are made of spans, which represent the units of work within a distributed trace.
Semantic conventions for CTG request and response spans captured on a CTG server or CTG client.
The span.kind is client for spans captured on a CTG client and is server for spans captured on a CTG server. This applies to both the request and the response.
CTG supports different request types, such as ECI, ESI, or EPI. The call type and response codes have different semantics depending on the request type. Furthermore, some fields are only populated for some request types.
Request spans have attributes conforming to the following table:
cics.transaction.user_idUSER1; anonctg.request.call_type2ctg.request.commarea_length0ctg.request.extend_mode11ctg.request.flow_type5ctg.request.gateway_urltcp://1.2.3.4:5678/ctg.request.object_namectg.request.server_idIPICTESTctg.request.term_idCN02ctg.request.typeBASEibm.cics.programEDUCHANserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]zos.transaction.idCEMT; DTAX; IVTNOThe values are defined in the IBM CTG API source code.
ctg.request.type MUST be one of the following:
ADMINAUTHBASEECIEPIESIXAnetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
Response spans have attributes conforming to the following table:
ctg.request.typeBASEctg.response.code-23ctg.request.type MUST be one of the following:
ADMINAUTHBASEECIEPIESIXACustom service spans represent entry points into a service or entry points to specific components within a larger application. OneAgent Code Modules can create custom service spans via automatic instrumentation rules or directly by OneAgent SDK APIs.
custom_service.methodstartTask; run; authenticatecustom_service.nameMyCustomService; AuthenticationComponentsupportability.custom_service.rule_id4d76194c11a9426197a9062548f9e66eSemantic conventions for database client spans. The span.kind for database client spans is client.
A span representing a database client operation might include the operation itself and the following result processing (for example, fetch from SQL result set, MongoDB cursor operations, etc.). Besides, several similar db operations might be aggregated into a single span for efficiency.
The aggregation attributes provide information on how many db operations have been aggregated.
The db.result.* attributes represent the details of the result processing.
db.affected_item_count32db.collection.namecustomers; public.usersdb.cosmosdb.request_charge4.95; 2.0db.dynamodb.table_names[Cats, Dogs]db.namespacecustomers; test.usersdb.operation.namedrop; findAndModify; SELECT; PREPARE; GetItem; set; LPUSH; mutateIn; ReadItemsdb.query.parameterssensitive-spans[{'name': 'paul', 'age': '23'}, {'name': 'mary', 'age': '32'}]; [{'0': 'paul', '1': '23'}, {'0': 'mary', '1': '32'}]db.query.textSELECT * FROM wuser_table; SET mykey "WuValue"db.result.duration_max345db.result.duration_min123db.result.duration_sum234db.result.exception_count2db.result.execution_count12db.result.roundtrip_count2db.systemmongodb; mysqlserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]Depending on the data provided on ingest, this attribute may be derived by e.g., parsing db.query.text. Parsing might fail, or the result might be inaccurate.
The value may be sanitized to exclude sensitive information.
db.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
adabascachecassandraclickhousecloudscapecockroachdbcoldfusioncosmosdbcouchbasecouchdbdb2derbydl/idynamodbedbelasticsearchfilemakerfirebirdfirstsqlgeodeh2hanadbhbasehivehsqldbinformixingresinstantdbinterbasemariadbmaxdbmemcachedmongodbmssqlmssqlcompactmysqlneo4jnetezzaopensearchoracleother_sqlpervasivepointbasepostgresqlprogressredisredshiftspannersqlitesybaseteradataverticanetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
db.dli.pcb3; MYPCBNAMdb.dli.segment_namePARTROOTdb.dli.segment_level3; 24db.dli.processing_optionsGRdb.dli.terminal_nameHWSAM5ZD; 10505db.dli.status_codeQCdb.dli.pcb_typeDC; DL/I; F/Pdb.dli.pcb_type MUST be one of the following:
DCDL/IF/PThe semantic conventions for the Dynatrace span and the fields the user can expect.
The following hierarchical attributes are mandatory.
span.alternate_parent_idspan.id of this span's parent span. If a trace is monitored by more tracing systems (for example, OneAgent and OpenTelemetry), there might be two parent spans. If the two parent spans differ, span.parent_id holds the ID of the parent span originating from same tenant of the span while span.alternate_parent_id holds the other parent span ID. The span.alternate_parent_id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288cspan.idspan.id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288cspan.is_subroutinetrue, it indicates that this span is a subroutine of its parent span. The spans represent functions running on the same thread on the same call stack.span.kindserverspan.parent_idspan.id of this span's parent span. The span.parent_id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288ctrace.idtrace.id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8Attributes start_time, end_time and duration are mandatory for all spans.
The attributes in the span.timing namespace are optional and represent measurements provided by the OneAgent.
durationstart_time and end_time in nanoseconds.42end_timestart_time.1649822520123123165span.timing.cpuspan.timing.cpu_selfstart_timeend_time.1649822520123123123OneAgent might aggregate spans having the same parent span into a single one. The aggregated span contains attributes to indicated that aggregation happened and to allow to reconstruct details.
For aggregated spans the start_time holds the earliest start_time and end_time holds the latest end_time of all aggregated spans. Like for non aggreated spans duration is the difference between start_time and end_time which might differ from aggregation.duration_sum because aggregated spans were executed in parallel or there were gaps between the spans.
aggregation.count3aggregation.duration_max482aggregation.duration_min42aggregation.duration_samples[42, 482, 301]aggregation.duration_sum123aggregation.exception_count0; 6aggregation.parallel_executiontrue indicates that aggregated spans may have been executed in parallel. Therefore, start_time + duration_sum may exceed end_time.If the span does not represent a single span, it can have attributes to support extrapolation of its values.
sampling.thresholdsampling.threshold with respect to a 56-bit random integer rv. A span is sampled if rv >= sampling.threshold; the sampling threshold acts as a rejection threshold and can be interpreted as the number of spans discarded out of 2^56. The attribute is only available if the sampling.threshold is not 0, and therefore sampling happened. The relationship between sampling probability and threshold is sampling probability = (2^56-sampling.threshold) * 2^(-56). Hence, sampling.threshold=0 means 100% sampling probability (collect all data), sampling.threshold=2^55 corresponds to a sampling probability of 50%, sampling.threshold=2^54 corresponds to a sampling probability of 75%.36028797018963968supportability.alr_sampling_ratiosupportability.alr_sampling_ratio would be 8 and the numerator is 1.8supportability.atm_sampling_ratiosupportability.atm_sampling_ratio would be 16 and the numerator is 1.16Currently sampling can happen two stages in the data processing. Independend where sampling happens the span has the sampling.threshold for calculation of the combined (effective) sample rate. Supportability attributes help the understand the sampling on the different stages.
supportability.atm_sampling_ratio is added to all effected spans.supportability.alr_sampling_ratio is added to all effected spans.If for example OneAgents samples with a probability of 25% the spans would contain the attributes sampling.threshold=54043195528445952 and supportability.atm_sampling_ratio=4.
Details about adaptive traffic management for distributed tracing can be found in the documentation.
When a span logically represents the execution of a function, it will have code.* attributes, describing that function.
invoked.code.* attributes describe the function in which a span has been started, but not necessarily ended. It often represents the function that has been instrumented in order to start a span. These attributes are only populated, if they mismatch with code.*.
code.functionserveRequestcode.namespacecode.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit.com.example.MyHttpServicecode.filepath/usr/local/MyApplication/content_root/app/index.phpcode.line.number1337code.invoked.functioncode.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.invokecode.invoked.namespacecode.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.com.sun.xml.ws.server.InvokerTube$2code.invoked.filepathcode.filepath, only it represents the file path of the function that was active when a span has been started. Typically, it is the function that has been instrumented. It should only be set if it differs from code.filepath./usr/local/MyApplication/content_root/app/index.phpcode.call_stackcode.function. The call stack starts with the code.function, and the stack frames are separated by a line feed.com.example.SampleClass.doProcessing(SampleClass.java) com.example.SampleClass.doSomeWork(SampleClass.java) com.example.SampleClass.main(SampleClass.java)span.eventssupportability.dropped_events_count1Span events have their own semantics defined here.
If the span was exited by an exception or contains other exception events, the following fields are available to provide a reference to the correct exception in the list of the span.events.
span.exit_by_exception_idexception.id of the exception the its span.events with the current span exited. The referenced exception has set the attribute exception.escaped to true.span.is_exit_by_exceptiontrue if an exception exited the span. If set to false, the span has exception events, but none exited the span.span.linkssupportability.dropped_links_count1Span links have their own semantics defined here.
Fields that can be expected for a failure detection on a Dynatrace span.
Failure detection will be applied to spans that represent requests on endpoints and incoming Istio services mesh proxies.
A request is considered failed if at least one failure reason is detected and no success forcing rule matches. The combined result (failure or success) will be stored in the attribute request.is_failed (see also Request).
To modify failure detection behavior, modify its configuration.
dt.failure_detection.ruleset_idid of the failure detection rule set (failure detection v2) that was applied to that span (uid128).4d76194c11a9426197a9062548f9e66edt.failure_detection.general_parameters_idid of the failure detection general parameters (failure detection v1) that were applied to that span (uid128).4d76194c11a9426197a9062548f9e66fdt.failure_detection.http_parameters_idid of the failure detection HTTP parameters (failure detection v1) that were applied to that span (uid128).4d76194c11a9426197a9062548f9e66adt.failure_detection.global_rule_idid of the global failure detection rule (failure detection v1) that was applied to that span (uid128).dt.failure_detection.global_parameters_id.4d76194c11a9426197a9062548f9e66bdt.failure_detection.global_parameters_idid of the global failure detection parameters (failure detection v1) that were applied to that span (uid128).dt.failure_detection.global_rule_id.4d76194c11a9426197a9062548f9e66cdt.failure_detection.resultsFailure detection has its own semantics defined here.
These attributes may be used to describe the client and server in a connection-based network interaction where there is one side (the client) that initiates the connection. This covers all TCP network interactions since TCP is connection-based and one side initiates the connection (an exception is made for peer-to-peer communication over TCP where the "user-facing" surface of the protocol / API does not expose a clear notion of client and server). This also covers UDP network interactions where one side initiates the interaction, e.g. QUIC (HTTP/3) and DNS.
In an ideal situation, not accounting for proxies, multiple IP addresses or host names, the server.* attributes are the same on the client and server span.
server.addressexample.comserver.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]server.port65123; 80server.addressFor IP-based communication, the name should be a DNS host name of the service. On client side it matches remote service name, on server side, it represents local service name as seen externally on clients.
When connecting to a URL https://example.com/foo, server.address matches "example.com" on both client and server side.
On client side, it's usually passed in form of a URL, connection string, host name, etc. Sometimes host name is only available as a string which may contain DNS name or IP address.
If network.transport is pipe, the absolute path to the file representing it is used as server.address.
For Unix domain socket, server.address attribute represents the remote endpoint address on client side and local endpoint address on server side.
client.ipsensitive-spans194.232.104.141; 2a01:468:1000:9::140client.port65123; 80client.ispInternet Service Provider Nameclient.ip.is_publictrueclient.app.nameMS Outlookclient.addressclient.example.com; 10.1.2.80; [local]Supportability attributes help to understand the characteristics of the span.
supportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]trace.alternate_idtrace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.statef4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=barRequests are an important hierarchical grouping of spans within a trace. A request spawns all spans from the detected request root span until the service is left again. The grouping of spans into requests allows for very powerful queries and advanced analysis. Every request always has a request root span which is special as it is the trigger for detecting endpoints and hence defining the service API.
dt.endpoint_detection.rule_id4d76194c11a9426197a9062548f9e66eendpoint.namehttp.route or rpc.method. Endpoints are exclusively detected on request root spans.GET /; PUT /users/:userID?; GET /productpage; Reviews.GetReviewsrequest.is_failedrequest.is_root_spanRequest attributes allow you to enrich spans collected by OneAgents with deep-insight data which is not captured on trace data by default. They are modelled as:
The names of request and captured attributes are composed of the prefixes "captured_attribute" and "request_attribute" and the name given in the configuration by the user:
captured_attribute.__attribute_name____attribute_name__ defined by the request attribute configuration. The values are mapped as an array according to the type of the captured attributes, so either boolean, double, long, or string. If the captured attributes have mixed types (e.g. long and string, or double and long, etc.), all attributes are converted to string and stored as string array.[42]; ['Platinum']; [32, 16, 8]; ['Special Offer', '1702']; ['18.35', '16']request_attribute.__attribute_name____attribute_name__ defined by the request attribute configuration. The data type of the value depends on the request attribute definition.sensitive-spans42; Platinum; ['Product A', 'Product B']; ['Special Offer', '1702']The calculated sizes of a span in bytes. The dt.ingest.size is calculated when the span is ingested while the dt.retain.size is calculated before the span gets stored.
dt.ingest.size2005dt.retain.size2005A span contains a status consisting of a code and an optional descriptive message. The status is especially relevant if there is a known error in the application code, such as an exception in which case the span status can be set to error. The span status is only present if it is explicitly set to error or ok.
span.status_codeunset.errorspan.status_messagespan.status_code is error.Connection closed before message completed; Error sending request for urlThe following reasons cause the span.status_code to be error:
span.is_exit_by_exception is set to true.http.response.status_code values in the 5xx range.span.kind is client: For http.response.status_code values in the 4xx range.span.kind is client: For all rpc.grpc.status_code values except OK (0).span.kind is server: For rpc.grpc.status_code values UNKNOWN (2), DEADLINE_EXCEEDED (4), UNIMPLEMENTED (12), INTERNAL (13), UNAVAILABLE (14), DATA_LOSS (15).Beside the attributes listed above, arbitrary other attributes are allowed on a span.
Semantic conventions for span events on Dynatrace spans.
In general, a span event does not need to follow specific semantics, but typically, span events have the following common attributes.
span_event.nameexceptionsupportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]timestamp1649822520123123123Besides the common attributes, any arbitrary attributes are allowed for span events.
If exceptions happen and are captured during a span, they're available as span events. Exceptions events have the span_event.name set to exception. Besides the following exception event-specific attributes, all semantics of the common section apply.
exception.caused_by_idexception.id of the exception the current exception was caused by.exception.escapedtrue indicates that the exception was recorded at a point where it is known that the exception escaped the scope of the span.exception.idexception.is_caused_by_roottrue if the exception is the first exception caused by the chain.exception.messageDivision by zeroexception.stack_trace@https://www.foo.bar/path/main.js:59:26 e@https://www.foo.bar/path/lib/1.1/lib.js:2:30315exception.typejava.net.ConnectException; OSErrorspan_event.nameexception for exception events.exceptionA flag evaluation SHOULD be recorded as an event on the span during which it occurred. Feature flag events have the span_event.name set to feature_flag.
feature_flag.keylogo-colorfeature_flag.provider_nameFlag Managerfeature_flag.variantred; true; onspan_event.namefeature_flag for feature flag events.feature_flagA semantic identifier, commonly referred to as a variant, provides a means
for referring to a value without including the value itself. This can
provide additional context for understanding the meaning behind a value.
For example, the variant red maybe be used for the value #c05543.
A stringified version of the value can be used in situations where a semantic identifier is unavailable. The implementer should determine the string representation of the value.
If a span is related to a business event, the IDs of the business events are available as span events. The span events have the span_event.name set to bizevent.
event.id5547782627070661074_1647601320000span_event.namebizevent for bizevent events.bizeventSemantic conventions for well known span links.
A span link by span.id and trace.id references to a target span on a different trace.
span.idspan.id is an 8-byte ID and hex-encoded if shown as a string.f76281848bd8288csupportability.dropped_attributes_count1supportability.non_persisted_attribute_keys["my_span_attribute", "db.name"]trace.alternate_idtrace.alternate_id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.idtrace.id is a 16-byte ID and hex-encoded if shown as a string.357bf70f3c617cb34584b31bd4616af8trace.statef4fe05b2-bd92206c@dt=fw4;3;abf102d9;c4592;0;0;0;2ee;5607;2h01;3habf102d9;4h0c4592;5h01;6h5f9a543f1184a52b1b744e383038911c;7h6564df6f55bd6eae,apmvendor=boo,foo=barBesides the attributes listed above, arbitrary other attributes are allowed on a generic span link.
A span link by dt.tracing.custom_link.id references another span with a span link to the same dt.tracing.custom_link.id. This span link is used for technologies where full context propagation with span ID and trace ID is not possible, and only a limited amount of information can be propagated between tiers.
The dt.tracing.link.direction defines the hierarchy between two spans having span links with the same dt.tracing.custom_link.id.
dt.tracing.custom_link.id736bd2684696c4a8dt.tracing.custom_link.original_bytesycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==dt.tracing.custom_link.transformed_bytesycXlxUBAQEDee9lm8pBcA8nF5cVAQEBA3nvZZvKQXAPee9lm8s4SAQ==dt.tracing.custom_link.typedt.tracing.custom_link.original_bytes to the dt.tracing.custom_link.transformed_bytes was applied.genericdt.tracing.link.directionoutgoingdt.tracing.link.is_synctrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.timestampdt.tracing.link.direction set to outgoing.1649822520123123123A Dynatrace link provides additional details regarding the parent and child timing on top of span.parent_id.
This data is optional and not provided by all span data sources.
dt.tracing.link.directionoutgoingdt.tracing.link.iddt.tracing.link.is_synctrue indicates that the caller waits on the response. Only available on span links with dt.tracing.link.direction set to outgoing.timestampdt.tracing.link.direction set to outgoing.1649822520123123123A span link by dt.tracing.foreign_link refers to an upstream transaction. This could be a cross-environment link or a cross-product link to a distributed trace in the legacy AppMon product.
Depending on whether the link information was received in binary or text, either dt.tracing.foreign_link.bytes or dt.tracing.foreign_link.text will be set, respectively.
dt.tracing.foreign_link.bytes00000004000000010000000200000003000000040000002300000001dt.tracing.foreign_link.textFW4;129;12;-2023406815;4539717;0;17;66;c511;2h02;3h12345678;4h676767; FW1;129;4711;59959450;-1859959450;3;17;0A span link by dt.tracing.response.headers refers to a downstream transaction. This could be a cross-environment link.
dt.tracing.response.headers{'traceresponse': ['00-7b9e3e4068167838398f50017bfad358-d4ffc7e33530967a-01'], 'x-dt-tracestate': ['9651e1a8-19506b7c@dt']}Semantic conventions for ESB (Enterprise Service Bus) spans. An ESB span holds information about the realm in which the span is produced. This metadata includes, for example, the workflow in which the span is placed and the application or library to which the workflow belongs.
The workflow name is mandatory information and contributes the most to identifying where this span comes from.
The application and library provide a high-level view of which deployment the span belongs to.
It's also worth mentioning that application and library are typically seen as mutually exclusive, although this is not enforced in any way."
esb.application.namemyBusinessApp; YourServiceApp; any_workesb.library.namemyWebServicesLib; YourMessagingLibrary; any_toolsesb.vendoribm; tibcoesb.workflow.is_subprocessfalseesb.workflow.namemyMessageFlow; YourBusinessWorkflow; any_flowRecord containing fields that can be expected for a failure detection result. These results are part (as a record array) of a set of fields used by the failure detection feature and are referenced there via failure_detection.results.
This record relates to failure detection v1 and v2.
custom_rule_namecustom_rule reason.Fail on my.failure.attribute==failedexception_ids[123423523456, 523463467234]reasonexceptionrequest_attribute_namecustom_error_rule reason.my special method count RAverdictfailurereason MUST be one of the following:
custom_error_rulerequest_attribute_name field.custom_rulecustom_rule_name field.exceptiongrpc_codehttp_codespan_statusverdict MUST be one of the following:
failuresuccessFields that can be expected from serverless functions or Function as a Service (FaaS) on various cloud platforms. There are general attributes and attributes specific to incoming FaaS invocations (server side) and outgoing invocations (FaaS function as client calling out).
azure.class_nameHost.Functionsazure.site_namedt-function-scriptedcloud.account.id111111111111; opentelemetrycloud.provideralibaba_cloudcloud.regionus-east-1cloud.resource_idarn:aws:lambda:REGION:ACCOUNT_ID:function:my-function; //run.googleapis.com/projects/PROJECT_ID/locations/LOCATION_ID/services/SERVICE_ID; /subscriptions/<SUBSCIPTION_GUID>/resourceGroups/<RG>/providers/Microsoft.Web/sites/<FUNCAPP>/functions/<FUNC>faas.max_memoryfaas.namemy-function; myazurefunctionapp/some-function-name; test_functionfaas.version14; 254The prefix of the service matches the one specified in cloud.provider.
This is the name of the function as configured/deployed on the FaaS platform and is usually different from the name of the callback
function (which may be stored in the code.namespace/code.function span attributes).
Value of the field depends on a cloud provider. This field is not set for Azure.
This section describes incoming FaaS invocations as they are reported by the FaaS instance itself.
For incoming FaaS spans, the span.kind is either server or consumer.
aws.request_idx-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).0e7bc729-a468-57e8-8143-98f2eec5c925aws.xray.trace_idx-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678faas.coldstartfaas.document.collectionmy-coll-namefaas.document.namemy-file.jpg; 63eeb6e7d418cd98afb1c1d7faas.document.operationdeletefaas.document.time2020-03-08T00:30:12.456Zfaas.event.__key____key__ attribute in a Faas event represents the precise attribute name as received in the event. For example, it might be "faas.event.StackId" for the "StackId" attribute in an AWS CloudFormation event or "faas.event.IdentityPoolId" for the "IdentityPoolId" attribute in an AWS Cognito event. The value of this attribute is identical to the value received in the event.arn:aws:cloudformation:us-west-2:123456789012:stack/MyStack/1a2b3c4d-5678-90ab-cdef-EXAMPLE11111; eu-west-1:12345678-1234-1234-1234-123456789012faas.event_nameObjectCreated:Put (aws:s3); INSERT (aws:dynamodb)faas.event_sourceaws:cloudwatch; aws:cloudformationfaas.triggerdatasourceRelevant only for faas.trigger=datasource trigger
Relevant only for faas.trigger=datasource trigger
Relevant only for faas.trigger=datasource trigger
The value of this attribute is specific to the service that generated the event.
This section describes outgoing FaaS invocations as they are reported by a client calling a FaaS instance.
For outgoing FaaS spans, the span.kind is either client or producer.
aws.request_idx-amzn-requestid, x-amzn-request-id, or x-amz-request-id HTTP header, awsRequestId field in AWS lambda context object).0e7bc729-a468-57e8-8143-98f2eec5c925aws.xray.trace_idx-amzn-trace-id HTTP header, _X_AMZN_TRACE_ID environment variable on AWS lambda)Root=1-63441c4a-abcdef012345678912345678; Self=1-63441c4a-12456789abcdef012345678;Root=1-67891233-abcdef012345678912345678faas.invoked_namemy-functionfaas.invoked_providercloud.provider resource attribute.alibaba_cloudWill be equal to the invoked function's cloud.region resource attribute.
Semantic conventions for HTTP client and server spans. They can be used for HTTP and HTTPS schemes and various HTTP versions like 1.1, 2, and SPDY.
The common fields listed in this section apply to both HTTP clients and servers in addition to the specific fields listed in the HTTP client and HTTP server sections below.
http.request.body.size3495http.request.header.__key____key__ being the lowercase HTTP header name, for example "http.request.header.accept-encoding". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.sensitive-spanshttps://www.foo.bar/; gzip, deflate, br; 1.2.3.4, 1.2.3.5http.request.methodGET; POST; HEADhttp.response.body.size3495http.response.header.__key____key__ being the lowercase HTTP header name, for example, "http.response.header.content-type". The value is a string. If multiple headers have the same name or multiple header values, the values will be comma-separated into a single string.909; text/html; charset=utf-8; abc, defnetwork.protocol.version1.1; 3.1.1This span type represents an outbound HTTP request.
For an HTTP client span, span.kind is client.
server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]url.fullsensitive-spanshttps://www.foo.bar/docs/search?q=OpenTelemetry#SemConvThis span type represents an inbound HTTP request.
For an HTTP server span, span.kind MUST be server.
client.ipsensitive-spans194.232.104.141; 2a01:468:1000:9::140http.route/users/:userID?; Home/Index/{id?}server.addressexample.comserver.port65123; 80url.path/docs/searchurl.querysensitive-spansq=OpenTelemetryurl.schemehttps; ftp; telnetmessaging.batch.message_count1; 2; 3messaging.client.idaclient; myhost@68d46b89c9-c29qcmessaging.message.body.size2738messaging.message.conversation_idMyConversationIdmessaging.message.header.__key____key__ being the message header/attribute name, for example, "messaging.message.header.ExtendedPayloadSize". The data type of the value depends on the attribute.1024, "my-eu-bucket-3", ["a", "b"]messaging.message.id452a7c7c7c7048c2f887f61572b18fc2messaging.operation.typepeekmessaging.systemkafka; rabbitmqserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]If a custom value is used for messaging.operation.type, it MUST be of low cardinality.
Only necessary when the server.* attributes don't apply.
This should be the IP/hostname of the broker (or other network-level peer) this specific message is sent to/received from.
messaging.operation.type has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
peekprocesspublishreceivemessaging.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
activemqartemisaws_eventbridgeaws_snsaws_sqsazure_eventgridazure_eventhubsazure_servicebusgcp_pubsubhornetqjmskafkamqseriesmsmqrabbitmqrocketmqsag_webmethods_istibco_emsweblogicwebspherenetwork.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
A producer of a message should set the span kind to producer unless it synchronously waits for a response; then it should use client.
The following additional attributes describe message producer operations.
messaging.destination.kindqueue; topicmessaging.destination.temporaryManager name SHOULD uniquely identify the broker.
Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker.
messaging.destination.kind MUST be one of the following:
queuetopicThe consumer of a message should set the span kind to consumer unless it consistently sends back a reply that is directed to the producer of the message (as opposed to e.g., a queue on which the producer happens to listen): then it should use server.
The following additional attributes describe message consumer operations.
messaging.source.kindqueue; topicmessaging.source.temporaryManager name SHOULD uniquely identify the broker.
Source name SHOULD uniquely identify a specific queue, topic, or other entity within the broker.
messaging.source.kind MUST be one of the following:
queuetopicSender side via ActorRef.tell() or ActorSelection.tell() is represented by a span, with span.kind set to producer.
messaging.akka.actor.path/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$amessaging.akka.actor.systemRequesterSystem; ResponseSystemmessaging.akka.message.typejava.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessagemessaging.message.body.size2738server.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]The receiver side via ActorCell.invoke() (inproc) or Actor.receive() (tcp) is represented by a span, with span.kind set to consumer.
messaging.akka.actor.kindsystem; usermessaging.akka.actor.path/system/log1-Logging$DefaultLogger; /remote/akka.tcp/RequesterSystem@localhost:52133/user/requestActor/$amessaging.akka.actor.systemRequesterSystem; ResponseSystemmessaging.akka.actor.typecom.acme.RespondingActormessaging.akka.message.typejava.lang.String; akka.event.Logging$Info2; com.acme.twosuds.ResponseActor$RequestMessagemessaging.message.body.size2738network.transportnetwork.transport is set to inproc; for Akka remoting, it's set to tcp.inproc; tcpserver.addressexample.comUnavailable in case of network.transport = tcp
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = tcp
Unavailable in case of network.transport = inproc
Unavailable in case of network.transport = inproc
network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.
inprocotherpipetcpudpunixSignals that there is only in-process communication not using a "real" network protocol in cases where network attributes would typically be expected. Usually, all other network attributes can be left out.
This convention extends the default semantic convention for messaging systems.
messaging.destination.partition.id1messaging.kafka.message.keykey property of the message.mykeymessaging.kafka.message.tombstonetruemessaging.kafka.offset42If the message is a tombstone, the value is true. When missing, the value is assumed to be false.
messaging.consumer.group.namemy-group; indexermessaging.destination.partition.id1messaging.kafka.message.keykey property of the message.mykeymessaging.kafka.message.tombstonetruemessaging.kafka.offset42Kafka consumer group ID
If the message is a tombstone, the value is true. When missing, the value is assumed to be false.
Fields that describe remote procedure calls (also called "remote method invocations" / "RMI") with spans.
A remote procedure call is described by two separate spans, one on the client and one on the server side.
For outgoing requests, the SpanKind MUST be set to client and for incoming requests to server.
network.protocol.namerpc.system. See below for a list of well-known identifiers.grpc; rest_http; soap; dotnet_remoting; hessian; java_rmi; json_rpcrpc.namespacetempuri.orgrpc.servicemyservice.EchoServicerpc.systemapache_cxf; dotnet_wcf; grpc; jax_wsserver.addressexample.comserver.port65123; 80server.resolved_ipsserver.address.[194.232.104.141, 2a01:468:1000:9::140]This is the logical name of the method from the RPC interface perspective, which can be different from the name of any implementing method/function. The code.function attribute may be used to store the latter (e.g., method executing the call on the server side, RPC client stub method on the client side).
This is the logical name of the service from the RPC interface perspective, which can be different from the name of any implementing class. The code.namespace attribute may be used to store the latter (despite the attribute name, it may include a class name, e.g., class with method executing actually executing the call on the server side, RPC client stub class on the client side).
This span type represents an inbound RPC request.
For an RPC server span, SpanKind is Server.
code.functionserveRequestcode.invoked.functioncode.function, only it represents the function that was active when a span has been started. Typically, it's the function that has been instrumented. The spans duration does not reflect the duration of this function execution. It should only be set if it differs from code.function.invokecode.invoked.namespacecode.namespace, only it represents the namespace of the function that was active when a span has been started. Typically, it's the function that has been instrumented. It should only be set if it differs from code.namespace.com.sun.xml.ws.server.InvokerTube$2code.namespacecode.function is defined. Usually, the qualified class or module name, such that code.namespace + some separator + code.function forms a unique identifier for the code unit. 2com.example.MyHttpServiceIn the case of RPC, code.function represents the handler function that processes the RPC.
In the case of RPC, code.namespace represents the namespace of the handler function that processes the RPC.
Additional conventions for remote procedure calls via [gRPC][].
rpc.framework and rpc.protocol MUST be set to "grpc".
Additional conventions for remote procedure calls via [RMI][].
rpc.framework and rpc.protocol MUST be set to "java-rmi".
rpc.rmi.registryCalculatorz/OS Connect EE (Enterprise Edition) is an IBM product that exposes applications and data in z/OS subsystems such as CICS, IMS, or MQ via a RESTful API. The z/OS Connect EE (v3.0) implementation is built on the WebSphere Application Server for the z/OS Liberty profile.
zosconnect.api.descriptionThe API for the CICS catalog manager sample application.zosconnect.api.namecatalogzosconnect.api.version1.0.0zosconnect.request.body.size234zosconnect.request.id2215zosconnect.response.body.size125zosconnect.service.descriptionEDUCHAN service using the CICS Service Providerzosconnect.service.nameplaceOrderzosconnect.service.provider.nameCICS-1.0zosconnect.service.version2.0zosconnect.sor.identifierlocalhost:8080zosconnect.sor.referencecicsConnzosconnect.sor.resource01,DFH0XCMNzosconnect.sor.typeCICSOnly applicable if zosconnect.sor.type is CICS
zosconnect.request.type MUST be one of the following:
ADMINAPISERVICEUNKNOWNzosconnect.sor.type MUST be one of the following:
CICSIMSMQRESTWOLAThe semantic conventions for z/OS client and server spans. You can use them for CICS and IMS schemes.
The common attributes listed in this section apply to both CICS and IMS clients and servers, in addition to the specific attributes listed in the CICS common, CICS client, and CICS server sections below.
zos.transaction.call_typeCTGzos.transaction.job_nameCICSAOR0; CTGATM00; IMSCR15zos.transaction.lpar_nameS0W1; ABCDcics.transaction.system_idC259; CICScics.transaction.task_id1234This span type represents an outbound CICS request.
For a CICS client span, span.kind is client.
cics.transaction.system_idC259; CICSzos.transaction.lpar_nameS0W1; ABCDThis span type represents an inbound CICS request.
For a CICS server span, span.kind MUST be server.
cics.transaction.system_idC259; CICSzos.transaction.lpar_nameS0W1; ABCD