Instrument your Elixir application with OpenTelemetry

This walkthrough shows how to add observability to your Elixir application using the OpenTelemetry Elixir libraries and tools.

Feature	Supported
Automatic Instrumentation	No
Automatic OneAgent Ingestion	No

Prerequisites

Dynatrace version 1.222+
For tracing, W3C Trace Context is enabled
1. From the Dynatrace menu, go to Settings > Preferences > OneAgent features.
2. Turn on Send W3C Trace Context HTTP headers.

Get the Dynatrace access details

Determine the API base URL

For details on how to assemble the base OTLP endpoint URL, see Export with OTLP. The URL should end in /api/v2/otlp.

Get API access token

The access token for ingesting traces, logs, and metrics can be generated in your Dynatrace menu under Access tokens.

Export with OTLP has more details on the format and the necessary access scopes.

Set up OpenTelemetry

Add the current versions of the following dependencies to mix.exs.

1defp deps do
2  [
3    {:opentelemetry_api, "~> 1.2.1"},
4    {:opentelemetry, "~> 1.3.0"},
5    {:opentelemetry_exporter, "~> 1.4.1"}
6  ]
7end

Add a release section to the application definition in mix.exs.

1releases: [
2  <project_name>: [
3    version: "<project_version>",
4    applications: [opentelemetry_exporter: :permanent, opentelemetry: :temporary]
5  ]
6]

Enable the context propagation dependencies with the following line in runtime.exs.
```
1text_map_propagators: [:baggage, :trace_context],
```

Add the following configuration to config/runtime.exs and replace [URL] and [TOKEN] with the respective values for the Dynatrace URL and access token.

1import Config
2
3config :opentelemetry,
4  resource: [service: %{name: "elixir-quickstart", version: "1.0.1"}], #TODO Replace with the name and version of your application
5  span_processor: :batch,
6  traces_exporter: :otlp,
7  resource_detectors: [
8    :otel_resource_app_env,
9    :otel_resource_env_var,
10    ExtraMetadata
11  ]
12
13config :opentelemetry_exporter,
14  otlp_protocol: :http_protobuf,
15  otlp_traces_endpoint: "[URL]", #TODO Replace [URL] to your SaaS/Managed URL as mentioned in the next step
16  otlp_traces_headers: [{"Authorization", "Api-Token [TOKEN]"}] #TODO Replace [TOKEN] with your API Token as mentioned in the next step

Save the following code in lib/extra_metadata.ex.

1defmodule ExtraMetadata do
2
3  @behaviour :otel_resource_detector
4
5  def get_resource(_) do
6    metadata = read_file("/var/lib/dynatrace/enrichment/dt_metadata.properties") |> unwrap_lines
7    file_path = read_file("dt_metadata_e617c525669e072eebe3d0f08212e8f2.properties") |> unwrap_lines
8    metadata2 = read_file(file_path) |> unwrap_lines
9    attributes = get_attributes(Enum.concat(metadata, metadata2))
10    metadata3 = read_file("/var/lib/dynatrace/enrichment/dt_host_metadata.properties") |> unwrap_lines
11    attributes = get_attributes(Enum.concat(metadata, metadata2) ++ metadata3)
12    :otel_resource.create(attributes)
13  end
14
15  defp unwrap_lines({:ok, metadata}), do: metadata
16  defp unwrap_lines({:error, _}), do: []
17
18  defp read_file(file_name) do
19    try do
20      {:ok, String.split(File.read!(file_name), "\n")}
21    rescue
22      File.Error ->
23        {:error, "File does not exist, safe to continue"}
24    end
25  end
26
27  defp get_attributes(metadata) do
28    Enum.map(metadata, fn(line) ->
29      if String.length(line) > 0 do
30        [key, value] = String.split(line, "=")
31        {key, value}
32      else
33        {:error, "EOF"}
34      end
35    end)
36  end
37end

Instrument your application

Add tracing

Spans are started with the macro with_span and accept an optional list of span attributes, as well as the code block for this span. The span automatically finishes when the code block returns.

1require OpenTelemetry.Tracer, as: Tracer
2
3def hello do
4  Tracer.with_span "my-span", %{attributes: [{<<"my-key-1">>, <<"my-value-1">>}]} do #TODO add attributes at span creation
5    Tracer.set_attributes([{"another-key-1", "another-value-1"}]) #TODO add attributes after span creation
6    # Your code goes here
7  end
8end

Collect metrics

OpenTelemetry metrics for Elixir are under development.

Connect logs

OpenTelemetry logging for Elixir is under development.

Ensure context propagation optional

Context propagation is particularly important when network calls (for example, REST) are involved.

Extracting the context when receiving a request

To extract information on an existing context, we pass the headers to the otel_propagator_text_map.extract function, which parses the context information provided by the headers and sets the current context based on that.

1#extract headers
2:otel_propagator_text_map.extract(conn.req_headers)
3
4span_ctx = OpenTelemetry.Tracer.start_span(<<"span-name">>)
5ctx = OpenTelemetry.Ctx.get_current()
6
7task = Task.async(fn ->
8    OpenTelemetry.Ctx.attach(ctx)
9    OpenTelemetry.Tracer.set_current_span(span_ctx)
10    # do work here
11
12    OpenTelemetry.Tracer.end_span(span_ctx)
13end)

Injecting the context when sending requests

The following example uses otel_propagator_text_map:inject to provide the HTTP headers (necessary for context propagation) in merged_headers. The headers are then passed to HTTPoison.get, which allows the receiving endpoint to continue the trace with the provided information.

1OpenTelemetry.Tracer.with_span "span-name" do
2    ...
3    # do work here
4    ...
5    headers = [{"content-type", "application/json"}, {"X-Custom-Header", "some-value"}]
6
7    merged_headers = :otel_propagator_text_map.inject(headers)
8
9    case HTTPoison.get(URL, merged_headers, []) do
10        {:ok, res} -> IO.puts("Response: #{inspect(res)}")
11        {:error, _} -> raise "request failed"
12
13    end
14end

Configure data capture to meet privacy requirements optional

While Dynatrace automatically captures all OpenTelemetry attributes, only attribute values specified in the allowlist are stored and displayed in the Dynatrace web UI. This prevents accidental storage of personal data, so you can meet your privacy requirements and control the amount of monitoring data stored.

To view your custom attributes, you need to allow them in the Dynatrace web UI first. To learn how to configure attribute storage and masking, see Attribute redaction.

Verify data ingestion into Dynatrace

Once you have finished the instrumentation of your application, perform a couple of test actions to create and send demo traces, metrics, and logs and verify that they were correctly ingested into Dynatrace.

To do that for traces, in the Dynatrace menu, go to Distributed traces and select the Ingested traces tab. If you use OneAgent, select PurePaths instead.

Metrics and logs can be found under their respective entries at Observe and explore.