Generic log ingestion (Logs Classic)

Log Monitoring Classic

Dynatrace automatically collects log and event data from a vast array of technologies. With generic log ingestion, you can stream log records to a system and have Dynatrace transform the stream into meaningful log messages.

Generic log ingestion allows you to stream log records to the system. It is available via Log Monitoring API - POST ingest logs for JSON and text format or the OTLP endpoint for OTLP binary protobuf format.

  • For Dynatrace SaaS, the generic logs ingest endpoint is available in your environment.

  • If the Environment ActiveGate is your choice for an endpoint in your local environment, install an ActiveGate instance:

    In Dynatrace Hub, select ActiveGate > Set up. The generic log ingest API v2 is automatically enabled on ActiveGate. ActiveGate is responsible for serving the endpoint, collecting the data, and forwarding it to Dynatrace in batches.

  • Saas endpoints:
    • https://{your-environment-id}.live.dynatrace.com/api/v2/logs/ingest
    • https://{your-environment-id}.live.dynatrace.com/api/v2/otlp/v1/logs
  • Environment ActiveGate endpoints:
    • https://{your-activegate-domain}:9999/e/{your-environment-id}/api/v2/logs/ingest
    • https://{your-activegate-domain}:9999/e/{your-environment-id}/api/v2/otlp/v1/logs

Ingest endpoint will collect and attempt to automatically transform any log data containing the following JSON elements:

  • Log content
  • Timestamp
  • Key-Values attributes

To view all predefined key-value attributes, such as the supported semantic attribute keys, check Log Monitoring API - POST ingest logs.

Log data queue on Environment ActiveGate

You can customize the log data queue properties by editing the custom.properties file (see Configuration properties and parameters of ActiveGate) on your ActiveGate to set the following values:

[generic_ingest]
#disk_queue_path=<custom_path> # defaults to temp folder
#disk_queue_max_size_mb=<limit> # defaults to 300 MB
503 Usable space limit reached

The log data ingestion API returns a 503 Usable space limit reached error when the ingested log data exceeds the configured queue size. Typically, this is a temporary situation that occurs only during spikes. If this error persists, increase the value of disk_queue_max_size_mb in custom.properties to allow log ingestion spikes to be queued.

Example

In this example, the API request ingests log data that will create a log event with defined log attributes content, status, service.name, and service.namespace.

The API token is passed in the Authorization header.

The response contains response code 204.

Curl

curl -X POST \
   https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest \
  -H 'Content-Type: application/json; charset=utf-8' \
  -H 'Authorization: Api-Token dt0c01.abc123.abcdefjhij1234567890' \
  -d '[
    {
    "content": "Exception: Custom error log sent via Generic Log Ingest",
    "status": "error",
    "service.name": "log-monitoring-tenant",
    "service.namespace": "dev-stage-cluster"
    }
]'

Request URL

https://environment.activegate.domain.com:9999/e/abc123a/api/v2/logs/ingest

Response content

Success

Response code

204

Related topics