This page explains how the OneAgent Log module operates, how it reads, buffers, and delivers logs, and the resilience and connectivity mechanisms behind it.
On Kubernetes, the Dynatrace Operator deploys and manages a single log module.
The operation, buffering, and connectivity mechanisms described on this page apply to both forms. For deployment options, supported platforms, and installation, see Kubernetes log monitoring.
The Log module reads log files, extracts relevant data (timestamps, severity, content, topological context), and prepares the logs for transmission. It handles common log file rotation techniques and auto-discovers log files.
On Kubernetes, a OneAgent deployed on a node in Full-Stack mode can access node and OS logs. For containers, the Log module captures logs written to the stdout and stderr streams. Kubernetes stores these streams as files on the node, while the Log module picks them up and streams them to Dynatrace. For logs written inside a container filesystem, use a log shipper sidecar to redirect them to stdout/stderr so the Log module can pick them up.
For details about remote log ingest configuration, see Log ingest rules.
If a log file rotation compresses the rotated file immediately, the Log module can't read it. Immediate compression on rotation is an atypical setup and should be avoided.
The Log module provides built-in self-monitoring and troubleshooting functionality, called Log module self-monitoring events (SFM events). See Monitor log source health with SFM events for details.
A dedicated OneAgent process monitors the Log module's operations; if the module doesn't respond within 15 minutes, the process restarts the module. If a restart fails, the delay before the next attempt doubles: starting at ten seconds and capping at one hour. This delay resets after the Log module has run successfully for 20 minutes. There's no limit on the number of restart attempts.
OneAgent opens log files and reads up to 10 MB of log data at a time. After reading, the file handle is immediately closed and the logs are processed. This procedure repeats until one of the following conditions is met.
The polling interval is 60 seconds. During one interval, OneAgent opens log files and reads up to 10 MB of log data at a time. After reading, the file handle is immediately closed and the logs are processed.
The Log module checks for log file rotations every five minutes, but detects a rotation immediately if it occurs while a file is being read.
If a log file appears and disappears within the 60-second polling interval, the Log module might not report it. With OneAgent version 1.311+, the file handle is kept open until all logs are parsed.
On Kubernetes, the Log module prolongs the lifetime of container log files by keeping their file handles open for up to five minutes. This lets it finish reading logs from short-lived containers and from containers that are winding down, even after the container itself is gone.
If your use case requires shorter polling intervals, contact Dynatrace support.
To ensure reliable log delivery, the OneAgent Log module uses a dual-layer buffering system:
The persistent storage ensures resilience across restarts of either the Log module or the host. It stores logs in a compressed format with metadata and only contains processed and enriched data from the last processing interval. Data is deleted from the persistent storage as soon as your Dynatrace environment acknowledges receipt.
If your Dynatrace environment or the Environment ActiveGate becomes temporarily unreachable, the Log module stops reading new logs; it resumes reading once connectivity is restored. During the outage, the original log files act as the buffer—on Kubernetes, these are the container log files stored on the node. If these files are rotated or deleted before connectivity is restored, the data is lost permanently.
For example, a Kubernetes cluster temporarily loses connectivity to its Environment ActiveGates: Buffered and in-flight logs are delivered once the connection recovers, while logs from files that were rotated or removed during the outage are lost. On Kubernetes, keeping container log file handles open for up to five minutes (see Polling intervals) reduces—but doesn't eliminate—this risk for short-lived containers.
Short outages are generally recoverable. If connectivity problems persist, the underlying issue must be resolved before log delivery can catch up.
During a rapid shutdown, the Log module attempts to send logs before going offline. If the host doesn't restart, some data may be lost.
The following limits are optimized for resilient, large-scale operations. Modifying them requires support consultation and use case analysis.
| Type | Default | Description |
|---|---|---|
Storage capacity | 2 GB | Compressed logs, including metadata. |
Log processing rate | 500 MB/min | Can be increased to 1 GB/min if resources allow. |
The Log module supports two transmission paths:
Network zones enforce the connectivity order for OneAgents. See Network zones for details.
Buffered log messages are sent with unique message IDs, and the Log module waits for an acknowledgment (ACK) to confirm delivery. If an ACK isn't received, the Log module resends the message using an exponential backoff algorithm. Each message is independently verified for acknowledgment, ensuring reliable delivery even during simultaneous log transmissions. There's no retry limit; however, log records older than the log age limit are removed from the sending queue.
Log records with a timestamp older than 2 hours don't produce log-based metric points.
When an Environment ActiveGate is used as a routing component, it acts as a pass-through for logs received from the Log module. These are immediately forwarded to your Dynatrace environment without being retained locally on the ActiveGate. If your environment is temporarily unreachable, logs aren't stored on the ActiveGate for later delivery.
The Log module includes the following reliability mechanisms for ActiveGate communication:
ACK for each message sent to the ActiveGate. If an ACK isn't received in time, the message is resent.ACK and exponential backoff techniques to handle scenarios where it crashes or restarts during transmission.When log throughput produced by a host exceeds what Dynatrace can ingest (whether due to network limitations between the host and the environment, ActiveGate routing constraints, or limits of the target environment), the Log module falls behind: acknowledgments are delayed, unacknowledged records queue up, and the original log files act as the buffer. If the log files are rotated or deleted before the Log module catches up, data may be lost.