This page explains how the OneAgent Log module operates, how it reads, buffers, and delivers logs, and the resilience and connectivity mechanisms behind it.
The Log module reads log files, extracts relevant data (timestamps, severity, content, topological context), and prepares the logs for transmission. It handles common log file rotation techniques and auto-discovers log files.
On Kubernetes, a OneAgent deployed on a node in Full-Stack mode can access node and OS logs. For containers, it monitors stdout and stderr. For logs written inside a container filesystem, use a log shipper sidecar to redirect them to stdout/stderr so the Log module can pick them up. See Stream Kubernetes logs with Dynatrace Log Module for details.
For details about remote log ingest configuration, see Log ingest rules.
If a log file rotation compresses the rotated file immediately, the Log module can't read it. Immediate compression on rotation is an atypical setup and should be avoided.
The Log module provides built-in self-monitoring and troubleshooting functionality, called Log module self-monitoring events (SFM events). See Implement Log module self-monitoring for details.
A dedicated OneAgent process monitors the Log module's operations; if the module does not respond within 15 minutes, the process restarts the module. If a restart fails, the delay before the next attempt doubles: starting at ten seconds and capping at 1 hour. This delay resets after the Log module has run successfully for 20 minutes. There is no limit on the number of restart attempts.
OneAgent opens log files and reads up to 10 MB of log data at a time. After reading, the file handle is immediately closed and the logs are processed. This procedure repeats until one of the following conditions is met.
The polling interval is 60 seconds by default and is not user-configurable. During one interval, OneAgent opens log files and reads up to 10 MB of log data at a time. After reading, the file handle is immediately closed and the logs are processed.
The Log module checks for log file rotations every five minutes, but detects a rotation immediately if it occurs while a file is being read.
If a log file appears and disappears within the 60-second polling interval, the Log module might not report it. With OneAgent version 1.311+, the file handle is kept open until all logs are parsed.
If your use case requires shorter polling intervals, contact Dynatrace support.
To ensure reliable log delivery, the OneAgent Log module uses a dual-layer buffering system:
The persistent storage ensures resilience across restarts of either the Log module or the host. It stores logs in a compressed format with metadata and only contains processed and enriched data from the last processing interval. Data is deleted from the persistent storage as soon as your Dynatrace environment acknowledges receipt.
If your Dynatrace environment or the Environment ActiveGate becomes temporarily unreachable, the Log module stops reading new logs; it resumes reading logs once connectivity is restored. If the original log files on the host are rotated or deleted, the data will be lost permanently.
Short outages are generally recoverable. If connectivity problems persist, the underlying issue must be resolved before log delivery can catch up.
During a rapid shutdown, the Log module attempts to send logs before going offline. If the host doesn't restart, some data may be lost.
The following limits are optimized for resilient, large-scale operations. Modifying them requires support consultation and use case analysis.
| Type | Default | Description |
|---|---|---|
Storage capacity | 2 GB | Compressed logs, including metadata. |
Log processing rate | 500 MB/min | Can be increased to 1 GB/min if resources allow. |
The Log module supports two transmission paths:
Network zones enforce the connectivity order for OneAgents. See Network zones for details.
Buffered log messages are sent with unique message IDs, and the Log module waits for an acknowledgment (ACK) to confirm delivery. If an ACK isn't received, the Log module resends the message using an exponential backoff algorithm. Each message is independently verified for acknowledgment, ensuring reliable delivery even during simultaneous log transmissions. There's no retry limit; however, log records older than the log age limit are removed from the sending queue.
Log records with a timestamp older than 2 hours don't produce log-based metric points.
When an Environment ActiveGate is used as a routing component, it acts as a pass-through for logs received from the Log module. These are immediately forwarded to your Dynatrace environment without being retained locally on the ActiveGate. If your environment is temporarily unreachable, logs aren't stored on the ActiveGate for later delivery.
The Log module includes the following reliability mechanisms for ActiveGate communication:
ACK for each message sent to the ActiveGate. If an ACK isn't received in time, the message is resent.ACK and exponential backoff techniques to handle scenarios where it crashes or restarts during transmission.When log throughput produced by a host exceeds what Dynatrace can ingest (whether due to network limitations between the host and the environment, ActiveGate routing constraints, or limits of the target environment), the Log module falls behind: acknowledgments are delayed, unacknowledged records queue up, and the original log files act as the buffer. If the log files are rotated or deleted before the Log module catches up, data may be lost.