Log Monitoring Classic
Log module self-monitoring events (SFM events) help you detect log ingestion and log source health issues early.
SFM events are turned off by default for Logs Classic.
To enable or customize SFM events, configure the builtin:logmonitoring.log-sfm-settings schema via the Settings API.
You need an access token that has settings.write and settings.read permissions.
To turn on all SFM event types, create a payload.json file with the following content and POST it:
[{"schemaId": "builtin:logmonitoring.log-sfm-settings","scope": "tenant","value": {"enabled": true,"config-item-title": "Enable all SFMs","send-to-storage": true,"matchers": []}}]
curl -X POST \"<environment_url>/api/v2/settings/objects" \-H "Authorization: Api-Token <token>" \-H "Content-Type: application/json" \--data @payload.json
You can configure rules at the following scopes.
Use the broadest scope that fits your needs for simple management.
SFM events fall into two categories with different volumes and cost implications.
Operational issue events: timestamp.*, data_loss.*, ingest.*, and pgi.*.
These warn you about actual Log module problems, such as data loss, missing timestamp patterns, or blocked log sources. They're only generated when something goes wrong, so their volume is typically low.
Log source status events: log_source.status.
These are generated for every log source known to the Log module, including healthy ones. They provide coverage metadata such as file status, ingest status, and log source origin. Because they're generated for every log source on every reporting cycle, they can produce a significantly higher event volume.
If you want to minimize ingestion costs while still detecting issues, enable only the operational issue events by suppressing log_source.status. See Suppress a specific SFM event type.
After you opt in:
In Log Viewer, use the event.type attribute to filter for specific SFM events.
To find all data loss events:
event.type = "data_loss.network"
To find events for a specific log source:
event.type = "log_source.status" AND log.source = "/var/log/app.log"
Additional attributes available on SFM events:
event.type: The specific SFM event type, for example data_loss.network or timestamp.no_pattern.log.source: The log source name or file path.log.source.file_status: File monitoring state, for example OK, Not Exist, or Binary. Available on log_source.status events.log.source.ingest_status: Log content ingestion status, for example Fully ingested or Not ingested. Available on log_source.status events.log.source.origin: Indicates whether the log source was automatically detected or configured as a custom log source. Available on log_source.status events.Use the workflow below to monitor and troubleshoot with SFM events.
log_source.status, timestamp.*, data_loss.*, ingest.*, and pgi.*.data_loss.* and ingest.*) to track trends over time.Start with the following event type families.
data_loss.* for ingestion data loss conditions.ingest.* for ingest pipeline and access-related issues.timestamp.* for timestamp detection and parsing issues.log_source.status for source-level status and coverage context.pgi.* for process-group-related source assignment issues.The following sections describe each SFM event type and the recommended actions.
info)Explanation
It provides additional information and is generated for each log source known to the Log module.
log.source.file_status: Current monitoring state of a log source (for example, OK or File not exists).log.source.ingest_status: Log content ingestion status (for example, Fully ingested, Partially ingested, or Not ingested).log.source.origin: Indicates where the log derives from (for example, Automatically detected or Custom log source).Required actions
Use the file status and ingest status to decide what to do next.
File status OK: No action required.
File status Not Exist, but the file is actually present on the host (this only happens for a custom log source). Check the following:
dtuser), and read+execute privileges on each remote directory in the file path.File status Binary, but the file looks like text. There might still be a portion of binary content inside the file, or there used to be binary content in a log file that is part of the log source, even if such a file has already been removed. Consider the following options:
If it's a custom log source, and some files matching the pattern are binary and you don't want to ingest them, adjust the custom log source pattern, or split the custom log source definition into multiple narrower patterns so that those patterns don't match binary files.
If it's a custom log source, and some files matching the pattern have occasional binary content but you still want to ingest them, turn on Allow binary files in the custom log source definition.
If it's an automatically detected file, create one or more custom log source definitions and adjust them as in the points above.
If you don't want to apply any of the above and the binary content no longer exists (because it was in a file that was deleted) and you don't expect it to appear again, you might consider removing the Log module persistence file. The path to the persistence file (Log Analytics persistence) is described in OneAgent files and disk space requirements on Linux.
Before you delete the persistence file, make sure the Log module isn't running. Deletion might cause some data loss or data duplication around the time of removal.
File status Sensitive Masking Timeout: See the ingest.masking_timeout SFM event type below.
Ingest status isn't as expected: Review your log ingest rules. Typical mistakes are:
/path/a.2026.log doesn't match a log source named /path/a.#.log, even if a file path /path/a.2026.log exists in the filesystem.warning)Explanation
The Log module hasn't detected a timestamp in log content of a log source. In this case, the Log module sets a timestamp from the metadata (if available) or the timestamp when the Log module acquires the log record from the log source.
Required actions
When there's no timestamp:
When the timestamp has a non-standard format, configure the non-standard timestamp format in your log monitoring settings.
info)Explanation
The Log module has detected multiple timestamp patterns being present simultaneously in a single log source.
Required actions
If it's expected, no action is required.
If the Log module incorrectly considers a part of a log message as a log record timestamp, configure the timestamp format explicitly in your log monitoring settings.
error)Explanation
The Log module suspects that the time zone isn't properly detected or configured. For example, the Log module is configured to use the local time zone (for example, UTC+1), while the given log source is writing logs in UTC.
Required actions
The log record timestamps are likely incorrect, and you need to configure the time zone manually in your log monitoring settings.
When working with timestamps, watch for these typical mistakes:
error)Explanation
The Log module was unable to read a log file or a part of it, as the file has been removed, compressed, or moved to another location. This results in data loss.
Required actions
The Log module supports typical log rotation patterns, but it expects the first rotated file to remain uncompressed and in the same directory. It also expects log files to stay available for a short time after they're written. If files are removed or compressed before the Log module can read them, adjust your log rotation settings so that files are kept longer than the Log module read interval.
The Log module read interval can be unusually long if the Log module experiences performance issues. The most frequent risk factors are:
error)Explanation
The Log module was unable to send log records to the Dynatrace environment quickly enough. This results in data loss.
Required actions
The issue might be caused by network connectivity issues between the Log module and the Dynatrace environment. Check your network connectivity health.
If the Log module is configured to ingest a large volume of logs, this might exceed the ingest limits or the environment's performance. In this case, reduce the number of ingested logs.
The Log module itself can also be too slow to send log records in time. The most frequent risk factors are:
warning)Explanation
The Log module can't ingest log records because the Log Content Access flag (--set-app-log-content-access) is turned off.
Required actions
onagentctl to unblock log ingest by setting --set-app-log-content-access=true.error)Explanation
The ingestion of a log source stopped because applying sensitive information masking had timed out.
This issue is also reflected in the log source status SFM event for the log source.
Required actions
Review the masking rules enabled for the log source. There's a high chance the root cause is an improperly written regular expression. For guidance on writing efficient regular expressions, see Regular expressions in Dynatrace. The log source is ingested again after you change the configuration, or after you restart the Log module.
warning)Explanation
Multiple process group instances (PGIs) are assigned to a single log source. This is problematic because, for example, some processing rules might expect that a log record is enriched with a single PGI.
In context of a custom log source, it means that multiple PGIs are configured explicitly.
In context of an automatically detected log source, it can mean one of the following:
Required actions
In context of a custom log source, review the custom log source rule.
In context of an automatically detected log source:
The custom log source split solution from scenario A2 can be applied regardless of whether the initial problem occurred for automatically detected or custom log sources.
error)Explanation
The Log module automatically detected too many log sources for a single process group instance.
Required actions
If you need to ingest these log sources, create a custom log source rule with a wildcard pattern that covers all log files written by the process group instance.
warning)Explanation
Some or all log files detected for a log source have been blocked by security rules.
Required actions
Review the security rules and adjust them to fit your needs.
If you've enabled all SFM events but need to exclude a particular event type, create an additional exclusion rule.
This exclusion rule must have a higher priority than the rule that enables all SFM events. To place the exclusion rule at the top of the rule list, set insertAfter to an empty value in the payload. If you don't include the insertAfter parameter, the rule is placed at the bottom and can be overridden by all other rules within the given scope.
Use the same POST endpoint as in Enable SFM events, but with the following payload. This example suppresses timestamp.no_pattern events across the entire tenant:
[{"schemaId": "builtin:logmonitoring.log-sfm-settings","scope": "tenant","insertAfter": "","value": {"enabled": true,"config-item-title": "Suppress warning about the absence of timestamp pattern","send-to-storage": false,"matchers": [{ "attribute": "event.type", "operator": "MATCHES", "values": ["timestamp.no_pattern"]}]}}]
There are situations when you want to suppress an SFM event type only for one log source you already know about, while still being notified if the same event occurs elsewhere.
For example, suppose one of your log sources, a.log, has no timestamp in its content. You're aware of that and don't want to be notified about it. However, you still want to be notified about the timestamp.no_pattern event for any other log source.
To do this, add a second matcher for log.source:
[{"schemaId": "builtin:logmonitoring.log-sfm-settings","scope": "tenant","insertAfter": "","value": {"enabled": true,"config-item-title": "Suppress 'no timestamp pattern' warning only for a.log","send-to-storage": false,"matchers": [{ "attribute": "event.type", "operator": "MATCHES", "values": ["timestamp.no_pattern"] },{ "attribute": "log.source", "operator": "MATCHES", "values": ["*a.log"] }]}}]
Both matchers must match for the rule to apply, so the event is suppressed only when its type is timestamp.no_pattern and its log source is a.log. The event is still raised for every other log source.
Keep the suppression scope as narrow as needed. A good exclusion rule must match the exact situation you've already investigated and accepted. If you find yourself adding very broad matchers (for example, suppressing an entire event type across an entire tenant), confirm that you're not also hiding events that point to a real issue elsewhere.