Automatic log enrichment

Dynatrace enables you to transform logs ingested via OneAgent.

Transform the OneAgent-ingested logs

During log ingest via OneAgent, the severity of logs is determined.

Log severity

By default, the log event severity is detected through a keyword search performed on the first 100 characters of the log content, within the first two lines of text.

To adjust these limits

In the Dynatrace menu, go to Settings.
Select Log Monitoring > Advanced log settings.
Adjust the following settings as needed.
- Severity search chars limit is the number of characters in each log line, starting from the first character, to search for severity.
- Severity search lines limit is the number lines in each log entry, starting from the first line, to search for severity.

There are 19 keywords that correspond with 9 severity levels as per the table below:

Keyword	Severity level
trace	DEBUG
debug	DEBUG
fine	DEBUG
finer	DEBUG
finest	DEBUG
notice	NOTICE
info	INFO
information	INFO
warn	WARN
warning	WARN
severe_warning	WARN
severe	SEVERE
err	ERROR
error	ERROR
crit	CRITICAL
critical	CRITICAL
alert	ALERT
fatal	EMERGENCY
emerg	EMERGENCY

A match occurs and severity is determined when

The keyword found is a single word/phrase from the above list, and it is preceded and followed by a space.
The keyword found is a single word/phrase from the above list, and it is preceded and followed by one of the four predefined non-alphanumeric symbols, as in the example below:

[error]
{error}
{{error}}
<error>

Transform all types of logs

Additionally, for each log event, a status attribute is created with a value that is a sum of loglevel values based on the following grouping:

Included `loglevel` values	Combined `status` attribute value
`SEVERE`, `ERROR`, `CRITICAL`, `ALERT`, `FATAL`, `EMERGENCY`	`ERROR`
`WARN`	`WARN`
`INFO`, `TRACE`, `DEBUG`, `NOTICE`	`INFO`
`NONE`	`NONE`

For example: The level severity key in the generic log ingestion API request parameter contains the value serious.

The level severity key is transformed into the loglevel attribute with the serious value mapped to SEVERE based on the above table.
The loglevel attribute containing the SEVERE value is grouped into status attribute. Based on the grouping table above, the status attribute will contain the ERROR value.
For the log event details, the log viewer will report the following:

status - ERROR
loglevel - SEVERE

Attributes added during a log ingest via OneAgent

During the log ingestion via OneAgent, the following attributes are added automatically:

General attributes (via OneAgent)

container.name
container.image.name
container.id
dt.host_group.id
dt.kubernetes.cluster.id
dt.kubernetes.cluster.name
dt.kubernetes.node.system_uuid
dt.process.name
event.type
host.name
k8s.cluster.name
k8s.namespace.name
k8s.pod.name
k8s.pod.uid
k8s.container.name
k8s.deployment.name
log.iostream
loglevel
log.source
process.technology
span_id
status
trace_id
web_server.iis.site_id
web_server.iis.site_name
web_server.iis.application_pool

dt entity model attributes (via OneAgent)

dt.entity.cloud_application
dt.entity.cloud_application_instance
dt.entity.cloud_application_namespace
dt.entity.container_group
dt.entity.container_group_instance
dt.entity.host
dt.entity.kubernetes_cluster
dt.entity.kubernetes_node
dt.entity.process_group
dt.entity.process_group_instance
dt.source_entity

Attributes automatically extracted from log content via OneAgent

OneAgent automatically extract attributes found in form \[!dt key1=value1, key2=value2] and the section itself is removed from content.

For instance:

1127.0.0.1 - [21/Oct/2021:10:33:28 +0200] GET /index.htm HTTP/1.1 404 597 [!dt dt.trace_id=aa764ee37ebaa764ee37eaa764ee37e, dt.span_id=b93ede8b93ede8]

will result in additional dt.trace_id and dt.span_id attributes for log record and actuall content sent will be:

1127.0.0.1 - [21/Oct/2021:10:33:28 +0200] GET /index.htm HTTP/1.1 404 597