Syslog ingestion
ActiveGate version 1.295+
Syslog, short for system logging protocol, is a logging mechanism that enables system administrators to oversee and control log files from various system components, such as network devices, Linux host syslog, syslog servers, or other syslog producers.
This guide shows you how to configure your Environment ActiveGate on Linux to collect syslog logs in your network and ingest them to Dynatrace.
Prerequisites
- Environment ActiveGate version 1.295+ on Linux installed to monitor remote technologies.
- Your network devices have the syslog enabled or you have other syslog producers configured in your network. Refer to RFC3164 and RFC5424 for details. Dynatrace supports a wide variety of syslog implementations, including RSysLog, Syslog-NG, NXLog, and others.
Who is it for?
This guide is intended for network and Dynatrace admins who are tasked to enable the syslog log ingestion into Dynatrace.
Enable syslog ingestion
Enabling syslog log ingestion requires you to:
- Deploy Environment ActiveGate in a place ensuring the connectivity between ActiveGate and monitored devices.
- Enable syslog ingestion on ActiveGate.
- optional in some cases, you'll need to adapt the default syslog receiver configuration.
-
Deploy Environment ActiveGate.
See instructions for Linux. Use the remote technologies monitoring purpose.
-
Enable syslog ingestion on your ActiveGate.
Edit the
/var/lib/dynatrace/remotepluginmodule/agent/conf/extensionsuser.conf
file and add the following flag:syslogenabled=true -
optional Edit the syslog receiver configuration.
ActiveGate uses an embedded Dynatrace OpenTelemetry Collector instance and stores the receiver configuration in the
/var/lib/dynatrace/remotepluginmodule/agent/conf/syslog.yaml
file. The Collector is installed by default.Use this configuration only for syslog ingestion.
If your syslog producers use the default ports per supported protocols, your syslog-enabled ActiveGate should receive syslog records right away.
You only need to modify the configuration if your syslog producers cast events on custom ports.
receivers:syslog/udp:udp:listen_address: '0.0.0.0:514'add_attributes: trueprotocol: rfc5424operators:- type: syslog_parserprotocol: rfc5424syslog/tcp:tcp:listen_address: '0.0.0.0:601'add_attributes: trueprotocol: rfc5424operators:- type: syslog_parserprotocol: rfc5424# syslog/tcp_tls:# tcp:# listen_address: "0.0.0.0:6514"# tls:# cert_file: "/absolute/path/to/server.crt"# key_file: "/absolute/path/to/server.key"# protocol: rfc5424# operators:# - type: syslog_parser# protocol: rfc5424#DO.NOT.MODIFYexporters:otlphttp/syslog: ${file:syslogendpoint.yaml}processors:batch:send_batch_size: 512send_batch_max_size: 1024transform:log_statements:- context: logstatements:- set(body, attributes["message"])attributes:actions:- key: net.host.nameaction: delete- key: net.peer.nameaction: delete- key: net.peer.portaction: delete- key: net.transportaction: delete- key: net.host.ipaction: delete- key: dt.ingest.portfrom_attribute: net.host.portaction: upsert- key: dt.ingest.source.ipfrom_attribute: net.peer.ipaction: upsert- key: net.peer.ipaction: delete- key: net.host.portaction: delete- key: syslog.hostnamefrom_attribute: hostnameaction: upsert- key: hostnameaction: delete- key: syslog.facilityfrom_attribute: facilityaction: upsert- key: facilityaction: delete- key: syslog.priorityfrom_attribute: priorityaction: upsert- key: priorityaction: delete- key: syslog.proc_idfrom_attribute: proc_idaction: upsert- key: proc_idaction: delete- key: syslog.versionfrom_attribute: versionaction: upsert- key: versionaction: delete- key: syslog.appnamefrom_attribute: appnameaction: upsert- key: appnameaction: delete- key: messageaction: deleteservice:telemetry:metrics:level: nonepipelines:logs/udp:receivers: [syslog/udp]processors: [transform, attributes, batch]exporters: [otlphttp/syslog]logs/tcp:receivers: [syslog/tcp]processors: [transform, attributes, batch]exporters: [otlphttp/syslog]# logs/tcp_tls:# receivers: [syslog/tcp_tls]# processors: [transform, attributes, batch]# exporters: [otlphttp/syslog]You can also modify the default configuration if you want to group a set of various devices by configuring them to use a specific port. For example, you can enrich your syslog events cast on specific TCP ports using the configuration as in the example below
receivers:syslog/f5:tcp:listen_address: "0.0.0.0:54526"protocol: rfc5424operators:- type: addfield: attributes.log.sourcevalue: syslog- type: addfield: attributes.dt.ip_addressesvalue: "1xx.xx.xx.xx1"- type: addfield: attributes.instance.namevalue: "ip-1xx-xx-x-xx9.ec2.internal"- type: addfield: attributes.device.typevalue: "f5bigip"syslog/host:tcp:listen_address: "0.0.0.0:54527"protocol: rfc5424operators:- type: addfield: attributes.log.sourcevalue: syslog- type: addfield: attributes.device.typevalue: "ubuntu-syslog"Note: Do NOT modify the exporter configuration. The default configuration points to the embedded Collector.
For more information on syslog receiver configuration, see Ingest syslog data.
-
Verify the syslog ingestion is enabled.
After you enable syslog ingestion, check the following log files to verify it:
Open the newest
ruxit_extensionmodule_*.log
log file in theextensions
log directory:- Linux:
/var/lib/dynatrace/remotepluginmodule/log/extensions
It should contain the following line:
Otel syslog enabled: true - Linux:
-
Enable syslog on the devices you want to monitor.
The way you enable syslog depends on the device and its platform, refer to specific documentation for details.
Example Configure Rsyslog on Linux Ubuntu to forward syslog logs to a remote server.
Add the following line to the syslog daemon configuration file (
/etc/rsyslog.conf
)- UDP
*.* @<ActiveGate host IP>:514
- TCP
@@ @<ActiveGate host IP>:601
*.*
or@@
instruct the daemon to forward all messages to the specified ActiveGate listening on the provided port and IP address.<ActiveGate host IP>
needs to point to the IP address of a syslog-enabled ActiveGate.For more examples, see Syslog via OpenTelemetry Collector
- UDP
-
Verify ActiveGate receives the syslog events.
After your syslog producers start to cast log records, open the latest
dynatracesourceotelcollector.*.log
file in/var/lib/dynatrace/remotepluginmodule/agent/datasources/otelSyslog
.If ActiveGate receives the log records you should see entries as in the example below:
[otelSyslog][otelSyslog][37448][err]LogRecord #3[otelSyslog][oteiSyslog][37448][err]ObservedTimestamp: 2024-05-06 @9:52:10.6748723 +8000 UTC[otelSyslog][otelSyslog][37448][err]Timestamp: 2624-05-@6 11:52:16 +90e0 UTC[otelSyslog][otelsyslog][37448][err]SeverityText: info[otelSyslog][otelSyslog][37443][err]SeverityNumber: Info(9)[otelSyslog][otelSyslog][37448][err]Body: Str(<30>May 6 11:52:10 SOME-HOST systemd[1]: Finished Load Kernel Module fuse.)[otelSyslog][otelSyslog][37448][err]Attributes:[otelSyslog][otelSyslog][37448][err] -> priority: Int(3)[otelSyslog][otelSyslog][37448][err] -> facility: Int(3)[otelSyslog][otelSyslog][37448][err] -> appname: Str(systemd)[otelSyslog][otelSyslog][37448][err] -> proc_id: Str(1)[otelSyslog][otelSyslog][37443][err] -> log: Map({“source": “syslog"})[otelSyslog][otelSyslog][37443][err] -> hostname: Str(SOME-HOST)[otelSyslog][otelSyslog][37443][err] -> message: Str(Finished Load Kernel Module fuse.)[otelSyslog][otelSyslog][37448][err]Trace ID:[otelSyslog][otelSyslog][37448][err]Span ID:[otelSyslog][otelSyslog][37443][err]Flags: 0
For more information on troubleshooting the syslog receiver, see Collector troubleshooting.
Next steps
After you enable the integration, the syslog-ingested events are enriched with the host-specific attributes and become available in Grail for Davis® AI-based data analysis, log processing, or querying via DQL.