Syslog ingestion with ActiveGate

• 8-min read
• Published Jun 18, 2024

ActiveGate version 1.295+ recommended

Syslog, short for system logging protocol, is a logging mechanism that enables system administrators to oversee and control log files from various system components, such as network devices, Linux host syslog, syslog servers, or other syslog producers.

This guide shows you how to configure your Environment ActiveGate on Linux to collect syslog logs in your network and ingest them to Dynatrace.

Prerequisites

Hardware requirements

Syslog ingestion is performed by an ActiveGate. The syslog ingestion throughput depends on the hardware your ActiveGate is deployed on.

Who is it for?

This guide is intended for network and Dynatrace admins who are tasked to enable the syslog log ingestion into Dynatrace.

Enable syslog ingestion

Enabling syslog log ingestion requires you to:

Mask sensitive data

ActiveGate syslog ingestion supports the OpenTelemetry Transform Processor and OpenTelemetry Transformation Language (OTTL) to process your syslog data at edge, before sensitive data leaves your network.

This way, you can use it to mask or hash sensitive data in your syslog lines, so that no sensitive information is ingested into Dynatrace.

Let's assume your credit card data is visible in syslog as:

<14>2 2024-07-19T14:53:55Z example-host 0OOButHPbR 1234 - - New operation for CreditCard 1234567891011124

To mask a credit card number, add the following configuration under the processors node of the syslog.yaml file:

processors:
   transform/redact_credict_cart:
       log_statements:
         - context: log
           statements:
             - replace_pattern(body, "\\d{15,16}", "REDACTED")

The replace_pattern function replaces the credit card number with the REDACTED string. The credit card number in the content is matched by the body, "\\d{15,16}" pattern.

Add custom attributes

You can also modify the default configuration if you want to group a set of various devices by configuring them to use a specific port. For example, using very generic log messages, you can enrich your syslog events cast on specific TCP ports with custom attributes using the configuration as in the example below.

receivers:
 syslog/f5:
    tcp:
      listen_address: "0.0.0.0:54526"
    protocol: rfc5424
    operators:
      - type: add
        field: attributes.log.source
        value: syslog
      - type: add
        field: attributes.dt.ip_addresses
        value: "1xx.xx.xx.xx1"
      - type: add
        field: attributes.instance.name
        value: "ip-1xx-xx-x-xx9.ec2.internal"
      - type: add
        field: attributes.device.type
        value: "f5bigip"
 syslog/host:
    tcp:
      listen_address: "0.0.0.0:54527"
    protocol: rfc5424
    operators:
      - type: add
        field: attributes.log.source
        value: syslog
      - type: add
        field: attributes.device.type
        value: "ubuntu-syslog"

You can also use:

For example, if we can read the net.peer.port attribute, its value is used for custom.remote.port. Otherwise, the custom.report.port isn't set.

attributes:
    actions:
      - key: custom.remote.port
        from_attribute: net.peer.port
        action: upsert

For more information on attributes configuration, see Attributes Processor.

Filter data

You can filter the syslog data to drop irrelevant log lines and reduce your consumption at edge, before the data leaves your network.

For example, assume we want to ignore log lines categorized with syslog facility 21,

<21> 2024-07-19T14:53:55Z example-host 0OOButHPbR 1234 - - Spam mail

Add the following filter to the syslog.yaml file.

filter/mail:
logs:
  log_record:
    - attributes["syslog.facility"] == 21

The log line isn't ingested based on the 21 syslog facility.

Process non-standard syslog with OpenPipeline

Some devices send syslog in a non-standard format that doesn't comply with the RFC3164 and RFC5424 standards.

For example, the Cisco switch series C9300 produces the following non-standard lines:

<189>: Jul 18 07:15:58: %WEBSERVER-5-SESS_TIMEOUT: Switch 1 Session timeout from host 172.31.100.31 by user 'snmptest' using crypto cipher 'TLS_AES_256_GCM_SHA384'

In comparison with the RFC3164 standard, there are two additional colon (:) characters, one after the priority value (PRIVAL) <189> and the other after the timestamp. Additionally, instead of hostname and tag, error description %WEBSERVER-5-SESS_TIMEOUT is provided.

The message format according to the standard is:

<PRIVAL>TIMESTAMP HOSTNAME TAG: MESSAGE

To ingest such lines, after you enable the syslog ingestion, you can transform them with the Dynatrace OpenPipeline to the supported format.

To fix the format from the example above, create a new logs pipeline or use an existing logs pipeline and add the following DQL processor:

When you select Run sample data, the processor returns the following preview matching the standard syslog format:

{
	"results": [
		{
			"matched": true,
			"record": {
				"syslog.facility_type": "SEC_LOGIN",
				"syslog.severity": 5,
				"syslog.mnemonic": "LOGIN_SUCCESS",
				"syslog.priority": 189,
				"loglevel": "INFO",
				"dt.ingest.port": "514",
				"syslog.facility": 23,
				"content": "Login Success [user: snmptest] [Source: 10.107.0.239] [localport: 22] at 09:08:38 UTC Mon Jul 29 2024",
				"timestamp": "2024-07-19T11:49:32.319000000Z",
				"status": "INFO"
			}
		}
	]
}

For general instructions on processors, see Configure a processing pipeline.