AWS Lambda log collection
Dynatrace version 1.263
You can collect logs directly from your AWS Lambda functions and send them to Dynatrace for analysis. The solution is an alternative to the CloudWatch log forwarder with benefits in terms of cost and latency, and is also easier to set up, in particular if AWS Lambda tracing is already in place. As part of the OneAgent installation process, this feature provides a streamlined solution for collecting logs from your Lambda functions.
Prerequisites
- An ActiveGate with the Log analytics collector module enabled. See Log Monitoring API - POST ingest logs.
Deploy
For Python, Node.js, and Java Lambda functions, Dynatrace provides a single Lambda layer that enables both trace and log collection. For .NET, Dynatrace provides a standalone layer that collects only logs.
To deploy the Dynatrace Lambda extension, follow the instructions from Trace Python, Node.js, and Java Lambda functions but with the following two differences:
- Under I want to enable, select Traces and Logs.
- Either select Create token to create a new access token or enter an existing access token with the
logs.ingestpermission into the Access Token input field.
For .NET Lambda functions, follow the steps described in Configuration for monitoring AWS Lambda with OpenTelemetry but with the following two differences:
- Under I want to enable, select Logs.
- Either select Create token to create a new access token or enter an existing access token with the
logs.ingestscope into the Access Token input field.
If you are currently using it, you must disable CloudWatch log forwarding for functions on which you wish to use this log collection feature in order to avoid duplicate log exports. See AWS Log monitoring using log forwarder.
Usage
After deployment, collected logs for each future function invocation and initialization can be found in the Related logs card on the Lambda function's service page in Dynatrace, and in the Log viewer. You can inspect the log details to find the type of the log under the telemetryevent.type attribute, among other metadata. Note that the content of platform logs will be JSON data, while the content of function logs will be plain text.
Configuration
Log event types
Dynatrace collects two log event types, platform events and function logs. To configure which log event types are collected, use the following syntax.
| Configure with | Location | Default value | Syntax |
|---|---|---|---|
JSON file |
|
| Array of strings, for example |
Environment variables |
|
| Colon-separated list of log event types, for example |
Set the value to ["function"] (or function) to collect only function logs.
Endpoint
OneAgent version 1.275+
The endpoint that is used for exporting logs to is derived from the base URL of your configuration deployment screen. You can override the default value by setting the override-value location to a specific full endpoint URL that contains also the path.
| Configure with | Default-value location | Override-value location |
|---|---|---|
JSON file |
| Add the |
Environment variables |
|
|
If you're using an Environment ActiveGate, make sure the environment ID is included in the default value location or set its value to the full URL of the log ingest endpoint (for example, https://{activegate-host}:9999/e/{your-environment-id}/api/v2/logs/ingest).
If you're using Environment ActiveGate (which is usually the case, if you deployed ActiveGate by yourself) and don't have /e/{your-environment-id} as part of your DT_CONNECTION_BASE_URL (Connection.BaseUrl), you need to set either the
DT_LOG_COLLECTION_ENDPOINTenvironment variable- or
LogCollection.EndpointJSON property
to the full URL of the log ingest endpoint (i.e. https://{activegate-host}:9999/e/{your-environment-id}/api/v2/logs/ingest).
The environment ID is typically the tenant ID.
Limitations
- The
extensionevent type currently is not supported: if you try to configure theextensionevent type, an error is shown and log collection does not start. - By default, logs collected by the log collector are not associated with traces and are not shown in the trace view in the Logs tab, and will only be shown on the Related logs card or in the Log viewer. However, you may use manual log enrichment in order to connect your function logs to your traces. See AWS Lambda logs in context of traces for language-specific examples of manual log enrichment.
- In Dynatrace, logs are marked as error logs only if the log message contains the string
[error]or[ERROR]. - We do not recommend combining the standalone collector layer with a OneAgent layer. If you want tracing and log collection for your Lambda functions, use the combined Traces and Logs layer instead.