Log Monitoring Classic
Dynatrace integration with Amazon Data Firehose provides a simple and safe way to ingest AWS logs. To enable AWS log forwarding, you need to create Amazon Data Firehose instance and configure it with your Dynatrace environment as a destination. Then you can connect your CloudWatch log groups by creating a subscription filter or send logs directly to Data Firehose from services that support it (e.g. Amazon Managed Streaming for Apache Kafka). Data Firehose and other created cloud resources incur AWS costs according to standard AWS billing policy. See the Cloud log forwarding page to learn about all the options for AWS log ingestion.
Create an API token in your Dynatrace environment and enable the Ingest logs permission.
Determine the API URL for your environment:
For Dynatrace SaaS recommended
https://<your_environment_ID>.live.dynatrace.com
For ActiveGate (additional setup required)
https://<your_active_gate_IP_or_hostname>/e/<your_environment_ID>
To determine <your_environment_ID>
, see What is a monitoring environment?.
You can set up an Amazon Data Firehose delivery stream with a CloudFormation template or in the AWS console. Check the instructions below.
If you choose another deployment method (for example, Terraform or custom script), use the full URL: https://<your_environment_ID>.live.dynatrace.com/api/v2/logs/ingest/aws_firehose
in the Firehose HTTP endpoint destination configuration.
CloudFormation allows you to deploy an Amazon Data Firehose delivery stream using a single deployment command to create a stack that groups multiple AWS resources. This approach is faster and makes AWS resource management easier.
To fetch the CloudFormation template and deploy it to your AWS account, run the command below.
Make sure to replace <your_API_URL>
and <your_API_token>
with your values.
Consult the parameters table that follows for more details.
STACK_NAME
dynatrace-log-delivery-stream
If you have AWS CLI configured, you can use a Bash-compliant shell. Otherwise, you can use CloudShell available in the AWS console.
DYNATRACE_API_URL=<your_API_URL>DYNATRACE_API_KEY=<your_API_token>STACK_NAME=dynatrace-log-delivery-streamwget -O dynatrace-firehose-log-stream.yaml https://assets.cloud.dynatrace.com/awslogstreaming/dynatrace-firehose-log-stream.yaml && \aws cloudformation deploy --capabilities CAPABILITY_NAMED_IAM --template-file ./dynatrace-firehose-log-stream.yaml --stack-name $STACK_NAME --parameter-overrides DtApiUrl=$DYNATRACE_API_URL DtApiToken=$DYNATRACE_API_KEY
To ensure that the Amazon Data Firehose delivery stream was deployed correctly, follow the steps below:
CloudWatchSubscriptionFilterRoleArn
- ARN of the IAM role to use when creating CloudWatch subscription filter;FirehoseArn
- ARN of the newly created Firehose delivery stream.After creating a Firehose delivery stream and IAM role, you need to subscribe to the CloudWatch log groups whose logs you want to forward to Dynatrace. You can subscribe to log groups using shell script or in the AWS console. See the instructions below.
To fetch the shell script, run the command below in a bash shell.
wget -O dynatrace-firehose-logs.sh https://assets.cloud.dynatrace.com/awslogstreaming/dynatrace-firehose-logs.sh && chmod +x dynatrace-firehose-logs.sh
If you have AWS CLI configured, you can use a Bash-compliant shell. Otherwise, you can use CloudShell, which is available in the AWS console.
Usage recommendation: Use this option if the number of log groups you'd like to subscribe to is small.
To subscribe: Run the command below, making sure to replace <your_log_group_list>
with a space-separated list of the log group names you want to subscribe to.
Example list: /aws/lambda/my-lambda /aws/apigateway/my-api
./dynatrace-firehose-logs.sh subscribe --log-groups <your_log_group_list>
No additional parameters are needed if the Firehose delivery stream was created with a CloudFormation template using the default stack name.
Add [--stack-name <your_stack_name>]
parameter if you used a different stack name.
./dynatrace-firehose-logs.sh subscribe --log-groups <your_log_group_list> --stack-name <your_stack_name>
If the Firehose delivery stream was created in a different way (AWS console or other tools), add the following parameters:
[--firehose-arn <firehose_arn>]
[--role-arn <role_arn>]
./dynatrace-firehose-logs.sh subscribe --log-groups <your_log_group_list> --firehose-arn <firehose_arn> --role-arn <role_arn>
Usage recommendation: Use this option if the number of log groups you'd like to subscribe to is large.
Create a file and enter each log group name on a separate line.
Save the file.
Run the command below, making sure to replace <your_file_name>
with the actual file name.
./dynatrace-firehose-logs.sh subscribe --log-groups-from-file <your_file_name>
To simplify file creation, you can use the auto-discovery command below to list the names of all log groups in your account. You can adjust the list manually before subscribing.
Make sure to replace <your_log_groups_file>
with the name of the file to which you want to redirect the output.
./dynatrace-firehose-logs.sh discover-log-groups > <your_log_groups_file>
Usage recommendation: By default, you subscribe to all the logs in the log group. Use this option if you want to restrict the logs you subscribe to. See Filter and Pattern Syntax for details on the pattern syntax.
Limitation: You can use only two subscription filters per log group, so the possibility of creating multiple filters with different patterns is limited. If you create a subscription filter that exceeds the limit, an AWS LimitExceededException
occurs.
To subscribe: Run the command below, making sure to replace <your_log_group_list>
and <your_filter_pattern>
with your values.
./dynatrace-firehose-logs.sh subscribe --log-groups <your_log_group_list> --filter-pattern <your_filter_pattern>
For additional subscription options, see the commands below.
Consult the Subscription parameters table for the commands below when replacing placeholders (<...>
) with your values.
dynatrace-firehose-logs.sh subscribe {--log-groups <your_log_group_list> | --log-groups-from-file <your_file_name>}[--stack-name <your_stack_name>] [--filter-pattern <your_filter_pattern>] [--role-arn <role_arn>] [--firehose-arn <firehose_arn>]
--log-groups
LOG_GROUPS_LIST
/aws/lambda/my-lambda /aws/apigateway/my-api
.--log-groups-from-file
LOG_GROUPS_FILE
--filter-pattern
FILTER_PATTERN
--stack-name
STACK_NAME
dynatrace-aws-logs
--firehose-arn
FIREHOSE_ARN
$DEFAULT_STACK_NAME
default value or the one specified with the --stack-name <your_stack_name>
option.--role-arn
ROLE_ARN
$DEFAULT_STACK_NAME
default value or the one specified with the --stack-name <your_stack_name>
option.If you don't want to forward logs to Dynatrace anymore, use one of the two options below to unsubscribe from log groups.
Run the command below, making sure to replace <your_log_group_list>
with a space-separated list of the log group names you want to unsubscribe from.
./dynatrace-firehose-logs.sh unsubscribe --log-groups <your_log_group_list>
Run the command below, making sure to replace <your_file_name>
with the file name you created to subscribe by reading log groups from file.
./dynatrace-firehose-logs.sh unsubscribe --log-groups-from-file <your_file_name>
For additional unsubscribe options, see the commands below.
Consult the Unsubscribe parameters table for the commands below when replacing the placeholders (<...>
) with your values.
dynatrace-firehose-logs.sh unsubscribe {--log-groups <your_log_group_list> | --log-groups-from-file <your_file_name>} [--stack-name <your_stack_name>]
--log-groups
LOG_GROUPS_LIST
/aws/lambda/my-lambda /aws/apigateway/my-api
.--log-groups-from-file
LOG_GROUPS_FILE
--stack-name
STACK_NAME
dynatrace-aws-logs
To configure logs not stored in CloudWatch for services that send them directly to Firehose, refer to specific service documentation, for example:
For logs from AWS services that are sent to S3—not Firehose or CloudWatch—see GitHub documentation.
After configuring Data Firehose streaming, you can view and analyze AWS logs in Dynatrace: Go to Logs & Events or Notebooks, and filter for AWS logs. Logs ingested via Amazon Data Firehose will have aws.data_firehose.arn
attribute set to ARN of Firehose that streamed the data into Dynatrace. Logs from AWS services with entity linking support will automatically be displayed in the Cloud application for in context analysis.
If you see logs coming in, you managed to configure AWS logs streaming successfully.
If there are no logs within 10 minutes, check out the Troubleshooting guide section of the page.
Amazon Data Firehose includes optional parameters (key-value pairs) in each HTTP call. These instance parameters can help you identify and manage your destinations since they're processed and added automatically to ingested log records as attributes.
Supported services
You can modify the AWS Lambda log group name. For log enrichment, use the default log group name /aws/lambda/<function name>
.
You can modify the AWS CloudTrail log group name. For log enrichment, start the log group name with aws-cloudtrail-logs
.
ActiveGate version 1.287+
By default, Environment ActiveGate listens for API requests on port 9999. However, currently, only port 443 is supported for HTTP endpoint data delivery for Amazon Data Firehose.
Your ActiveGate needs to be configured with a valid CA-signed SSL certificate to be able to receive logs from AWS Data Firehose.
To successfully deliver data from Amazon Data Firehose to the Environment ActiveGate API endpoint, we recommend setting up port forwarding from port 443 to 9999 on ActiveGate host.
Below we have included a few examples of such configurations. Consult the documentation specific to your operating system and networking solutions for details.
firewalld
provides a dynamically managed firewall. See the documentation for details.
To add port forwarding with firewalld
(note: this actions need to be done using the root account):
firewall-cmd --zone=public --add-forward-port=port=443:proto=tcp:toport=9999 --permanentfirewall-cmd --zone=public --add-port=9999/tcp --permanent
The Uncomplicated Firewall (ufw
) is a frontend for iptables. See the documentation for details.
To add port forwarding with ufw
(note: this actions need to be done using the root account):
/etc/ufw/before.rules
file, let’s add a NAT table after the filter table (the table that starts with *filter
and ends with COMMIT
):*nat:PREROUTING ACCEPT [0:0]-A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 9999COMMIT
ufw allow 443/tcpufw allow 9999/tcp
ufw
.Network shell (netsh) is a command-line utility that allows you to configure and display the status of various network communications server roles and components. See the documentation for details.
To add port forwarding with netsh interface portproxy
:
netsh interface portproxy add v4tov4 listenport=443 connectport=9999 connectaddress=<the current IP address of your computer>
Using the netsh interface portproxy add
v4tov6
/v6tov4
/v6tov6
options, you can create port forwarding rules between IPv4 and IPv6 addresses.
In case the logs forwarded from Data Firehose are not available in your environment, follow the steps below:
dsfm:active_gate.rest.request_count:filter(and(or(eq(operation,"POST /logs/ingest/aws_firehose")))):splitBy(response_code):sort(value(auto,descending)):limit(20)
There should be metric data, and the response\_code
should only have the value 200
.
The ingest throughput is limited by Amazon Data Firehose. For more details, see Amazon Data Firehose Quota. Amazon can increase firehose limits on request.
AWS Firehose does not support connections through VPC for HTTP endpoints.