Kubernetes log monitoring

5-min read

Dynatrace provides integrated Log management and analytics for your Kubernetes environments. We recommend collecting logs in Kubernetes using our fully managed Dynatrace Log module, either integrated in the OneAgent deployed on the node (OneAgent Log module) or without OneAgent as a standalone deployment (Kubernetes Log module). Dynatrace Operator configures and manages the Dynatrace Log module for both approaches. Alternatively, you can stream logs to Dynatrace using log collectors such as Fluent Bit, Dynatrace OpenTelemetry Collector, Logstash, or Fluentd.

Use cases

Deployment options for Kubernetes log monitoring using the Dynatrace Log module

Dynatrace provides a flexible approach to Kubernetes observability where you can pick and choose the level of observability you need for your Kubernetes clusters. The Dynatrace Operator manages all the components needed to get the data into Dynatrace for you. This also applies to collecting logs from Kubernetes containers. Depending on the selected observability option, the Dynatrace Operator configures and manages the Dynatrace Log module to work in conjunction with or without a OneAgent on the node. The Kubernetes Log module is used in combination with Kubernetes platform monitoring or Application observability, whereas the OneAgent Log module is used as part of Full-Stack observability.

Log monitoring value

Kubernetes Log module

OneAgent Log module

Auto discovery of container logs

Control ingest via annotations and labels

Log enrichment with Kubernetes metadata

Logs in context of traces

Log enrichment with process context

Report logs to different Dynatrace environments

Dynatrace Operator for managing the rollout and lifecycle

Deployment

OneAgent Log module integrates with OneAgent on node

Kubernetes Log module deployed as DaemonSet

Supported Observability Options

Kubernetes platform monitoring + Full-Stack observability

Kubernetes platform monitoring and Kubernetes platform monitoring + Application observability

Managed by Dynatrace Operator

For pods with Application observability enabled

Automated updates of the Kubernetes Log module are planned for future releases

Supported versions

The Dynatrace Log module reads logs from containerd and cri-o containers. Other container runtimes aren't supported. It only captures logs that are written to the container's stdout/stderr streams.

See supported Kubernetes/OpenShift platform versions and distributions to learn more.

Dynatrace version 1.310+
Dynatrace Operator version 1.4.2+
Dynatrace OneAgent version 1.309+

Before you begin

Before installing Dynatrace on your Kubernetes cluster, ensure that you meet the following requirements:

Your kubectl CLI is connected to the Kubernetes cluster that you want to monitor.
You have sufficient privileges on the monitored cluster to run kubectl or oc commands.

Cluster setup and configuration

You must allow egress for Dynatrace pods (default: Dynatrace namespace) to your Dynatrace environment URL
For OpenShift Dedicated, you need the cluster-admin role
Helm installation Use Helm version 3

Installation options

Stream logs using the OneAgent Log module (Full-Stack observability)

Installing Dynatrace with Full-Stack observability automatically deploys the OneAgent on each node in your Kubernetes cluster. The OneAgent running on the node already includes the OneAgent Log module. To enable log monitoring, you only need to add the spec.logMonitoring: {} section to your DynaKube custom resource. Below is an example configuration:

apiVersion: dynatrace.com/v1beta5
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
  #  annotations:
#    feature.dynatrace.com/oneagent-privileged: "true" # Required on Openshift
# Link to api reference for further information: https://docs.dynatrace.com/docs/ingest-from/setup-on-k8s/reference/dynakube-parameters
spec:

  apiUrl: https://ENVIRONMENTID.live.dynatrace.com/api

  metadataEnrichment:
    enabled: true

  oneAgent:
    cloudNativeFullStack:
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
          operator: Exists
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
          operator: Exists

  activeGate:
    capabilities:
      - routing
      - kubernetes-monitoring
      - debugging
    resources:
      requests:
        cpu: 500m
        memory: 1.5Gi
      limits:
        cpu: 1000m
        memory: 1.5Gi

  logMonitoring: {}

We recommend to review the Collect all containers logs feature flag within your settings to ensure best coverage of your logs within Kubernetes. For advanced configuration options, see Stream Kubernetes logs with OneAgent Log Module.

Stream logs using the Kubernetes Log module (Kubernetes platform monitoring or Kubernetes platform monitoring + Application observability)

The following guides assumes that you've already succesfully installed the Dynatrace Operator on your Kubernetes cluster. If you haven't done so, please follow the instructions in Install Dynatrace on Kubernetes.

To add the Kubernetes Log module to your existing Dynatrace installation, follow these steps:

Edit your existing DynaKube custom resource YAML file.

You can review the available parameters or how-to guides, and adapt the DynaKube custom resource according to your requirements.

In order to enable the Kubernetes log module, you need to add two sections to your DynaKube custom resource. The first section enables the Log module (spec.logMonitoring), and the second section configures the Log module (spec.templates.logMonitoring). Below is an example configuration:

apiVersion: dynatrace.com/v1beta5
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
#  annotations:
#    feature.dynatrace.com/oneagent-privileged: "true" # Required on Openshift
spec:
  # Link to api reference for further information: https://docs.dynatrace.com/docs/ingest-from/setup-on-k8s/reference/dynakube-parameters

  apiUrl: https://ENVIRONMENTID.live.dynatrace.com/api

  metadataEnrichment:
    enabled: true

  logMonitoring: {}

  activeGate:
    capabilities:
      - kubernetes-monitoring
    resources:
      requests:
        cpu: 500m
        memory: 512Mi
      limits:
        cpu: 1000m
        memory: 1.5Gi
        
  templates:
    logMonitoring:
      imageRef:
        repository: public.ecr.aws/dynatrace/dynatrace-logmodule
        tag: <tag>
      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
          operator: Exists
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
          operator: Exists
      # Optionally set resource requests/limits for the
      # Kubernetes log module. (applies to init and main container)
      # resources:
      #   requests:
      #     cpu: 
      #     memory: 
      #   limits:
      #     cpu: 
      #     memory:

To retrieve the <tag> version for the logMonitoring template:

Go to the ECR Public Gallery.
Select Image tabs, and use one of the versions. For example, 1.311.70.20250416-094918.

To update the Kubernetes Log module, you'll need to manually update the tag in the logMonitoring template and apply the changes to your DynaKube custom resource. Automatic updates of the Kubernetes Log module are planned for future releases.

Re-Apply the DynaKube custom resource

Run the command below to apply the DynaKube custom resource, making sure to replace <your-DynaKube-CR> with your actual DynaKube custom resource file name. A validation webhook will provide helpful error messages if there's a problem.
```
kubectl apply -f <your-DynaKube-CR>.yaml
```

optional Verify deployment

Verify that your DynaKube is running and all pods in your Dynatrace namespace are running and ready.

> kubectl get dynakube -n dynatrace
NAME         APIURL                                          STATUS     AGE
dynakube     https://<ENVIRONMENTID>.live.dynatrace.com/api  Running    45s

In a default DynaKube configuration, you should see the following pods:

> kubectl get pods -n dynatrace
NAME                                  READY   STATUS    RESTARTS   AGE
dynakube-activegate-0                 1/1     Running   0          55s
dynakube-logmonitoring-grrnd          1/1     Running   0          55s
dynakube-logmonitoring-ptjgk          1/1     Running   0          55s
dynakube-logmonitoring-rtc2p          1/1     Running   0          55s
dynatrace-oneagent-csi-driver-2twgv   4/4     Running   0          5m
dynatrace-oneagent-csi-driver-jbwdv   4/4     Running   0          5m
dynatrace-oneagent-csi-driver-t68tt   4/4     Running   0          5m
dynatrace-operator-74dbb44b57-g58mn   1/1     Running   0          5m
dynatrace-webhook-59b69958d6-82wlr    1/1     Running   0          5m
dynatrace-webhook-59b69958d6-d9vqd    1/1     Running   0          5m

As the OneAgent Log module is deployed as DaemonSet you should have a Log monitoring pod on each node.

Licensing

Kubernetes Log monitoring requires Dynatrace Platform Subscription (DPS) and is licensed by the ingested gibibyte (GiB) volume.

Control log ingest with Kubernetes metadata

You can configure log ingestion rules in Dynatrace to control which logs should be collected from your Kubernetes environment. The rules leverage Kubernetes metadata and other common log entry attributes, such as the Kubernetes namespace name, to determine which logs are to be ingested. The standard log processing features from OneAgent, including sensitive data masking, timestamp configuration, log boundary definition, and automatic enrichment of log records, are also available for Kubernetes logs.

See Stream Kubernetes logs with Dynatrace Log Module for a detailed description, use cases, and REST API examples.