Configure monitoring for namespaces and pods

5-min read
Published Jul 28, 2023

cloudNativeFullStack

applicationMonitoring

As part of monitoring your Kubernetes cluster with cloud-native full-stack or application monitoring, you might want to restrict to certain namespaces and pods.

By default, Dynatrace Operator injects OneAgent into all namespaces, except for:

Namespaces prefixed with kube- or openshift-.
The namespace where Dynatrace Operator was installed.

Monitor specific namespaces

To configure the Dynatrace Operator to inject OneAgent into only certain namespaces or apply metadata enrichment, set the namespaceSelector parameter in the DynaKube custom resource.

For more information, see DynaKube parameters for Dynatrace Operator (.spec.metadataEnrichment, .spec.oneAgent.cloudNativeFullStack, and .spec.oneAgent.applicationMonitoring fields).

Label your namespaces.

kubectl label namespace <my_namespace> dynatrace.com/inject=true

Modify your DynaKube by adding the namespaceSelector to specify the label for monitoring.
```
...
namespaceSelector:
  matchLabels:
    dynatrace.com/inject: "true"
```

For more details about configuring labels for selective monitoring, see Labels and selectors.

To add exceptions for specific pods within the selected namespaces, you can annotate the respective pods.

Exclude specific namespaces

To exclude certain namespaces from being monitored, modify the DynaKube custom resource as follows.

key defines the key of the label. Starting with Kubernetes version 1.22, a default label kubernetes.io/metadata.name is added to namespaces.
values define the value of the label.

...
namespaceSelector:
  matchExpressions:
  - key: LabelKey
    operator: NotIn
    values:
    - LabelValue

If you run kubectl describe namespace dynatrace, you'll see:

metadata:
  name: dynatrace
  labels:
    kubernetes.io/metadata.name=dynatrace

A valid selector example to exclude dynatrace would be:

...
namespaceSelector:
  matchExpressions:
  - key: kubernetes.io/metadata.name
    operator: NotIn
    values:
    - dynatrace

The webhook will inject every namespace that matches all namespaceselector.

For more details, see Resources that support set-based requirements.

Exclude specific pods in monitored namespaces

To exclude specific pods within monitored namespaces, annotate the pods accordingly.

...
metadata:
  annotations:
    ...
    oneagent.dynatrace.com/inject: "false"

Annotations available for fine-grained control include.

dynatrace.com/inject: Disables all injection when set to false.
metadata-enrichment.dynatrace.com/inject: Prevents metric enrichment file addition when false.
oneagent.dynatrace.com/inject: Disables OneAgent modifications when set to false.

Exclude specific containers in monitored pods

Dynatrace Operator version 1.0.0+

To exclude specific container images within monitored namespaces, annotate the pods or DynaKube accordingly (this can be useful to, for example, exclude side car containers).

...
metadata:
  annotations:
    ...
    container.inject.dynatrace.com/<container-name>: "false"

This annotation can be applied at the DynaKube level (affecting all pods) or at the individual pod level (affecting only the specified pod).

Monitor only specific pods

Dynatrace Operator version 0.8.0+

Dynatrace Operator can be set to monitor namespaces without injecting into any pods, so you can choose which pods to monitor.

Disable the automatic injection feature for the DynaKube deployment to your cluster.

apiVersion: dynatrace.com/v1beta2
kind: DynaKube
metadata:
  name: dynakube
  namespace: dynatrace
  annotations:
    feature.dynatrace.com/automatic-injection: "false"
spec:
  oneAgent:
    cloudNativeFullStack:
      namespaceSelector:
        matchLabels:
          dynatrace.com/inject: "true"
...

Use label selectors or manual annotations on the namespaces you want to monitor selectively.
```
kubectl label namespace <my_namespace> dynatrace.com/inject=true
```

Annotate the pods you intend to monitor.

...
metadata:
  annotations:
    ...
    oneagent.dynatrace.com/inject: "true"

Fine-tuning of injection for `applicationMonitoring` without CSI driver

This section has been deprecated with Dynatrace Operator version 1.5.0 and superseded by the new node image pull feature.

oneagent.dynatrace.com/flavor: Set to default or musl to specify the binary compatibility. This indicates whether glibc or musl binaries should be downloaded, with glibc as the default setting. For containers based on musl (for example, Alpine), specify this annotation to ensure proper monitoring.
oneagent.dynatrace.com/technologies: A comma-separated list of technologies. This filters the code modules to be downloaded, defaulting to all. Use this to tailor the OneAgent to monitor specific technologies within your application.
oneagent.dynatrace.com/install-path: Specifies the path where the OneAgent directory will be mounted. By default, it is set to /opt/dynatrace/oneagent-paas. Adjust this path based on your environment or requirements.
oneagent.dynatrace.com/installer-url: Indicates the URL from which the OneAgent app-only package will be downloaded. It automatically defaults to the Dynatrace environment API configured on the API URL of DynaKube.

Below is an example showcasing how to apply these annotations within your deployment.

...
metadata:
  annotations:
    oneagent.dynatrace.com/technologies: "java,nginx"
    oneagent.dynatrace.com/flavor: "musl"
    oneagent.dynatrace.com/install-path: "/dynatrace"
    oneagent.dynatrace.com/installer-url: "https://my-custom-url/route/file.zip"