Try it free

Google Cloud Dataproc monitoring

  • How-to guide
  • 1-min read
  • Published Jan 17, 2022

Dynatrace Google Cloud integration leverages data collected from the Google Operation API to constantly monitor health and performance of Google Cloud Services. While combining all relevant data into dashboards, it also enables alerting and event tracking.

Prerequisites

Set up integration

Add services and feature sets Optional

After integration, Dynatrace automatically monitors a number of preset Google Cloud services and feature sets (metrics). Besides these, you can add more services or feature sets to monitoring. For details, see Add or remove services.

For a list of feature sets available for this service, see Metric table.

View metrics

After deploying the integration, you can see metrics from monitored services in the Metrics browser, Data Explorer, and your dashboard tiles.

Metric table

The following feature sets are available for Google Cloud Dataproc.

Feature setNameUnitGCP metric identifier

cloud_dataproc_cluster/default_metrics

HDFS DataNodes

Count

dataproc.googleapis.com/cluster/hdfs/datanodes

cloud_dataproc_cluster/default_metrics

HDFS capacity

GibiByte

dataproc.googleapis.com/cluster/hdfs/storage_capacity

cloud_dataproc_cluster/default_metrics

HDFS storage utilization

Count

dataproc.googleapis.com/cluster/hdfs/storage_utilization

cloud_dataproc_cluster/default_metrics

Unhealthy HDFS blocks by status

Count

dataproc.googleapis.com/cluster/hdfs/unhealthy_blocks

cloud_dataproc_cluster/default_metrics

Job duration

Second

dataproc.googleapis.com/cluster/job/completion_time

cloud_dataproc_cluster/default_metrics

Job state duration

Second

dataproc.googleapis.com/cluster/job/duration

cloud_dataproc_cluster/default_metrics

Failed jobs

Count

dataproc.googleapis.com/cluster/job/failed_count

cloud_dataproc_cluster/default_metrics

Running jobs

Count

dataproc.googleapis.com/cluster/job/running_count

cloud_dataproc_cluster/default_metrics

Submitted jobs

Count

dataproc.googleapis.com/cluster/job/submitted_count

cloud_dataproc_cluster/default_metrics

Operation duration

Second

dataproc.googleapis.com/cluster/operation/completion_time

cloud_dataproc_cluster/default_metrics

Operation state duration

Second

dataproc.googleapis.com/cluster/operation/duration

cloud_dataproc_cluster/default_metrics

Failed operations

Count

dataproc.googleapis.com/cluster/operation/failed_count

cloud_dataproc_cluster/default_metrics

Running operations

Count

dataproc.googleapis.com/cluster/operation/running_count

cloud_dataproc_cluster/default_metrics

Submitted operations

Count

dataproc.googleapis.com/cluster/operation/submitted_count

cloud_dataproc_cluster/default_metrics

YARN allocated memory percentage

Count

dataproc.googleapis.com/cluster/yarn/allocated_memory_percentage

cloud_dataproc_cluster/default_metrics

YARN active applications

Count

dataproc.googleapis.com/cluster/yarn/apps

cloud_dataproc_cluster/default_metrics

YARN containers

Count

dataproc.googleapis.com/cluster/yarn/containers

cloud_dataproc_cluster/default_metrics

YARN memory size

GibiByte

dataproc.googleapis.com/cluster/yarn/memory_size

cloud_dataproc_cluster/default_metrics

YARN NodeManagers

Count

dataproc.googleapis.com/cluster/yarn/nodemanagers

cloud_dataproc_cluster/default_metrics

YARN pending memory size

GibiByte

dataproc.googleapis.com/cluster/yarn/pending_memory_size

cloud_dataproc_cluster/default_metrics

YARN virtual cores

Count

dataproc.googleapis.com/cluster/yarn/virtual_cores

Related topics

  • Google Cloud integrations
Related tags
Infrastructure Observability