Google Cloud Dataproc monitoring

  • How-to guide
  • 1-min read
  • Published Jan 17, 2022

Dynatrace Google Cloud integration leverages data collected from the Google Operation API to constantly monitor health and performance of Google Cloud Services. While combining all relevant data into dashboards, it also enables alerting and event tracking.

Prerequisites

Set up integration

Add services and feature sets optional

After integration, Dynatrace automatically monitors a number of preset Google Cloud services and feature sets (metrics). Besides these, you can add more services or feature sets to monitoring. For details, see Add or remove services.

For a list of feature sets available for this service, see Metric table.

View metrics

After deploying the integration, you can see metrics from monitored services in the Metrics browser, Data Explorer, and your dashboard tiles.

Metric table

The following feature sets are available for Google Cloud Dataproc.

Feature setNameUnitGCP metric identifier
cloud_dataproc_cluster/default_metricsHDFS DataNodesCountdataproc.googleapis.com/cluster/hdfs/datanodes
cloud_dataproc_cluster/default_metricsHDFS capacityGibiBytedataproc.googleapis.com/cluster/hdfs/storage_capacity
cloud_dataproc_cluster/default_metricsHDFS storage utilizationCountdataproc.googleapis.com/cluster/hdfs/storage_utilization
cloud_dataproc_cluster/default_metricsUnhealthy HDFS blocks by statusCountdataproc.googleapis.com/cluster/hdfs/unhealthy_blocks
cloud_dataproc_cluster/default_metricsJob durationSeconddataproc.googleapis.com/cluster/job/completion_time
cloud_dataproc_cluster/default_metricsJob state durationSeconddataproc.googleapis.com/cluster/job/duration
cloud_dataproc_cluster/default_metricsFailed jobsCountdataproc.googleapis.com/cluster/job/failed_count
cloud_dataproc_cluster/default_metricsRunning jobsCountdataproc.googleapis.com/cluster/job/running_count
cloud_dataproc_cluster/default_metricsSubmitted jobsCountdataproc.googleapis.com/cluster/job/submitted_count
cloud_dataproc_cluster/default_metricsOperation durationSeconddataproc.googleapis.com/cluster/operation/completion_time
cloud_dataproc_cluster/default_metricsOperation state durationSeconddataproc.googleapis.com/cluster/operation/duration
cloud_dataproc_cluster/default_metricsFailed operationsCountdataproc.googleapis.com/cluster/operation/failed_count
cloud_dataproc_cluster/default_metricsRunning operationsCountdataproc.googleapis.com/cluster/operation/running_count
cloud_dataproc_cluster/default_metricsSubmitted operationsCountdataproc.googleapis.com/cluster/operation/submitted_count
cloud_dataproc_cluster/default_metricsYARN allocated memory percentageCountdataproc.googleapis.com/cluster/yarn/allocated_memory_percentage
cloud_dataproc_cluster/default_metricsYARN active applicationsCountdataproc.googleapis.com/cluster/yarn/apps
cloud_dataproc_cluster/default_metricsYARN containersCountdataproc.googleapis.com/cluster/yarn/containers
cloud_dataproc_cluster/default_metricsYARN memory sizeGibiBytedataproc.googleapis.com/cluster/yarn/memory_size
cloud_dataproc_cluster/default_metricsYARN NodeManagersCountdataproc.googleapis.com/cluster/yarn/nodemanagers
cloud_dataproc_cluster/default_metricsYARN pending memory sizeGibiBytedataproc.googleapis.com/cluster/yarn/pending_memory_size
cloud_dataproc_cluster/default_metricsYARN virtual coresCountdataproc.googleapis.com/cluster/yarn/virtual_cores
Related tags
Infrastructure Observability