Google Cloud Dataproc monitoring

Dynatrace Google Cloud integration leverages data collected from the Google Operation API to constantly monitor health and performance of Google Cloud Services. While combining all relevant data into dashboards, it also enables alerting and event tracking.

Prerequisites

Set up integration

Add services and feature sets optional

After integration, Dynatrace automatically monitors a number of preset Google Cloud services and feature sets (metrics). Besides these, you can add more services or feature sets to monitoring. For details, see Add or remove services.

For a list of feature sets available for this service, see Metric table.

View metrics

After deploying the integration, you can see metrics from monitored services in the Metrics browser, Data Explorer, and your dashboard tiles.

Metric table

The following feature sets are available for Google Cloud Dataproc.

Feature set
Name
Unit
GCP metric identifier
cloud_dataproc_cluster/default_metrics
HDFS DataNodes
Count
dataproc.googleapis.com/cluster/hdfs/datanodes
cloud_dataproc_cluster/default_metrics
HDFS capacity
GibiByte
dataproc.googleapis.com/cluster/hdfs/storage_capacity
cloud_dataproc_cluster/default_metrics
HDFS storage utilization
Count
dataproc.googleapis.com/cluster/hdfs/storage_utilization
cloud_dataproc_cluster/default_metrics
Unhealthy HDFS blocks by status
Count
dataproc.googleapis.com/cluster/hdfs/unhealthy_blocks
cloud_dataproc_cluster/default_metrics
Job duration
Second
dataproc.googleapis.com/cluster/job/completion_time
cloud_dataproc_cluster/default_metrics
Job state duration
Second
dataproc.googleapis.com/cluster/job/duration
cloud_dataproc_cluster/default_metrics
Failed jobs
Count
dataproc.googleapis.com/cluster/job/failed_count
cloud_dataproc_cluster/default_metrics
Running jobs
Count
dataproc.googleapis.com/cluster/job/running_count
cloud_dataproc_cluster/default_metrics
Submitted jobs
Count
dataproc.googleapis.com/cluster/job/submitted_count
cloud_dataproc_cluster/default_metrics
Operation duration
Second
dataproc.googleapis.com/cluster/operation/completion_time
cloud_dataproc_cluster/default_metrics
Operation state duration
Second
dataproc.googleapis.com/cluster/operation/duration
cloud_dataproc_cluster/default_metrics
Failed operations
Count
dataproc.googleapis.com/cluster/operation/failed_count
cloud_dataproc_cluster/default_metrics
Running operations
Count
dataproc.googleapis.com/cluster/operation/running_count
cloud_dataproc_cluster/default_metrics
Submitted operations
Count
dataproc.googleapis.com/cluster/operation/submitted_count
cloud_dataproc_cluster/default_metrics
YARN allocated memory percentage
Count
dataproc.googleapis.com/cluster/yarn/allocated_memory_percentage
cloud_dataproc_cluster/default_metrics
YARN active applications
Count
dataproc.googleapis.com/cluster/yarn/apps
cloud_dataproc_cluster/default_metrics
YARN containers
Count
dataproc.googleapis.com/cluster/yarn/containers
cloud_dataproc_cluster/default_metrics
YARN memory size
GibiByte
dataproc.googleapis.com/cluster/yarn/memory_size
cloud_dataproc_cluster/default_metrics
YARN NodeManagers
Count
dataproc.googleapis.com/cluster/yarn/nodemanagers
cloud_dataproc_cluster/default_metrics
YARN pending memory size
GibiByte
dataproc.googleapis.com/cluster/yarn/pending_memory_size
cloud_dataproc_cluster/default_metrics
YARN virtual cores
Count
dataproc.googleapis.com/cluster/yarn/virtual_cores