Google Dataproc monitoring
Dynatrace GCP integration leverages data collected from the Google Operation API to constantly monitor health and performance of Google Cloud Platform Services. While combining all relevant data into dashboards, it also enables alerting and event tracking.
Prerequisites
Add services and feature sets optional
After integration, Dynatrace automatically monitors a number of preset GCP services and feature sets (metrics). Besides these, you can add more services or feature sets to monitoring. For details, see Add or remove services.
For a list of feature sets available for this service, see Metric table.
View metrics
After deploying the integration, you can see metrics from monitored services in the Metrics browser, the Data explorer, and your dashboard tiles.
Metric table
The following feature sets are available for Google Dataproc.
| Feature set | Name | Unit | GCP metric identifier |
|---|---|---|---|
| cloud_dataproc_cluster/default_metrics | HDFS DataNodes | Count | dataproc.googleapis.com/cluster/hdfs/datanodes |
| cloud_dataproc_cluster/default_metrics | HDFS capacity | GibiByte | dataproc.googleapis.com/cluster/hdfs/storage_capacity |
| cloud_dataproc_cluster/default_metrics | HDFS storage utilization | Count | dataproc.googleapis.com/cluster/hdfs/storage_utilization |
| cloud_dataproc_cluster/default_metrics | Unhealthy HDFS blocks by status | Count | dataproc.googleapis.com/cluster/hdfs/unhealthy_blocks |
| cloud_dataproc_cluster/default_metrics | Job duration | Second | dataproc.googleapis.com/cluster/job/completion_time |
| cloud_dataproc_cluster/default_metrics | Job state duration | Second | dataproc.googleapis.com/cluster/job/duration |
| cloud_dataproc_cluster/default_metrics | Failed jobs | Count | dataproc.googleapis.com/cluster/job/failed_count |
| cloud_dataproc_cluster/default_metrics | Running jobs | Count | dataproc.googleapis.com/cluster/job/running_count |
| cloud_dataproc_cluster/default_metrics | Submitted jobs | Count | dataproc.googleapis.com/cluster/job/submitted_count |
| cloud_dataproc_cluster/default_metrics | Operation duration | Second | dataproc.googleapis.com/cluster/operation/completion_time |
| cloud_dataproc_cluster/default_metrics | Operation state duration | Second | dataproc.googleapis.com/cluster/operation/duration |
| cloud_dataproc_cluster/default_metrics | Failed operations | Count | dataproc.googleapis.com/cluster/operation/failed_count |
| cloud_dataproc_cluster/default_metrics | Running operations | Count | dataproc.googleapis.com/cluster/operation/running_count |
| cloud_dataproc_cluster/default_metrics | Submitted operations | Count | dataproc.googleapis.com/cluster/operation/submitted_count |
| cloud_dataproc_cluster/default_metrics | YARN allocated memory percentage | Count | dataproc.googleapis.com/cluster/yarn/allocated_memory_percentage |
| cloud_dataproc_cluster/default_metrics | YARN active applications | Count | dataproc.googleapis.com/cluster/yarn/apps |
| cloud_dataproc_cluster/default_metrics | YARN containers | Count | dataproc.googleapis.com/cluster/yarn/containers |
| cloud_dataproc_cluster/default_metrics | YARN memory size | GibiByte | dataproc.googleapis.com/cluster/yarn/memory_size |
| cloud_dataproc_cluster/default_metrics | YARN NodeManagers | Count | dataproc.googleapis.com/cluster/yarn/nodemanagers |
| cloud_dataproc_cluster/default_metrics | YARN pending memory size | GibiByte | dataproc.googleapis.com/cluster/yarn/pending_memory_size |
| cloud_dataproc_cluster/default_metrics | YARN virtual cores | Count | dataproc.googleapis.com/cluster/yarn/virtual_cores |