Monitor Google Cloud projects using auto-discovery

  • How-to guide
  • 6-min read
  • Published Sep 25, 2023

Early Adopter

Starting from the Google Cloud Monitor integration version 1.3.0+, you can extend your metric coverage beyond the predefined set of services supported by extensions.

  • For a complete list of supported services, see Google Cloud supported service metrics.
  • To monitor any metric from any Google resource not covered by those services, follow the instructions on this page to activate auto-discovery.

Possible auto-discovery scenarios depend on the services from which you want to monitor metrics:

  • Expand metrics for already supported services
  • Integrate any service based on the resource name

Before you begin

This guide is a component of the main Dynatrace Google Cloud Metric and Log Integration setup, and focuses solely on auto-discovery configuration.

Minimum version

If Google Cloud integration has been previously set up using a version earlier than 1.3.0, you need to run a fresh deployment of the complete solution using version 1.3.0+.

Set up auto-discovery monitoring

Ensure permissions

If you are managing permissions manually, ensure the following permissions are added to the Dynatrace Google Cloud Metrics Monitor role.

  • monitoring.metricDescriptors.list
  • monitoring.monitoredResourceDescriptors.get
  • monitoring.monitoredResourceDescriptors.list

Enable auto-discovery metric enrichment

To enable the auto-discovery metric enrichment, edit the following parameter in values.yaml:

metricAutodiscovery: "true"

Expand metrics for already supported services

Within the scope of standard supported services, a predetermined set of metrics is routinely sent to Dynatrace. For details on how to enable those services, see Choose services for metrics monitoring.

Auto-discovery mode, however, brings more flexibility by allowing the collection of all available metrics for a particular service.

Enable auto-discovery

To enable it, you need to modify the values.yaml file by searching for the specific service as follows:

- service: cloud_service_name
allowAutodiscovery: True

Supported services

Expand the following table for a list of resources currently monitored by supported services that can be extended with auto-discovery.

Monitored resourceService name
alloydb.googleapis.com/DatabaseGoogle AlloyDB
alloydb.googleapis.com/InstanceGoogle AlloyDB
apiGoogle Cloud APIs
apigee.googleapis.com/EnvironmentGoogle Apigee
apigee.googleapis.com/ProxyGoogle Apigee
apigee.googleapis.com/ProxyV2Google Apigee
apigee.googleapis.com/TargetV2Google Apigee
assistant_action_projectGoogle Assistant Smart Home
autoscalerGoogle Compute Engine
aws_ec2_instanceGoogle Compute Engine
baremetalsolution.googleapis.com/InstanceGoogle Compute Engine
bigquery_biengine_modelGoogle BigQuery
bigquery_projectGoogle BigQuery
bigtable_clusterGoogle Cloud Bigtable
bigtable_tableGoogle Cloud Bigtable
cloud_composer_environmentGoogle Cloud Composer
cloud_dataproc_clusterGoogle Dataproc
cloud_dlp_projectGoogle Cloud Data Loss Prevention
cloud_functionGoogle Cloud Functions
cloud_run_jobGoogle Cloud Run
cloud_run_revisionGoogle Cloud Run
cloud_tasks_queueGoogle Cloud Tasks
cloudiot_device_registryGoogle Cloud IoT Core
cloudml_jobGoogle AI Platform
cloudml_model_versionGoogle AI Platform
cloudsql_databaseGoogle Cloud SQL
cloudsql_instance_databaseGoogle Cloud SQL
cloudvolumesgcp-api.netapp.com/CloudVolumeNetApp on Google Cloud
consumed_apiGoogle Cloud APIs
consumer_quotaGoogle Cloud APIs
dataflow_jobGoogle Dataflow
datastore_requestGoogle Firestore in Datastore mode
dns_queryGoogle Cloud DNS
filestore_instanceGoogle Cloud Filestore
firebase_domainGoogle Firebase
firebase_namespaceGoogle Firebase
firestore.googleapis.com/DatabaseGoogle Cloud Filestore
firestore_instanceGoogle Cloud Filestore
gae_appGoogle App Engine
gae_instanceGoogle App Engine
gce_routerGoogle Hybrid Connectivity
gce_zone_network_healthGoogle Network Topology
gcs_bucketGoogle Cloud Storage
https_lb_ruleGoogle Cloud Load Balancing
instance_groupGoogle Compute Engine
interconnectGoogle Hybrid Connectivity
interconnect_attachmentGoogle Hybrid Connectivity
internal_http_lb_ruleGoogle Cloud Load Balancing
internal_tcp_lb_ruleGoogle Cloud Load Balancing
internal_udp_lb_ruleGoogle Cloud Load Balancing
istio_control_planeGoogle Kubernetes Engine
k8s_containerGoogle Kubernetes Engine
k8s_nodeGoogle Kubernetes Engine
k8s_podGoogle Kubernetes Engine
loadbalancing.googleapis.com/ExternalNetworkLoadBalancerRuleGoogle Cloud Load Balancing
microsoft_ad_domainGoogle Managed Microsoft AD
nat_gatewayGoogle Cloud Router
netapp_cloud_volumeNetApp on Google Cloud
network_security_policyGoogle Network Security
produced_apiGoogle Cloud APIs
producer_quotaGoogle Cloud APIs
pubsub_snapshotGoogle Pub/Sub
pubsub_subscriptionGoogle Pub/Sub
pubsub_topicGoogle Pub/Sub
pubsublite_subscription_partitionGoogle Pub/Sub Lite
pubsublite_topic_partitionGoogle Pub/Sub Lite
recaptchaenterprise.googleapis.com/KeyGoogle reCAPTCHA Enterprise
redis_instanceGoogle Memorystore
spanner_instanceGoogle Cloud Spanner
tcp_lb_ruleGoogle Cloud Load Balancing
tcp_ssl_proxy_ruleGoogle Cloud Load Balancing
tpu_workerGoogle Compute Engine
transfer_service_agentGoogle Cloud Storage Transfer Service
udp_lb_ruleGoogle Cloud Load Balancing
vpc_access_connectorGoogle Virtual Private Cloud
vpn_gatewayGoogle Hybrid Connectivity

Integrate any service based on resource name

For situations where the extended monitoring capabilities of supported services are insufficient, Google Cloud auto-discovery provides a solution to monitor any service offered by Google.

Limitations

  • Auto-discovery mode is designed to detect only metrics linked to a single resource.
  • These metrics are not attached to any supported service, so there will be no entity or any enrichment extracted from them.

Configure parameter values

The helm deployment package contains the autodiscovery-values.yaml file with the necessary configuration for this scenario. Edit this file to set the required and optional parameter values as follows.

Parameter nameDescriptionDefault value
autodiscoveryQueryIntervalrequired Auto-discovery polling interval in minutes. Minimum value: 60.60
autodiscoveryIncludeAlphaMetricsoptional When set to true, it enables the automatic discovery of experimental Google Cloud metrics withthe 'Alpha' label.true
autodiscoveryAddLabelsoptional When set to true, it appends the label [Autodiscovered] to metric names during automatic discovery.true
autodiscoveryResourcesYamloptional Configuration file for auto-discovery resources.
autodiscoveryBlockListYamloptional Configuration file for auto-discovery metrics name prefixes that shoud not be included.

Select resources

Specify the metrics to be sent to Dynatrace by listing the resource types of the monitored resources in the autodiscoveryResourcesYaml section. To identify the resources for monitoring within your project, see Google Cloud metrics.

Example:

autodiscoveryResourcesYaml: |
autodicovery_config:
searched_resources:
- memcache_node

Disable selected metrics

To exclude specific metrics, list their prefixes in the autodiscoveryBlockListYaml section. Any matching metric will be excluded.

Example:

autodiscoveryBlockListYaml: |
block_list:
- memcache.googleapis.com/node/
Related tags
Infrastructure Observability