Amazon EMR (Elastic MapReduce) monitoring

How-to guide
12-min read
Published Oct 15, 2020

Dynatrace ingests metrics for multiple preselected namespaces, including Amazon EMR. You can view metrics for each service instance, split metrics into multiple dimensions, and create custom charts that you can pin to your dashboards.

Prerequisites

To enable monitoring for this service, you need

ActiveGate version 1.181+, as follows:
- For Dynatrace SaaS deployments, you need an Environment ActiveGate or a Multi-environment ActiveGate.
- For Dynatrace Managed deployments, you can use any kind of ActiveGate.
  
  For role-based access (whether in a SaaS or Managed deployment), you need an Environment ActiveGate installed on an Amazon EC2 host.
Dynatrace version 1.182+
An updated AWS monitoring policy to include the additional AWS services.
To update the AWS IAM policy, use the JSON below, containing the monitoring policy (permissions) for all supporting services.

JSON predefined policy for all supporting services

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "acm-pca:ListCertificateAuthorities",
        "apigateway:GET",
        "apprunner:ListServices",
        "appstream:DescribeFleets",
        "appsync:ListGraphqlApis",
        "athena:ListWorkGroups",
        "autoscaling:DescribeAutoScalingGroups",
        "cloudformation:ListStackResources",
        "cloudfront:ListDistributions",
        "cloudhsm:DescribeClusters",
        "cloudsearch:DescribeDomains",
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics",
        "codebuild:ListProjects",
        "datasync:ListTasks",
        "dax:DescribeClusters",
        "directconnect:DescribeConnections",
        "dms:DescribeReplicationInstances",
        "dynamodb:ListTables",
        "dynamodb:ListTagsOfResource",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeInstances",
        "ec2:DescribeNatGateways",
        "ec2:DescribeSpotFleetRequests",
        "ec2:DescribeTransitGateways",
        "ec2:DescribeVolumes",
        "ec2:DescribeVpnConnections",
        "ecs:ListClusters",
        "eks:ListClusters",
        "elasticache:DescribeCacheClusters",
        "elasticbeanstalk:DescribeEnvironmentResources",
        "elasticbeanstalk:DescribeEnvironments",
        "elasticfilesystem:DescribeFileSystems",
        "elasticloadbalancing:DescribeInstanceHealth",
        "elasticloadbalancing:DescribeListeners",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeRules",
        "elasticloadbalancing:DescribeTags",
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticmapreduce:ListClusters",
        "elastictranscoder:ListPipelines",
        "es:ListDomainNames",
        "events:ListEventBuses",
        "firehose:ListDeliveryStreams",
        "fsx:DescribeFileSystems",
        "gamelift:ListFleets",
        "glue:GetJobs",
        "inspector:ListAssessmentTemplates",
        "kafka:ListClusters",
        "kinesis:ListStreams",
        "kinesisanalytics:ListApplications",
        "kinesisvideo:ListStreams",
        "lambda:ListFunctions",
        "lambda:ListTags",
        "lex:GetBots",
        "logs:DescribeLogGroups",
        "mediaconnect:ListFlows",
        "mediaconvert:DescribeEndpoints",
        "mediapackage-vod:ListPackagingConfigurations",
        "mediapackage:ListChannels",
        "mediatailor:ListPlaybackConfigurations",
        "opsworks:DescribeStacks",
        "qldb:ListLedgers",
        "rds:DescribeDBClusters",
        "rds:DescribeDBInstances",
        "rds:DescribeEvents",
        "rds:ListTagsForResource",
        "redshift:DescribeClusters",
        "robomaker:ListSimulationJobs",
        "route53:ListHostedZones",
        "route53resolver:ListResolverEndpoints",
        "s3:ListAllMyBuckets",
        "sagemaker:ListEndpoints",
        "sns:ListTopics",
        "sqs:ListQueues",
        "storagegateway:ListGateways",
        "sts:GetCallerIdentity",
        "swf:ListDomains",
        "tag:GetResources",
        "tag:GetTagKeys",
        "transfer:ListServers",
        "workmail:ListOrganizations",
        "workspaces:DescribeWorkspaces"
      ],
      "Resource": "*"
    }
  ]
}

If you don't want to add permissions to all services, and just select permissions for certain services, consult the table below. The table contains a set of permissions that are required for All AWS cloud services and, for each supporting service, a list of optional permissions specific to that service.

Permissions required for AWS monitoring integration:

"cloudwatch:GetMetricData"
"cloudwatch:GetMetricStatistics"
"cloudwatch:ListMetrics"
"sts:GetCallerIdentity"
"tag:GetResources"
"tag:GetTagKeys"
"ec2:DescribeAvailabilityZones"

Complete list of permissions for cloud services

Name	Permissions
All monitored Amazon services Required	`cloudwatch:GetMetricData`,`cloudwatch:GetMetricStatistics`,`cloudwatch:ListMetrics`,`sts:GetCallerIdentity`,`tag:GetResources`,`tag:GetTagKeys`,`ec2:DescribeAvailabilityZones`
AWS Certificate Manager Private Certificate Authority	`acm-pca:ListCertificateAuthorities`
Amazon MQ
Amazon API Gateway	`apigateway:GET`
AWS App Runner	`apprunner:ListServices`
Amazon AppStream	`appstream:DescribeFleets`
AWS AppSync	`appsync:ListGraphqlApis`
Amazon Athena	`athena:ListWorkGroups`
Amazon Aurora	`rds:DescribeDBClusters`
Amazon EC2 Auto Scaling	`autoscaling:DescribeAutoScalingGroups`
Amazon EC2 Auto Scaling (built-in)	`autoscaling:DescribeAutoScalingGroups`
AWS Billing
Amazon Keyspaces
AWS Chatbot
Amazon CloudFront	`cloudfront:ListDistributions`
AWS CloudHSM	`cloudhsm:DescribeClusters`
Amazon CloudSearch	`cloudsearch:DescribeDomains`
AWS CodeBuild	`codebuild:ListProjects`
Amazon Cognito
Amazon Connect
Amazon Elastic Kubernetes Service (EKS)	`eks:ListClusters`
AWS DataSync	`datasync:ListTasks`
Amazon DynamoDB Accelerator (DAX)	`dax:DescribeClusters`
AWS Database Migration Service (AWS DMS)	`dms:DescribeReplicationInstances`
Amazon DocumentDB	`rds:DescribeDBClusters`
AWS Direct Connect	`directconnect:DescribeConnections`
Amazon DynamoDB	`dynamodb:ListTables`
Amazon DynamoDB (built-in)	`dynamodb:ListTables`,`dynamodb:ListTagsOfResource`
Amazon EBS	`ec2:DescribeVolumes`
Amazon EBS (built-in)	`ec2:DescribeVolumes`
Amazon EC2 API
Amazon EC2 (built-in)	`ec2:DescribeInstances`
Amazon EC2 Spot Fleet	`ec2:DescribeSpotFleetRequests`
Amazon Elastic Container Service (ECS)	`ecs:ListClusters`
Amazon ECS Container Insights	`ecs:ListClusters`
Amazon ElastiCache (EC)	`elasticache:DescribeCacheClusters`
AWS Elastic Beanstalk	`elasticbeanstalk:DescribeEnvironments`
Amazon Elastic File System (EFS)	`elasticfilesystem:DescribeFileSystems`
Amazon Elastic Inference
Amazon Elastic Map Reduce (EMR)	`elasticmapreduce:ListClusters`
Amazon Elasticsearch Service (ES)	`es:ListDomainNames`
Amazon Elastic Transcoder	`elastictranscoder:ListPipelines`
Amazon Elastic Load Balancer (ELB) (built-in)	`elasticloadbalancing:DescribeInstanceHealth`,`elasticloadbalancing:DescribeListeners`,`elasticloadbalancing:DescribeLoadBalancers`,`elasticloadbalancing:DescribeRules`,`elasticloadbalancing:DescribeTags`,`elasticloadbalancing:DescribeTargetHealth`
Amazon EventBridge	`events:ListEventBuses`
Amazon FSx	`fsx:DescribeFileSystems`
Amazon GameLift	`gamelift:ListFleets`
AWS Glue	`glue:GetJobs`
Amazon Inspector	`inspector:ListAssessmentTemplates`
AWS Internet of Things (IoT)
AWS IoT Analytics
Amazon Managed Streaming for Kafka	`kafka:ListClusters`
Amazon Kinesis Data Analytics	`kinesisanalytics:ListApplications`
Amazon Data Firehose	`firehose:ListDeliveryStreams`
Amazon Kinesis Data Streams	`kinesis:ListStreams`
Amazon Kinesis Video Streams	`kinesisvideo:ListStreams`
AWS Lambda	`lambda:ListFunctions`
AWS Lambda (built-in)	`lambda:ListFunctions`,`lambda:ListTags`
Amazon Lex	`lex:GetBots`
Amazon Application and Network Load Balancer (built-in)	`elasticloadbalancing:DescribeInstanceHealth`,`elasticloadbalancing:DescribeListeners`,`elasticloadbalancing:DescribeLoadBalancers`,`elasticloadbalancing:DescribeRules`,`elasticloadbalancing:DescribeTags`,`elasticloadbalancing:DescribeTargetHealth`
Amazon CloudWatch Logs	`logs:DescribeLogGroups`
AWS Elemental MediaConnect	`mediaconnect:ListFlows`
AWS Elemental MediaConvert	`mediaconvert:DescribeEndpoints`
AWS Elemental MediaPackage Live	`mediapackage:ListChannels`
AWS Elemental MediaPackage Video on Demand	`mediapackage-vod:ListPackagingConfigurations`
AWS Elemental MediaTailor	`mediatailor:ListPlaybackConfigurations`
Amazon VPC NAT Gateways	`ec2:DescribeNatGateways`
Amazon Neptune	`rds:DescribeDBClusters`
AWS OpsWorks	`opsworks:DescribeStacks`
Amazon Polly
Amazon QLDB	`qldb:ListLedgers`
Amazon RDS	`rds:DescribeDBInstances`
Amazon RDS (built-in)	`rds:DescribeDBInstances`,`rds:DescribeEvents`,`rds:ListTagsForResource`
Amazon Redshift	`redshift:DescribeClusters`
Amazon Rekognition
AWS RoboMaker	`robomaker:ListSimulationJobs`
Amazon Route 53	`route53:ListHostedZones`
Amazon Route 53 Resolver	`route53resolver:ListResolverEndpoints`
Amazon S3	`s3:ListAllMyBuckets`
Amazon S3 (built-in)	`s3:ListAllMyBuckets`
Amazon SageMaker Batch Transform Jobs
Amazon SageMaker Endpoint Instances	`sagemaker:ListEndpoints`
Amazon SageMaker Endpoints	`sagemaker:ListEndpoints`
Amazon SageMaker Ground Truth
Amazon SageMaker Processing Jobs
Amazon SageMaker Training Jobs
AWS Service Catalog
Amazon Simple Email Service (SES)
Amazon Simple Notification Service (SNS)	`sns:ListTopics`
Amazon Simple Queue Service (SQS)	`sqs:ListQueues`
AWS Systems Manager - Run Command
AWS Step Functions
AWS Storage Gateway	`storagegateway:ListGateways`
Amazon SWF	`swf:ListDomains`
Amazon Textract
AWS IoT Things Graph
AWS Transfer Family	`transfer:ListServers`
AWS Transit Gateway	`ec2:DescribeTransitGateways`
Amazon Translate
AWS Trusted Advisor
AWS API Usage
AWS Site-to-Site VPN	`ec2:DescribeVpnConnections`
AWS WAF Classic
AWS WAF
Amazon WorkMail	`workmail:ListOrganizations`
Amazon WorkSpaces	`workspaces:DescribeWorkspaces`

Example of JSON policy for one single service.

JSON policy for Amazon API Gateway

{
  "Version": "2012-10-17",
  "Statement": [
          {
                  "Sid": "VisualEditor0",
                  "Effect": "Allow",
                  "Action": [
                          "apigateway:GET",
                          "cloudwatch:GetMetricData",
                          "cloudwatch:GetMetricStatistics",
                          "cloudwatch:ListMetrics",
                          "sts:GetCallerIdentity",
                          "tag:GetResources",
                          "tag:GetTagKeys",
                          "ec2:DescribeAvailabilityZones"
                  ],
                  "Resource": "*"
          }
      ]
}

In this example, from the complete list of permissions you need to select

"apigateway:GET" for Amazon API Gateway
"cloudwatch:GetMetricData", "cloudwatch:GetMetricStatistics", "cloudwatch:ListMetrics", "sts:GetCallerIdentity", "tag:GetResources", "tag:GetTagKeys", and "ec2:DescribeAvailabilityZones" for All AWS cloud services.

Enable monitoring

To learn how to enable service monitoring, see Enable service monitoring.

View service metrics

You can view the service metrics in your Dynatrace environment either on the custom device overview page or on your Dashboards page.

View metrics on the custom device overview page

To access the custom device overview page

Go to Technologies & Processes.
Filter by service name and select the relevant custom device group.
Once you select the custom device group, you're on the custom device group overview page.
The custom device group overview page lists all instances (custom devices) belonging to the group. Select an instance to view the custom device overview page.

View metrics on your dashboard

You can also view metrics in the Dynatrace web UI on dashboards. There is no preset dashboard available for this service, but you can create your own dashboard.

To check the availability of preset dashboards for each AWS service, see the list below.

Preset dashboard availability list

AWS service	Preset dashboard
Amazon EC2 Auto Scaling (built-in)
AWS Lambda (built-in)
Amazon Application and Network Load Balancer (built-in)
Amazon DynamoDB (built-in)
Amazon EBS (built-in)
Amazon EC2 (built-in)
Amazon Elastic Load Balancer (ELB) (built-in)
Amazon RDS (built-in)
Amazon S3 (built-in)
AWS Certificate Manager Private Certificate Authority
All monitored Amazon services
Amazon API Gateway
AWS App Runner
Amazon AppStream
AWS AppSync
Amazon Athena
Amazon Aurora
Amazon EC2 Auto Scaling
AWS Billing
Amazon Keyspaces
AWS Chatbot
Amazon CloudFront
AWS CloudHSM
Amazon CloudSearch
AWS CodeBuild
Amazon Cognito
Amazon Connect
AWS DataSync
Amazon DynamoDB Accelerator (DAX)
AWS Database Migration Service (AWS DMS)
Amazon DocumentDB
AWS Direct Connect
Amazon DynamoDB
Amazon EBS
Amazon EC2 Spot Fleet
Amazon EC2 API
Amazon Elastic Container Service (ECS)
Amazon ECS Container Insights
Amazon Elastic File System (EFS)
Amazon Elastic Kubernetes Service (EKS)
Amazon ElastiCache (EC)
AWS Elastic Beanstalk
Amazon Elastic Inference
Amazon Elastic Transcoder
Amazon Elastic Map Reduce (EMR)
Amazon Elasticsearch Service (ES)
Amazon EventBridge
Amazon FSx
Amazon GameLift
AWS Glue
Amazon Inspector
AWS Internet of Things (IoT)
AWS IoT Things Graph
AWS IoT Analytics
Amazon Managed Streaming for Kafka
Amazon Kinesis Data Analytics
Amazon Data Firehose
Amazon Kinesis Data Streams
Amazon Kinesis Video Streams
AWS Lambda
Amazon Lex
Amazon CloudWatch Logs
AWS Elemental MediaTailor
AWS Elemental MediaConnect
AWS Elemental MediaConvert
AWS Elemental MediaPackage Live
AWS Elemental MediaPackage Video on Demand
Amazon MQ
Amazon VPC NAT Gateways
Amazon Neptune
AWS OpsWorks
Amazon Polly
Amazon QLDB
Amazon RDS
Amazon Redshift
Amazon Rekognition
AWS RoboMaker
Amazon Route 53
Amazon Route 53 Resolver
Amazon S3
Amazon SageMaker Batch Transform Jobs
Amazon SageMaker Endpoints
Amazon SageMaker Endpoint Instances
Amazon SageMaker Ground Truth
Amazon SageMaker Processing Jobs
Amazon SageMaker Training Jobs
AWS Service Catalog
Amazon Simple Email Service (SES)
Amazon Simple Notification Service (SNS)
Amazon Simple Queue Service (SQS)
AWS Systems Manager - Run Command
AWS Step Functions
AWS Storage Gateway
Amazon SWF
Amazon Textract
AWS Transfer Family
AWS Transit Gateway
Amazon Translate
AWS Trusted Advisor
AWS API Usage
AWS Site-to-Site VPN
AWS WAF Classic
AWS WAF
Amazon WorkMail
Amazon WorkSpaces

Available metrics

JobFlowId is the main dimension.

Name	Description	Unit	Statistics	Dimensions
AppsCompleted	The number of applications submitted to YARN that have completed	Count	Sum	JobFlowId, JobId
AppsCompleted		Count	Sum	JobFlowId
AppsFailed	The number of applications submitted to YARN that have failed to complete	Count	Sum	JobFlowId, JobId
AppsFailed		Count	Sum	JobFlowId
AppsKilled	The number of applications submitted to YARN that have been killed	Count	Sum	JobFlowId, JobId
AppsKilled		Count	Sum	JobFlowId
AppsPending	The number of applications submitted to YARN that are in a Pending state	Count	Sum	JobFlowId, JobId
AppsPending		Count	Sum	JobFlowId
AppsRunning	The number of applications submitted to YARN that are running	Count	Sum	JobFlowId, JobId
AppsRunning		Count	Sum	JobFlowId
AppsSubmitted	The number of applications submitted to YARN	Count	Sum	JobFlowId, JobId
AppsSubmitted		Count	Sum	JobFlowId
BackupFailed	Shows if the last backup failed. Set to `0` by default and updated to `1` if the previous backup attempt failed. This metric is only reported for HBase clusters.	Count	Sum	JobFlowId, JobId
BackupFailed		Count	Sum	JobFlowId
CapacityRemainingGB	The amount of remaining HDFS disk capacity	Count	Sum	JobFlowId, JobId
CapacityRemainingGB		Count	Sum	JobFlowId
ContainerAllocated	The number of resource containers allocated by the resource manager	Count	Sum	JobFlowId, JobId
ContainerAllocated		Count	Sum	JobFlowId
ContainerPendingRatio	The ratio (in numbers) of pending containers to containers allocated (`ContainerPendingRatio` = `ContainerPending` / `ContainerAllocated`). If `ContainerAllocated` = `0`, then `ContainerPendingRatio` = `ContainerPending`.	Count	Sum	JobFlowId, JobId
ContainerPendingRatio		Count	Sum	JobFlowId
ContainerPending	The number of containers in the queue that have not yet been allocated	Count	Sum	JobFlowId, JobId
ContainerPending		Count	Sum	JobFlowId
ContainerReserved	The number of containers reserved	Count	Sum	JobFlowId, JobId
ContainerReserved		Count	Sum	JobFlowId
CoreNodesPending	The number of core nodes waiting to be assigned (pending requests)	Count	Sum	JobFlowId, JobId
CoreNodesPending		Count	Sum	JobFlowId
CoreNodesRequested		Count	Sum	JobFlowId, JobId
CoreNodesRequested		Count	Sum	JobFlowId
CoreNodesRunning	The number of working core nodes	Count	Sum	JobFlowId, JobId
CoreNodesRunning		Count	Sum	JobFlowId
CoreUnitsRequested		Count	Sum	JobFlowId, JobId
CoreUnitsRequested		Count	Sum	JobFlowId
CoreUnitsRunning		Count	Sum	JobFlowId, JobId
CoreUnitsRunning		Count	Sum	JobFlowId
CoreVCPURequested		Count	Sum	JobFlowId, JobId
CoreVCPURequested		Count	Sum	JobFlowId
CoreVCPURunning		Count	Sum	JobFlowId, JobId
CoreVCPURunning		Count	Sum	JobFlowId
CorruptBlocks	The number of blocks that HDFS reports as corrupted	Count	Sum	JobFlowId, JobId
CorruptBlocks		Count	Sum	JobFlowId
DfsPendingReplicationBlocks	The status of block replication: blocks being replicated, age of replication requests, and unsuccessful replication requests	Count	Sum	JobFlowId, JobId
DfsPendingReplicationBlocks		Count	Sum	JobFlowId
HDFSBytesRead	The number of bytes read from HDFS	Count	Sum	JobFlowId, JobId
HDFSBytesRead		Count	Sum	JobFlowId
HDFSBytesWritten	The number of bytes written to HDFS	Count	Sum	JobFlowId, JobId
HDFSBytesWritten		Count	Sum	JobFlowId
HDFSUtilization	The percentage of HDFS storage currently used	Percent	Average	JobFlowId, JobId
HDFSUtilization		Percent	Average	JobFlowId
HbaseBackupFailed	Shows if the last backup failed. Set to `0` by default and updated to `1` if the previous backup attempt failed. This metric is only reported for HBase clusters.	Count	Minimum	JobFlowId, JobId
HbaseBackupFailed		Count	Minimum	JobFlowId
IsIdle	Indicates that a cluster is no longer performing work, but is still alive and accruing charges. Set to `1` if no tasks are running and no jobs are running, and to `0` otherwise. This value is checked at five-minute intervals and a value of `1` indicates only that the cluster was idle when checked, not that it was idle for the entire five minutes.	Count	Minimum	JobFlowId, JobId
IsIdle		Count	Minimum	JobFlowId
JobsFailed	The number of jobs in the cluster that have failed	Count	Sum	JobFlowId, JobId
JobsFailed		Count	Sum	JobFlowId
JobsRunning	The number of jobs in the cluster that are currently running	Count	Sum	JobFlowId, JobId
JobsRunning		Count	Sum	JobFlowId
LiveDataNodes	The percentage of data nodes that are receiving work from Hadoop	Count	Sum	JobFlowId, JobId
LiveDataNodes		Count	Sum	JobFlowId
LiveTaskTrackers	The percentage of task trackers that are functional	Percent	Average	JobFlowId, JobId
LiveTaskTrackers		Percent	Average	JobFlowId
MRActiveNodes	The number of nodes presently running MapReduce tasks or jobs. Equivalent to YARN metric `mapred.resourcemanager.NoOfActiveNodes`	Count	Sum	JobFlowId, JobId
MRActiveNodes		Count	Sum	JobFlowId
MRDecommissionedNodes	The number of nodes allocated to MapReduce applications that have been marked in a Decommissioned state	Count	Sum	JobFlowId, JobId
MRDecommissionedNodes		Count	Sum	JobFlowId
MRLostNodes	The number of nodes allocated to MapReduce that have been marked in a Lost state	Count	Sum	JobFlowId, JobId
MRLostNodes		Count	Sum	JobFlowId
MRRebootedNodes	The number of nodes available to MapReduce that have been rebooted and marked in a Rebooted state	Count	Sum	JobFlowId, JobId
MRRebootedNodes		Count	Sum	JobFlowId
MRTotalNodes	The number of nodes presently available to MapReduce jobs	Count	Sum	JobFlowId, JobId
MRTotalNodes		Count	Sum	JobFlowId
MRUnhealthyNodes	The number of nodes available to MapReduce jobs marked in an Unhealthy state	Count	Sum	JobFlowId, JobId
MRUnhealthyNodes		Count	Sum	JobFlowId
MapSlotsOpen	The unused map task capacity. This is calculated as the maximum number of map tasks for a given cluster, less the total number of map tasks currently running in that cluster.	Count	Sum	JobFlowId, JobId
MapSlotsOpen		Count	Sum	JobFlowId
MapTasksRemaining	The number of remaining map tasks for each job	Count	Sum	JobFlowId, JobId
MapTasksRemaining		Count	Sum	JobFlowId
MapTasksRunning	The number of running map tasks for each job	Count	Sum	JobFlowId, JobId
MapTasksRunning		Count	Sum	JobFlowId
MemoryAllocatedMB	The amount of memory allocated to the cluster	Count	Sum	JobFlowId, JobId
MemoryAllocatedMB		Count	Sum	JobFlowId
MemoryAvailableMB	The amount of memory available for allocation	Count	Sum	JobFlowId, JobId
MemoryAvailableMB		Count	Sum	JobFlowId
MemoryReservedMB	The amount of memory reserved for allocation	Count	Sum	JobFlowId, JobId
MemoryReservedMB		Count	Sum	JobFlowId
MemoryTotalMB	The total amount of memory in the cluster	Count	Sum	JobFlowId, JobId
MemoryTotalMB		Count	Sum	JobFlowId
MissingBlocks	The number of blocks in which HDFS has no replicas. These might be corrupt blocks.	Count	Sum	JobFlowId, JobId
MissingBlocks		Count	Sum	JobFlowId
MostRecentBackupDuration	The amount of time it took the previous backup to complete. This metric is set regardless of whether the last completed backup succeeded or failed. While the backup is ongoing, this metric returns the number of minutes after the backup started. This metric is only reported for HBase clusters.	Count	Sum	JobFlowId, JobId
MostRecentBackupDuration		Count	Sum	JobFlowId
PendingDeletionBlocks	The number of blocks marked for deletion	Count	Sum	JobFlowId, JobId
PendingDeletionBlocks		Count	Sum	JobFlowId
ReduceSlotsOpen	Unused reduce task capacity. This is calculated as the maximum reduce task capacity for a given cluster, less the number of reduce tasks currently running in that cluster.	Count	Sum	JobFlowId, JobId
ReduceSlotsOpen		Count	Sum	JobFlowId
ReduceTasksRemaining	The number of remaining reduce tasks for each job. If you have a scheduler installed and multiple jobs running, multiple graphs are generated.	Count	Sum	JobFlowId
ReduceTasksRunning	The number of running reduce tasks for each job. If you have a scheduler installed and multiple jobs running, multiple graphs are generated.	Count	Sum	JobFlowId, JobId
ReduceTasksRunning		Count	Sum	JobFlowId
RemainingMapTasksPerSlot	The ratio of the total map tasks remaining to the total map slots available in the cluster	Percent	Average	JobFlowId, JobId
RemainingMapTasksPerSlot		Percent	Average	JobFlowId
S3BytesRead	The number of bytes read from Amazon S3. This metric aggregates MapReduce jobs only, and does not apply for other workloads on EMR.	Count	Sum	JobFlowId, JobId
S3BytesRead		Count	Sum	JobFlowId
S3BytesWritten	The number of bytes written to Amazon S3. This metric aggregates MapReduce jobs only, and does not apply for other workloads on EMR.	Count	Sum	JobFlowId, JobId
S3BytesWritten		Count	Sum	JobFlowId
TaskNodesPending	The number of task nodes waiting to be assigned (pending requests)	Count	Sum	JobFlowId, JobId
TaskNodesPending		Count	Sum	JobFlowId
TaskNodesRequested		Count	Sum	JobFlowId, JobId
TaskNodesRequested		Count	Sum	JobFlowId
TaskNodesRunning	The number of working task nodes	Count	Sum	JobFlowId, JobId
TaskNodesRunning		Count	Sum	JobFlowId
TaskUnitsRequested		Count	Sum	JobFlowId, JobId
TaskUnitsRequested		Count	Sum	JobFlowId
TaskUnitsRunning		Count	Sum	JobFlowId, JobId
TaskUnitsRunning		Count	Sum	JobFlowId
TaskVCPURequested		Count	Sum	JobFlowId, JobId
TaskVCPURequested		Count	Sum	JobFlowId
TaskVCPURunning		Count	Sum	JobFlowId, JobId
TaskVCPURunning		Count	Sum	JobFlowId
TimeSinceLastSuccessfulBackup	The number of elapsed minutes after the last successful HBase backup started on your cluster. This metric is only reported for HBase clusters.	Count	Sum	JobFlowId, JobId
TimeSinceLastSuccessfulBackup		Count	Sum	JobFlowId
TotalLoad	The total number of concurrent data transfers	Count	Sum	JobFlowId, JobId
TotalLoad		Count	Sum	JobFlowId
TotalNodesRequested		Count	Sum	JobFlowId, JobId
TotalNodesRequested		Count	Sum	JobFlowId
TotalNodesRunning		Count	Sum	JobFlowId, JobId
TotalNodesRunning		Count	Sum	JobFlowId
TotalUnitsRequested		Count	Sum	JobFlowId, JobId
TotalUnitsRequested		Count	Sum	JobFlowId
TotalUnitsRunning		Count	Sum	JobFlowId, JobId
TotalUnitsRunning		Count	Sum	JobFlowId
TotalVCPURequested		Count	Sum	JobFlowId
TotalVCPURunning		Count	Sum	JobFlowId, JobId
TotalVCPURunning		Count	Sum	JobFlowId
UnderReplicatedBlocks	The number of blocks that need to be replicated one or more times	Count	Sum	JobFlowId, JobId
UnderReplicatedBlocks		Count	Sum	JobFlowId
YARNMemoryAvailablePercentage	he percentage of remaining memory available to YARN (`YARNMemoryAvailablePercentage` = `MemoryAvailableMB` / `MemoryTotalMB`)	Percent	Average	JobFlowId, JobId
YARNMemoryAvailablePercentage		Percent	Average	JobFlowId