Carbon Impact

Latest Dynatrace

Dynatrace Carbon Impact enables you to calculate and monitor your IT carbon footprint. It tracks, reports, and helps you reduce the carbon emissions resulting from your cloud and on-premises infrastructure electricity consumption.

Carbon Impact capabilities

  • Calculates your IT carbon footprint at the data center and host levels.
  • Translates utilization metrics—including CPU, memory, disk, and network I/O—into energy consumption in kWh and CO2 equivalents (CO2e).
  • Reports carbon dioxide emissions in a single interface—the Carbon Impact landing page.
  • Identifies opportunities to reduce carbon emissions.

Prerequisites

The following are the Dynatrace prerequisites for Carbon Impact.

  • Existing infrastructure metrics on hosts

    Carbon dioxide emissions and energy consumption are calculated for hosts that are set up and monitored with OneAgent. Energy calculations are based on observed infrastructure metrics.

    1. In Dynatrace, go to Hosts.
    2. Check that you have at least one host instrumented.
  • Installation and user permissions

    The user who installs and sets up the app needs to have privileges to install apps, set up automation, and read or write business events.

    Copy the block below to add IAM policy statements based on the required permissions. Note that these statements are for the user who installs and sets up the app.

    ALLOW app-engine:functions:run;
    ALLOW app-settings:objects:read;
    ALLOW app-settings:objects:write;
    ALLOW automation:workflows:read;
    ALLOW automation:workflows:write;
    ALLOW environment-api:entities:read;
    ALLOW iam:bindings:read;
    ALLOW notification:self-notifications:read;
    ALLOW notification:self-notifications:write;
    ALLOW state:app-states:read, state:app-states:write;
    ALLOW storage:bizevents:read;
    ALLOW storage:buckets:read;
    ALLOW storage:entities:read;
    ALLOW storage:events:write;
    ALLOW storage:filter-segments:read;
    ALLOW storage:metrics:read;

    The Dynatrace Hub entry for Carbon Impact lists the required permissions to use the app.

    1. In Hub Hub, select Carbon Impact.
    2. Go to the Technical Information tab.
    3. Check the User permissions section for a list of all the permissions you need to include in the policies bound to user groups that are allowed to use Carbon Impact.

    For more information, see Manage user permissions with IAM policies and Workflow authorization settings.

  • Grant permissions to Workflows

    Check if Workflows has the required permissions to run automations. Open Workflows Workflows and select Settings > Authorization settings in the upper-right corner. Enable the settings below.

    • app-engine:apps:run
    • app-engine:functions:run
    • app-settings:objects:read
    • automation:workflows:read
    • automation:workflows:write
    • environment-api:entities:read
    • iam:bindings:read
    • state:app-states:read
    • state:app-states:write
    • storage:bizevents:read
    • storage:buckets:read
    • storage:entities:read
    • storage:events:write
    • storage:metrics:read

Install Carbon Impact

To set up Carbon Impact

  1. From your Dynatrace environment, go to Dynatrace Hub and install Carbon Impact.
  2. Open Carbon Impact for the first time and turn on Enable energy and emission data generation.
  3. You will be redirected to Carbon Impact once the last hour of data is calculated.

To deactivate the data generation, select Settings in the upper-right corner.

Data storage and retention

Carbon Impact calculates consumption data every hour, based on infrastructure monitoring metrics.

  • The first data point is created an hour after installation.
  • Consumption data is stored in Grail with a default retention period of 35 days. You can adjust this retention time by creating a custom bucket based on the event.types defined in the Dynatrace Semantic Dictionary.
Data schema

The data generated by Carbon Impact follows the schema defined in the Dynatrace Semantic Dictionary.

Data units

The units of data generated are as follows.

  • Electricity consumption is calculated in watt-hours (Wh).
  • Carbon dioxide emission is calculated in grams (g) but displayed in charts on the landing page in kilograms (kg).

Methodology to estimate electricity consumption

The deviation from the measured power cannot be reliably determined due to the large variety of devices in the market and lack of statistical analysis.

Electricity consumption and carbon dioxide emission are calculated based on guidelines from the Sustainable Digital Infrastructure Alliance (SDIA) with some contributions from the Cloud Carbon Footprint project and internal research from Dynatrace. All of the methodology is aligned with the GHG Protocol.

Energy is estimated based on the utilization metrics captured by Dynatrace OneAgent (in all modes for hosts) for CPU, memory, storage IO, and network. This estimation is performed every hour for each instrumented host through an automation.

The formula to estimate energy draw includes several parameters from external data sources and some assumptions.

External data sources

Assumptions for electricity consumption

  • CPU energy measurement:

    • Over a long period of time, a CPU running at 100% utilization will consume its TDP in power draw.
    • The base power draw of a CPU is 1/3 of its TDP It is known that true idle for CPUs is much lower, but CPU utilization hovers around a constant 10-20% for server applications, so true idle can never be reached.
    • One cloud vCPU equals one CPU thread
  • Memory energy measurement:

    • Random Access Memory uses a constant of 5 W of power per 8 GB regardless of utilization.
  • Storage energy measurement:

    • The power draw of a terabye of SSD storage is 1.2 W
    • All storage is assumed to be on an SSD
    • Storage power draw is constant
    • Cloud storage is replicated twice (or more)
  • Network energy measurement:

    • 1 GB of networking (upload or download) consumes 1 W if sent outside of the local network.
    • 1 GB of networking (upload or download) consumes 0.12 W if remaining within a local network.

Power Usage Effectiveness (PUE)

The final result of the formula includes the energy consumed by hardware and cooling. Cooling is included in a standard metric, Power Usage Effectiveness (PUE—not displayed directly in Carbon Impact but available in the raw data generated). This metric, used to measure the energy efficiency of a data center, is a ratio between the total power consumption of a data center—including cooling, lighting, and support equipment energy use—and the power consumed by the IT infrastructure. It describes how much energy IT equipment uses compared to other electrical devices.

The Power Usage Effectiveness varies depending on a data center, but its predefined values for the main cloud providers are publicly available. The ones used in the calculation are:

Carbon emissions calculation

Carbon emissions are estimated in CO2 equivalent grams. The calculation is multiplying carbon intensity factors by total energy consumed calculated with previous methodology.

Assumptions for carbon emissions

  • A data center’s carbon intensity is equal to the average carbon intensity of the country where it is located. This means that for large countries the national differences are not considered.
  • Carbon intensity values used on calculations are an annual average.

Carbon intensity data sources

The Carbon Impact landing page

The Carbon Impact landing page (Carbon Impact tab) provides an at-a-glance view of your IT carbon footprint.

  • The Carbon footprint summary reports total CO2e emissions for the selected and preceding timeframes for quick interval-based comparisons.

  • Optimization recommendations report idle hosts and underutilized hosts.

    You can adjust thresholds influencing these calculations to your needs in app Settings.

  • The table groups emissions by data center, listing your cloud and on-premises instances.

  • The Accumulated carbon footprint over time chart shows the accumulated carbon footprint and energy consumption over time.

  • The Carbon vs business KPI chart allows you to compare carbon emissions over time with a business key performance indicator (KPI) of your choice, derived from any of your captured business events.

Data centers and hosts

Data center emissions on the landing page shows energy and CO2e consumption per data center. Select a data center name to view details of its hosts.

The Hosts tab details energy and CO2e consumption per host. You can narrow your search using filters. For example, you can view underutilized hosts in a specific data center or top CO2e emitters.

Expand a hostname to see key infrastructure metrics: CPU in use, memory in use, and network traffic. Select a hostname to view the host details page.

Carbon Impact is automatically connected to Dynatrace Smartscape® topology modeling, so it’s easy to see the host details or use Notebooks for ad hoc analysis with DQL.

Advanced Analytics with Notebooks

You can use Notebooks Notebooks to analyze your carbon footprint data.

Example 1

Get all the business events from your OneAgent-monitored environment. The results of this query display a list of metrics repored by OneAgent instances for the selected timeframe in your Notebooks query.

fetch bizevents
| filter contains (event.provider, "dynatrace.biz.carbon")
| filter contains (event.type, "carbon.measurement")

The event.type == “carbon.report” is no longer supported after version 0.28.0. But old data can be explored blending new and old event.types, on an alternative query:

fetch bizevents
| filter contains (event.provider, "dynatrace.biz.carbon")
| filter in(event.type,"carbon.report", "carbon.measurement")

Example 2

Get the energy and CO2 consumption per host.

fetch bizevents
| filter contains (event.provider, "dynatrace.biz.carbon")
| filter contains (event.type, "carbon.measurement")
| summarize totalEnergy = sum(energy.measurement.computing), totalCO2e = sum(carbon.emissions), by:dt.entity.host

The attributes emissions and totalElectricConsumption are deprecated from the data model since version 0.28.0. In case, the query wants to make calculations with old and new scheme there is this alternative query:

fetch bizevents
| filter contains (event.provider, "dynatrace.biz.carbon")
| filter contains (event.type, "carbon.measurement")
| fieldsAdd energy.measurement.computing=if(isNotNull(totalElectricConsumption), totalElectricConsumption, else:energy.measurement.computing)
| fieldsAdd carbon.emissions=if(isNotNull(emissions), emissions, else:carbon.emissions)
| summarize totalEnergy = sum(energy.measurement.computing), totalCO2e = sum(carbon.emissions), by:dt.entity.host

The table below lists sample results of this query. totalCO2e is in grams and totalEnergy is in watt-hours.

dt.entity.host

totalEnergy

totalCO2e

HOST-1

14.02

7.55

HOST-2

21.65

11.67

HOST-3

48.79

26.30

Example 3

Get the emissions from the top ten hosts that are generating more CO2 eq in grams including the names of the hosts taking advantage of the lookup command.

fetch bizevents
| filter event.provider == "dynatrace.biz.carbon"
| filter contains (event.type, "carbon.measurement")
| fields dt.entity.host, carbon.emissions
| lookup sourceField:dt.entity.host, lookupField:id, prefix:"host.",[fetch dt.entity.host ]
| summarize totalEmissions = sum(carbon.emissions), by:{host.entity.name, dt.entity.host}
| sort (totalEmissions) desc
| limit 10

Shows results in a table view inside a Notebook.

Example 4

Distribute Carbon emissions by process group using CPU usage.

fetch dt.entity.process_group_instance
| fields entity.name, dt.entity.host=belongs_to[dt.entity.host],
dt.entity.process_group= instance_of[dt.entity.process_group],
dt.entity.process_group_instance = id
| lookup sourceField:dt.entity.host, lookupField:dt.entity.host, [fetch bizevents
| filter event.type=="carbon.measurement"
| summarize sum = sum(carbon.emissions), by:{dt.entity.host}], prefix:"carbon."
| lookup [timeseries ts=avg(dt.process.cpu.usage),
by:{dt.entity.process_group_instance}
| fields cpu_sum= arraySum(ts), dt.entity.process_group_instance],sourceField:dt.entity.process_group_instance,
lookupField:dt.entity.process_group_instance
| fieldsAdd cpu_sum=lookup.cpu_sum
| fieldsRemove "lookup*"
| fieldsRename carbon_per_host=carbon.sum
| lookup [timeseries ts=avg(dt.process.cpu.usage), by:{dt.entity.process_group_instance}
| lookup [fetch dt.entity.process_group_instance], sourceField:dt.entity.process_group_instance, lookupField:id
| fields ts, dt.entity.process_group_instance, dt.entity.host = lookup.belongs_to[dt.entity.host]
| fieldsAdd cpusum = arraySum(ts)
| summarize cpuHostSum = sum(cpusum), by:{dt.entity.host}],
sourceField:dt.entity.host, lookupField:dt.entity.host
| fieldsAdd carbon_per_process = (cpu_sum/lookup.cpuHostSum) * carbon_per_host
| fields dt.entity.process_group, dt.entity.process_group_instance,
dt.entity.host, carbon_per_process
| sort carbon_per_process desc

Query steps explained

  • Line 1

    fetch dt.entity.process_group_instance

    The table for process_group_instance is fetched.

  • Line 2

    | fields entity.name, dt.entity.host=belongs_to[dt.entity.host],
    dt.entity.process_group= instance_of[dt.entity.process_group],
    dt.entity.process_group_instance = id

    Collects from topology the hosts that are running those process group instances and the process group where those instances belong. Filter by hostId, if you only want to get data from that particular host.

  • Line 3

    | lookup sourceField:dt.entity.host, lookupField:dt.entity.host, [fetch bizevents
    | filter event.type=="carbon.measurement"
    | summarize sum = sum(carbon.emissions), by:{dt.entity.host}], prefix:"carbon."

    The lookup command collects all emissions from the hosts, matching with the hosts that are running processes. The fields that are collected from the lookup command will come with a prefix of carbon.

  • Line 4

    | lookup [timeseries ts=avg(dt.process.cpu.usage),
    by:{dt.entity.process_group_instance}
    | fields cpu_sum= arraySum(ts), dt.entity.process_group_instance],sourceField:dt.entity.process_group_instance,
    lookupField:dt.entity.process_group_instance

    The lookup command collects the average values of the metrics CPU usage from each process instance in the system.

  • Line 5

    | fieldsAdd cpu_sum=lookup.cpu_sum

    Creates a new field for the CPU usage of the process instance.

  • Line 6

    | fieldsRemove "lookup*"

    Removes all the fields that came from the last lookup (the default prefix by is lookup).

  • Line 7

    | fieldsRename carbon_per_host=carbon.sum

    Renames a field for a better understanding of the data contained.

  • Line 8

    | lookup [timeseries ts=avg(dt.process.cpu.usage), by:{dt.entity.process_group_instance}
    | lookup [fetch dt.entity.process_group_instance], sourceField:dt.entity.process_group_instance, lookupField:id
    | fields ts, dt.entity.process_group_instance, dt.entity.host = lookup.belongs_to[dt.entity.host]
    | fieldsAdd cpusum = arraySum(ts)
    | summarize cpuHostSum = sum(cpusum), by:{dt.entity.host}],
    sourceField:dt.entity.host, lookupField:dt.entity.host

    Extracts host CPU usage based on the total amount of it by processes through lookup on top of the result of a DQL query.

    Inside lookup, we have a timeseries command that extracts the average CPU usage for each process. The result of timeseries is enriched with one lookup to collect the hosts, and summarize the total use of CPU of each host comming from each process instance running on it.

  • Line 9

    | fieldsAdd carbon_per_process = (cpu_sum/lookup.cpuHostSum) * carbon_per_host

    A new field is calculated to distribute carbon emissions per process, multiplying the total emissions generated on a host, by the proportion of CPU usage of the process versus the total CPU usage of the host. All these parameters have been calculated during the previous steps of the DQL query.

  • Line 10

    | fields dt.entity.process_group, dt.entity.process_group_instance,
    dt.entity.host, carbon_per_process

    Selects only the relevant fields as an output of the DQL query.

  • Line 11

    | sort carbon_per_process desc

    Sorts the results by the processes that generate more emissions.

Settings

Select the gear icon in the upper-right corner of the page to access the app Settings menu.

You can deactivate the energy and emission data generation from here.

Optimization options

The Settings page offers two optimization options.

Idling optimization—Enables you to detect hosts that are turned on and running but not actively used or performing tasks.

Sizing optimization—Enables you to detect hosts that are not idle but use fewer resources than initially planned. You can also make changes to their configuration.

In both cases, you need to set parameters that define a host as idle or underutilized. Default values are based on Google Cloud criteria.

  • An idle host:

    • Uses less than 3% of CPU.
    • Has outgoing network traffic below 100.000 bytes/s or incoming network traffic below 120.000 bytes/s.
  • An underutilized host:

    • Uses more than 3% and less than 30% of CPU.
    • Uses less than 50% of memory.

You can adjust the values to your requirements.

Business health performance indicator

You can define a business health performance indicator of your choice, derived from any of your captured business events.

For example, your business health performance indicator could be the total revenue generated by all the orders completed in a purchase process that is captured as business events containing an attribute for the revenue value. This KPI is displayed compared to carbon emissions in the Carbon vs business KPI chart in the lower-right corner of the app landing page.

To set up a Business health performance indicator

  1. Select app Settings in the upper-right corner and scroll to the Business health performance indicator section.

  2. Select the business event (Select event) on which you want to base your business health performance indicator or Key Performance Indicator (KPI).

  3. optional Select the KPI from the list of available attributes (Select KPI). The listed attributes change based on the event selected. If no KPI is selected, the business health performance indicator is calculated as the count of business events.

  4. Define the unit of the KPI to be displayed in charts and single-value tiles. This unit is displayed in the right Y-axis of the Carbon vs business KPI chart. Be sure to use a unit that matches your selected KPI and limit the length of the string so that it's displayed correctly in the chart.

  5. Save your settings.