OS services monitoring

Dynatrace provides out-of-the-box availability monitoring of OS services.

You can monitor hosts in full-stack monitoring mode or use lightweight monitoring modes. For more information, see Infrastructure and Discovery monitoring modes.

Requirements

Linux To monitor relations between processes and OS services, systemd version 230+ is required on the monitored host.
Linux To improve performance by less frequent service refresh, systemd version 250+ is required.

Monitor a service

To monitor an OS service, perform the following steps.

Access OS services monitoring

Add service monitoring policy

Select services you want to monitor

Add custom properties

Access OS services monitoring

In Dynatrace, go to OS services monitoring for the level you are configuring.

Go to Hosts or Hosts Classic (latest Dynatrace).
Find and select your host to display the host overview page.
In the upper-right corner of the host overview page, select More (…) > Settings.

In the host settings, select OS services monitoring.

Go to Deployment Status and then select OneAgents.
On the OneAgent deployment page, turn off Show new OneAgent deployments.
Filter the table by Host group and select the host group you want to configure.
The Host group property is not displayed when the selected host doesn't belong to any host group.
This displays the OneAgent deployment page filtered by the selected host group. Each listed host has a Host group: <group name> link, where <group name> is the name of the host group that you want to configure.
Select the host group name in any row.
As you have filtered by host group, all displayed hosts go to the same host group.

In the host group settings, select OS services monitoring.

Go to Settings > Monitoring > OS services monitoring.

Add service monitoring policy

Based on the service state and the rules, the service monitoring policy defines the way Dynatrace is monitoring your service. By default, Dynatrace comes with Auto-start Windows OS Services and Auto-start Linux OS Services policies for auto-started Windows and Linux services with failed status.

The order of service monitoring policies is important. Policies that are higher on the list will proceed before those on lower positions until they are fulfilled. This allows for the creation of selective alerts or monitoring with minimal policies. If you want to monitor all auto-started services and not just those created by Microsoft, you need to add a policy with disabled alerting and/or monitoring that will verify if the manufacturer is Microsoft.

On OS services monitoring for the level you are configuring based on your OS, select Add policy and define the policy, which is a collection of rules.
System: select your operating system.
Rule name: enter the name that will be displayed in the Summary field.
Monitor: decide whether you want to monitor service availability using the OS service availability (builtin:osservice.availability) metric. If available, the metric sends the service status every 10 seconds. The status is carried by the Service status (dt.osservice.status) dimension.
Note that the metric consumes DDUs. For more information, see DDU consumption.
Alert: decide whether you want alerting for your policy.
OneAgent version 1.257+ Alert if service is not installed: whether you want to receive alerts about OS services that are not installed on the host.
Service status: set the service status for which an alert should be triggered.
You can use logic operations to monitor the service status. For example, $eq(running) monitors the running service state.
Available logic operations:
$not($eq(paused)) – Matches services that are in state different from paused.
$or($eq(paused),$eq(running)) – Matches services that are either in paused or running state.
These are the service statuses you can monitor. Use one of the following values as a parameter for this condition:
running
stopped
start_pending
stop_pending
continue_pending
pause_pending
paused
You can use logic operations to monitor the service status. For example, $eq(active) monitors the active service state.
Available logic operations:
$not($eq(active)) – Matches services with state different from active.
$or($eq(inactive),$eq(failed)) – Matches services that are either in inactive or failed state.
These are the service statuses you can monitor. Use one of the following values as a parameter for this condition:
reloading
activating
deactivating
failed
inactive
active
optional OneAgent version 1.257+ Alerting delay: the number of 10-second measurement cycles for a service to be in configured state before an event is generated.

Next, you need to select which services you want to monitor based on service properties.

Select services you want to monitor

Select Add rule.
optional Rule scope: select either OS Service or Host. By default, the OS Service option is selected.

If you selected Host:

OneAgent version 1.277+ Custom metadata used for matching:
- Key specifies the metadata key you want to match
- Condition in which you can define a string that:
  - $contains(production) – Matches if production appears anywhere in the host metadata value.
  - $eq(production) – Matches if production matches the host metadata value exactly.
  - $prefix(production) – Matches if production matches the prefix of the host metadata value.
  - $suffix(production) – Matches if production matches the suffix of the host metadata value.
  Available logic operations:
  - $not($eq(production)) – Matches if the host metadata value is different from production.
  - $and($prefix(production),$suffix(main)) – Matches if host metadata value starts with production and ends with main.
  - $or($prefix(production),$suffix(main)) – Matches if host metadata value starts with production or ends with main.
  When including special characters such as brackets ( and ) within your matching expressions, escape these characters with a tilde ~. For example, to match a metadata value that includes brackets, like my(amazing)property, you would write $eq(my~(amazing~)property).

If you selected OS Service, proceed according to your operating system.

Service property used for matching:
- Display name visible for system user.
- Path to the binary used by OS service.
- Manufacturer from the binary file of the service.
- Service Name
- Startup Type

A monitoring rule may consist of multiple detection rules. All detection rules must be satisfied for the OS Service to match, as a logical AND operation is applied across all specified conditions.

With these properties, we define the services to be monitored based on:

Display name visible to a system user
Path to the service binary
Manufacturer of the service
Service name representing the name or ID under which OS service is recognized
Condition in which you can define a string that:
- Starts with – use $prefix qualifier, for example $prefix(ss).
- Ends with – use $suffix qualifier, for example $suffix(hd).
- Equals – use $eq qualifier, for example $eq(sshd).
- Contains - use $contains qualifier , for example $contains(ssh).
Available logic operations:
- $not($eq(sshd)) – Matches if the service's property value is different from sshd.
- $and($prefix(ss),$suffix(hd)) – Matches if service's property value starts with ss and ends with hd.
- $or($prefix(ss),$suffix(hd)) – Matches if service's property value starts with ss or ends with hd.
When including special characters such as brackets ( and ) within your matching expressions, escape these characters with a tilde ~. For example, to match a property value that includes brackets, like my(amazing)property, you would write $eq(my~(amazing~)property).

With this property we define the services to be monitored based on their startup type.

Condition in which you can define a string that:
- Equals – use $eq qualifier, for example $eq(manual).
Available logic operations:
- $not($eq(auto)) – Matches services with startup type different from Automatic.
- $or($eq(auto),$eq(manual)) – Matches if service's startup type is either Automatic or Manual.
Use one of the following values as a parameter for this condition:
- manual for Manual - the service starts only if needed or if you invoke something to start the service.
- manual_trigger for Manual (Trigger Start) - the service starts along with the startup of another service.
- auto for Automatic - the service starts automatically.
- auto_delay for Automatic (Delayed Start) - the service startup is delayed until the system has finished booting.
- auto_trigger for Automatic (Trigger Start) - the service starts automatically on startup and may be started or stopped due to certain operating system events.
- auto_delay_trigger for Automatic (Delayed Start, Trigger Start)
- disabled for Disabled

Service property used for matching:

Service Name
Startup Type

With this property we define the services to be monitored based on the service name.

Condition in which you can define a string that:

$contains(ssh) – Matches if ssh appears anywhere in the service's property value.
$eq(sshd) – Matches if sshd matches the service's property value exactly.
$prefix(ss) – Matches if ss matches the prefix of the service's property value.
$suffix(hd) – Matches if hd matches the suffix of the service's property value.

Available logic operations:

$not($eq(sshd)) – Matches if the service's property value is different from sshd.
$and($prefix(ss),$suffix(hd)) – Matches if service's property value starts with ss and ends with hd.
$or($prefix(ss),$suffix(hd)) – Matches if service's property value starts with ss or ends with hd.

With this property, we define the services to be monitored based on their startup type.

Condition in which you can define a string that:

$eq(enabled) – Matches services with startup type equal to enabled.

Available logic operations:

$not($eq(enabled)) – Matches services with startup type different from enabled.
$or($eq(enabled),$eq(disabled)) - Matches services that are either enabled or disabled.

Use one of the following values as a parameter for this condition:


`enabled`, `enabled-runtime`	The service is marked as ready for startup.
`static`	The unit file is not enabled and has no provisions for enabling in the install unit file section. Static units are installed as dependencies and can only be masked, but are not always executed. They will be executed only if another unit depends on them or if they're manually started.
`disabled`	The unit file is not enabled, but it contains an install section with installation instructions.

Add custom properties optional

OneAgent version 1.247+

Dynatrace version 1.247+

Select Add property to specify a custom key-value property for the policy.
For example, a property with a Key set to custom.message and Value set to The {dt.osservice.name} is with status {dt.osservice.status} (including placeholders {dt.osservice.name} and {dt.osservice.status}) will extract the OS service name and status values once the rule is triggered. If the placeholder substitution fails, both the key and the value will be unavailable.
For OneAgent version 1.255+, the {dt.osservice.display_name} placeholder is available.
Additionally, you can utilize specific dt flags in the Key field to tailor the behavior of problem notifications and Davis AI analysis:
- dt.event.title: Customizes the title of the problem.
- dt.event.description: Provides a detailed description for the problem.
- dt.event.allow_davis_merge: Controls Davis AI's decision to merge events based on your settings.
Select Save changes.

Configure automatically applied tags and management zones

Dynatrace version 1.257+

You need to manually configure automatically applied tags and management zones for OS service entities, as Dynatrace no longer automatically associates OS services with their respective hosts.

To set up automatic tagging for OS services

Go to Settings > Tags > Automatically applied tags.
Select Create tag.
Enter a Tag name and an optional Description for your tag.
Select Add a new rule to define the conditions for the tag.
Rule type: select Entity selector.
Entity selector: define the conditions under which the tag should be applied to OS services.
- To assign a tag based on the host where the OS service is running, use the following format:
```
type(os:service),fromRelationship.runsOn(type(HOST),entityName.startsWith("<host name>"))
```
  The tag will be applied to any OS service that runs on a host with a name that starts with <host name>.
- To assign a tag based on the process group instance where the OS service is running, use the following format:
```
type(os:service),toRelationship.runsOn(type(PROCESS_GROUP_INSTANCE),entityName.startsWith("<PGI name>"))
```
  The tag will be applied to any OS service that runs on a PGI with a name that starts with <PGI name>.
Select Save changes.

To set up management zones for OS services

Go to Settings > Preferences > Management zones.
Select Add new management zone.
Enter a Management zone name and an optional Description for your management zone.
Select Add a new rule to define the conditions for the management zone.
Rule type: select Entity selector.
Entity selector: define the conditions under which the management zone should be applied to OS services.
- To assign a management zone based on the host where the OS service is running, use the following format:
```
type(os:service),fromRelationship.runsOn(type(HOST),entityName.startsWith("<host name>"))
```
  The management zone will be applied to any OS service that runs on a host with a name that starts with <host name>.
- To assign a management zone based on the process group instance where the OS service is running, use the following format:
```
type(os:service),toRelationship.runsOn(type(PROCESS_GROUP_INSTANCE),entityName.startsWith("<PGI name>"))
```
  The management zone will be applied to any OS service that runs on a PGI with a name that starts with <PGI name>.

Manage monitored OS services

To manage the OS services

In Dynatrace, go to OS services monitoring for the level you are configuring.
1. Go to Hosts or Hosts Classic (latest Dynatrace).
2. Find and select your host to display the host overview page.
3. In the upper-right corner of the host overview page, select More (…) > Settings.
1. In the host settings, select OS services monitoring.
1. Go to Deployment Status and then select OneAgents.
2. On the OneAgent deployment page, turn off Show new OneAgent deployments.
3. Filter the table by Host group and select the host group you want to configure.
  The Host group property is not displayed when the selected host doesn't belong to any host group.
  This displays the OneAgent deployment page filtered by the selected host group. Each listed host has a Host group: <group name> link, where <group name> is the name of the host group that you want to configure.
4. Select the host group name in any row.
  As you have filtered by host group, all displayed hosts go to the same host group.
1. In the host group settings, select OS services monitoring.
Go to Settings > Monitoring > OS services monitoring.
The OS services you monitor are displayed in a table under the Add policy button.
- To stop monitoring a listed service, turn the Enabled setting off.
- To delete a service from the table, select the delete button in the Delete column
- To view and edit details, select the expand control in the Details column.

Monitor service availability

Dynatrace version 1.243+

OneAgent version 1.243+

The Host overview page contains the OS services analysis section listing the OS services for which any policy (with active alerting or monitoring) is fulfilled.

Go to Hosts or Hosts Classic (latest Dynatrace).
In the OS services analysis section, select the service name to open the Service overview page.

For more information, see Host monitoring with Dynatrace

Configure at scale using Settings API

You can use the Settings API to configure your service availability monitoring at scale.

To learn the schema, use GET a schema with builtin:os-services-monitoring as the schemaId.
Based on the builtin:os-services-monitoring schema, create your configuration object.
To create your configuration, use POST an object.