Monitor service message processing
- Latest Dynatrace
- How-to guide
- 8-min read
Modern distributed systems rely heavily on asynchronous communication through message queues and streaming platforms. Understanding message flow, throughput, and processing failures is critical for maintaining system reliability.
Monitoring service message processing in Dynatrace addresses this need by providing comprehensive visibility into message-based transactions across your microservices architecture, helping you track throughput, identify bottlenecks, and resolve processing issues.
Sometimes, temporary queues are created with numerous unique identifiers in their names, generating thousands of distinct queue names that make aggregations unusable. For instructions on how to normalize queue names in the Messaging view, refer to Reduce cardinality of temporary queue names.
Coming from Classic Queue entities? See Upgrade queue monitoring to service message processing for how each Classic capability maps to its replacement, and how to rebuild your queue dashboards and alerts.
What is service message processing monitoring?
In Dynatrace, message processing refers to any transaction involving message queues or streaming platforms like Kafka, RabbitMQ, ActiveMQ, and AWS SQS. This includes messages published to topics, received from queues, and processed by consuming services.
Service message processing monitoring helps you understand how messages flow through your distributed systems, detect processing bottlenecks, and identify failures in asynchronous communication patterns.
You can access service message processing directly from 
Services, or through custom dashboards and alerts configured to monitor messaging metrics.
Service message processing monitoring targets SREs, developers, and platform engineers who need visibility into asynchronous communication patterns and message processing health.
Service message processing monitoring provides:
- Real-time visibility into message publish, receive, and process rates
- Identification of processing bottlenecks and processing lag
- Tracking of message flow between producing and consuming services
- Error rate monitoring for failed message processing
- Infrastructure dependency mapping for messaging platforms
- Integration with distributed traces for root cause analysis
Metrics reference
Dynatrace provides four core metrics for monitoring message processing:
| Metric | Description | Unit |
|---|---|---|
| Messages sent to queues/topics | count |
| Messages received from queues/topics | count |
| Messages successfully processed | count |
| Messages that failed processing | count |
Key dimensions
| Dimension | Description | Example values |
|---|---|---|
| Queue or topic name |
|
| Service identifier |
|
| Messaging platform |
|
| AWS account identifier |
|
| AWS region |
|
| Kubernetes cluster name |
|
| Kubernetes namespace |
|
Get started
To begin monitoring service message processing
- Go to Services > Explorer > Messaging.
- Select the service and queue/topic you want to monitor.
- View Publish rate, Receive rate, and Process rate in the time series charts.
- Identify lag by comparing receive rates to process rates.
- Drill down to distributed traces to investigate specific message failures.
List services that produce or consume messages
Every dt.service.messaging.* data point carries the Smartscape service ID in the dt.smartscape.service dimension. Combined with messaging.destination.name, this lets you reconstruct the full producer-to-consumer relationship for any queue or topic:
- A service reporting
dt.service.messaging.publish.countfor a queue is a producer for that queue. - A service reporting
dt.service.messaging.receive.countfor the same queue is a consumer of that queue.
Because both sides carry the same messaging.destination.name, that dimension is the join key that connects them.
To list all services interacting with a specific queue, together with their role:
timeseries {published = sum(dt.service.messaging.publish.count, scalar: true),received = sum(dt.service.messaging.receive.count, scalar: true)},by: { dt.smartscape.service, messaging.destination.name, messaging.system },nonempty: true, union: true| filter messaging.destination.name == "orderEvents"| fieldsAdd `Service` = getNodeField(dt.smartscape.service, "name")| summarize {`Messages published` = sum(published),`Messages received` = sum(received)},by: { `Service`, messaging.destination.name, messaging.system }| sort `Messages published` + `Messages received` desc
Replace orderEvents with your queue or topic name. Omit the filter line to get an overview of every queue in your environment and the services attached to it.
Primary Grail fields, including Kubernetes, AWS, Azure, and GCP dimensions, are present on the metrics themselves, so you can scope any messaging analysis to a cluster, namespace, or cloud account without additional configuration.
Service Map Early Access
Service Map is part of the Explorer (Early Access) view in Services. To enable it, see New Explorer view.
The queries above also drive queue visualization in Explore service topology with Service Map. Rather than requiring you to write DQL, Service Map renders the same producer-to-consumer relationships as a topology graph.
On the map
- Queues and topics appear as Queue nodes with their own icon, visually distinct from service and database nodes
- Messaging flows are drawn as edges between your services and those queue nodes, so a producer, its queue, and the consuming service are connected in a single view
- Hovering over a queue node shows Publish rate, Receive rate, and Process rate — the same
dt.service.messaging.*metrics described in this topic, aggregated for that queue - Health alert colors and alert counts appear on service nodes, not on queue nodes; queue nodes don't carry a health state of their own — check the producing or consuming service nodes to see active alerts, including any custom alerts you configure on the messaging metrics
Because Dynatrace derives topology from span attributes rather than from a dedicated integration, it supports both OneAgent and OpenTelemetry instrumentation, including environments that mix the two. No configuration beyond your existing instrumentation is required.
Dynatrace plans deeper analysis of cloud-managed queues, including resolving an AWS SQS queue to its cloud resource and inspecting backlog size, dead-letter statistics, and consumer health. In the meantime, use 
Clouds for provider-specific queue detail, or the DQL queries in this topic for custom analysis.
Alerting
You alert on message processing by creating custom alerts on the dt.service.messaging.* metrics in 
Anomaly Detection. Because the metrics are stored in Grail, an alert can evaluate any DQL query, including expressions that combine several metrics.
Alert on message processing failures
-
Go to
Anomaly Detection. -
Select Custom alert.
-
Enter a DQL query that returns the failure rate as a percentage.
timeseries {processed = sum(dt.service.messaging.process.count),failed = sum(dt.service.messaging.process.failure_count)},by: { dt.smartscape.service, messaging.destination.name },nonempty: true, union: true| fieldsAdd failure_rate = (failed[] / processed[]) * 100 -
Set a static threshold. For example, alert when
failure_ratestays above5for more than five minutes. -
Set the event severity and save the alert.
Because the query groups by dt.smartscape.service and messaging.destination.name, Dynatrace monitors every distinct service and queue combination separately and carries those dimension values on the resulting event, so you can tell which queue and which service triggered it.
messaging.destination.name is high-cardinality if your applications create temporary queues with generated names, and each distinct value produces a separately monitored dimension. Either normalize queue names first, as described in Reduce cardinality of temporary queue names, or use the alert identity fields option described in Anomaly detection configuration to keep the queue name as event context without monitoring each value.
Alert on processing lag
A consumer that falls behind shows a widening gap between its receive rate and its process rate. To detect this condition:
-
Go to
Anomaly Detection. -
Select Custom alert.
-
Enter a DQL query that returns the processing lag per service and queue.
timeseries {received = sum(dt.service.messaging.receive.count),processed = sum(dt.service.messaging.process.count)},by: { dt.smartscape.service, messaging.destination.name },nonempty: true, union: true| fieldsAdd lag = received[] - processed[] -
Set a static threshold appropriate for your throughput and SLA. Alert when
lagstays above that value for more than a few minutes. -
Set the event severity and save the alert.
For more information on configuring custom alerts, see Configure a simple custom alert.
Query examples
Monitor service messaging throughput:
timeseries throughput = sum(dt.service.messaging.process.count),by: {dt.smartscape.service}| fieldsAdd `Service` = getNodeField(dt.smartscape.service, "name"), dt.smartscape.service| summarize throughput = sum(throughput[]),by: { timeframe, interval, `Service`, dt.smartscape.service }
Calculate service messaging failure rate:
timeseries { throughput = sum(dt.service.messaging.process.count),failure_count = sum(dt.service.messaging.process.failure_count) },by: {dt.smartscape.service}, nonempty:true, union:true| fieldsAdd `Service` = getNodeField(dt.smartscape.service, "name"), dt.smartscape.service| summarize failure_rate = sum((failure_count[] / throughput[]) * 100),by: { timeframe, interval, `Service`, dt.smartscape.service }
FAQ
Do I need to reconfigure OneAgent to collect these metrics?
No. If OneAgent monitors your services, or if you've instrumented them with OpenTelemetry, Dynatrace collects the dt.service.messaging.* metrics automatically whenever those services interact with a supported messaging system. Go to Services > Explorer > Messaging to confirm that data is arriving.
Which messaging systems are covered?
Metrics are collected for any messaging system reported through the messaging.* semantic conventions, which includes Kafka, RabbitMQ, ActiveMQ, AWS SQS, AWS SNS, Azure Service Bus, and Google Pub/Sub. The messaging.system dimension tells you which system a given data point came from.
Can I still see a visual producer-to-consumer flow?
Yes. Service Map renders queues as their own node and draws messaging edges between producing and consuming services with the addition of live publish, receive, and process rates on each queue node.
Two further options complement the map:
- Use the DQL query in List services that produce or consume messages when you need a tabular breakdown, or want to pin per-queue producer and consumer counts to a dashboard.
- Use Distributed Tracing to follow an individual message through the producing and consuming services, which remains the most direct way to inspect a single asynchronous flow.
Services