Security data on Grail
Latest Dynatrace
Dynatrace Platform utilizes various data sources to store application-related data that provides the information and context for monitoring, analysis, and investigation use cases.
Data is either generated by Dynatrace or can be ingested into the platform from third-party tools.
Security data on Grail enables security data query, aggregation, visualization, and reporting on multiple levels, providing answers with different granularity and from various perspectives. It serves in multiple categories, including:
- Logs, for storing a high amount of unstructured data.
Examples: application and infrastructure logs, security tool logs.
- Events, for storing structured data.
Examples: vulnerability assessments, attack alerts, threat intelligence feeds.
- Metrics, for storing counts for easy processing and alert generation.
Examples: failure counts of applications and infrastructure, counts of authentication attempts, and more.
Security events are a type of security data used for storing various events coming from either internal or external data sources, such as vulnerability assessments, attack alerts, and threat intelligence feeds.
The Dynatrace semantic dictionary defines conventions for storing data in a normalized manner, regardless of the origin of the data. This is important for creating stable and robust applications and automations.
Security events are a special type of data representing various generated events.
In the events data store, security events are stored in a dedicated bucket (default_security_events) and come as an additional event kind (event.kind=="SECURITY_EVENT") for better access control, data separation, and data retention period control.
Schematics
Data categories
The following is a schematic representation of the different data categories that can be ingested and stored in Grail for later analysis and reporting use cases. The common fields of the schema allow dashboards, notebooks, and other apps to access and process the data uniformly.
Common fields differ between the data categories.
There are optional vendor-specific fields and Dynatrace-enriched fields.
Logical sections of security events schema
The following shows the logical sections of the security events schema.
Security event categorization
The following shows different categories of security events, with a focus on vulnerability management.
Security event groups
All security events can be categorized into two basic groups (event.group_label): change events and state reports.
Change events
Change events (CHANGE_EVENT) are changes that occur to vulnerabilities or their affected entities.
In the example below, note the event.status_transition and change_list fields containing the context of the change.
State reports
State reports (STATE_REPORT) represent the full historical state (of a vulnerability, for example) and are reported periodically over time.
OPEN(muted and not muted) vulnerabilities are reported every 15 minutes.RESOLVEDvulnerabilities are reported only once (when open vulnerabilities get resolved). To analyze resolved vulnerabilities, filter for the desired time range.
In the example below, note the environmental context and related fields, including the information about the directly affected entities and the wider impact on the related entities of the environment.
Security event types
The vulnerability management category includes the following event types (event.type):
VULNERABILITY_STATE_REPORT_EVENT: Historical vulnerability states reported periodically.VULNERABILITY_COVERAGE_REPORT_EVENT: Historical coverage events reported periodically.VULNERABILITY_STATUS_CHANGE_EVENT: Vulnerability status changes reported on change. These include resolution and mute statuses.VULNERABILITY_ASSESSMENT_CHANGE_EVENT: Vulnerability assessment changes reported on change. These include the Davis Security Score and Davis assessments.
Security event levels
Vulnerability-related events can be reported on two different levels (event.level):
VULNERABILITY: The vulnerability on the global level, including, for example, general information, global statuses, and changes. The unique identifier isvulnerability.idorvulnerability.display_id.ENTITY: The vulnerable entity with vulnerability-related information scoped to the entity. The unique identifier is a tuple of (affected_entity.id,vulnerability.id).
Limitations
Open with
- In Third-Party Vulnerabilities, on the Third-party vulnerabilities list page, Open with… is currently unavailable if you filter for recommended fixes from Davis Security Advisor.
- Using Open with… to explore security data from scratch doesn't include the aggregation to return the same amount of data as on the source Third-party vulnerabilities list page. To aggregate data, manually run the
summarizeDQL command.
DQL queries for a specific entity
When you run DQL queries for a specific entity, the vulnerable functions in use by all entities are reported, instead of those in use only by the specific entity.
Entity change events
- The
VULNERABILITY_ASSESSMENT_CHANGE_EVENTevent type isn't currently available. - Filtering by closed events (
event.status_transition == "CLOSE") doesn't return information about the vulnerable component of an affected entity if the resolved entity is the last affected entity of a vulnerability.
Davis Security Score
The Davis Security Score (DSS) calculation currently differs in Grail-powered apps (such as Dashboards, Notebooks, Workflows) from third-party vulnerabilities pages:
On third-party vulnerabilities pages, DSS is assessed based on the remediable entities within the selected scope.
In Grail-powered apps, DSS is assessed based on the DSS of the remediable entities within the selected scope.
Thus, the DSS (score and risk level) for vulnerabilities on Grail-powered apps can be lower than on the vulnerabilities pages.