Smartscape on Grail is a Grail-native storage that records topological data, such as monitored entities and relationships.
Smartscape on Grail is currently running side by side with the classic entity storage that still contains all entities. Because of this, Smartscape on Grail storage is empty by default.
Smartscape on Grail uses the power of DQL queries to:
This gives you deeper insight into your data records and allows you to manipulate and extract necessary information for further analysis.
Smartscape on Grail is fully integrated into DQL and introduces new commands and functions that you can use to:
The data stored in traditional buckets—such as logs, events, spans, or metric data points—is ingested for a particular timestamp and never changes. In comparison, Smartscape nodes and edges are mutable and can change over time.
Smartscape nodes are identified via an ID, represented by a Grail datatype Smartscape ID. Smartscape IDs are updated regularly through the upsert events.
A Smartscape ID consists of the entity type and a 16-symbol long number. It can be represented by a following string:
ENTITY_TYPE-000000000000007BThe Smartscape type is fully compatible with its string representation, meaning that you can compare a string to Smartscape ID.
Every node has a type field that describes the entity type and determines the schema of the entity. The semantic dictionary contains the schema for each of the types. By convention, node types are always formatted in uppercase, like HOST, K8S_NAMESPACE, and AWS_EC2_INSTANCE.
Because nodes are updated regularly, they don't have a single timestamp field. Instead, nodes have two timeframe fields that represent the node's lifetime:
lifetime.start: the first time when the node was discovered.lifetime.end: the time when the node was last observed.While the lifetime.start field will remain unchanged, the lifetime.end field will be continuously updated with every incoming upsert event as long as the node is still being observed.
Based on the query timeframe, you will see only those nodes that have a lifetime overlapping with the query timeframe. For example, a node with a lifetime.end field containing yesterday's data won't be included in a query result for the last 2 hours.
Edges are relationships that connect two nodes to each other. An edge always stores an edge type (such as runs_on, calls or relates_to), and two IDs of two different nodes.
How an edge is stored on a node depends on whether the edge is static or dynamic:
Edges stored statically are included in results where the query timeframe overlaps with the node's lifetime, whereas dynamic edges are included in query results based on a timeframe when the edge was recorded.
Static edges are mostly based on configuration, such as when a disk is configured to be attached to a specific host. Dynamic edges are typically based on monitoring signals that reveal a specific relationship between nodes, such as when a service based on traces calls another service.
Data retention is fixed at 35 days. This means that nodes whose lifetime.start and lifetime.end difference is longer than 35 days will be deleted, including all static edges. Nodes with dynamic edges will also be cleaned up after 35 days.
A signal's fields that start with dt.smartscape.__type__ and contain Smartscape IDs indicate that the signal has originated within a given node. A single signal can have multiple dt.smartscape.__type__ fields.
Optionally, a signal can also have:
dt.smartscape_source.id field that points to the exact source that produced the signal.dt.smartscape_source.type field that describes the type of the entity that produced the signal.Each node has a special field called tags that contains different nested fields recognized as tags.
Currently, tags can only be set at the data source. That means that, for example:
Similar to other data stored in Grail, Smartscape nodes have a dt.security_context field that can contain multiple values.
The dt.security_context field is an optional node field and is empty by default, since the regular permission fields are fully supported and often sufficient.
Node types used by Smartscape on Grail might be different from classic entity types. This means that the entity or node ID might also be different (for example, CLOUD_APPLICATION_INSTANCE is called K8S_POD in Smartscape on Grail). To avoid confusion, Smartscape nodes include an id_classic field that contains the entity ID of the corresponding classic entity. Classic entity IDs are available for K8s entities, core entities, and services.
If there are no corresponding classic entities (for example, with Cloud platform monitoring), there are no id_classic fields on those nodes.
Feature
Classic entity store
Smartscape on Grail
Query all entities regardless of their type
Not supported
smartscapeNodes "*"
Query HOST entities
fetch dt.entity.host
smartscapeNodes HOST
Query all relationships regardless of type, source or target
Not supported
smartscapeEdges "*"
References in Signal data
dt.entity.__type__
dt.smartscape.__type__
Enrich entity name
entityName(dt.entity.__type__)
getNodeName(dt.smartscape.__type__)
Enrich any other field (for example, tags)
entityAttr(dt.entity.__type__, "tags")
getNodeField(dt.smartscape.__type__, "tags")
Get a list of entity IDs based on entity selector
classicEntitySelector("type(HOST),toRelationships.runsOn(type(SERVICE),tag("owner:Joe"))")
Not supported
Get a list of entity IDs based on entity traversal
Not supported
smartscapeNodes SERVICE| filter tags[owner] == “Joe”| traverse runs_on, HOST| fields id
Smartscape nodes can be filtered with the help of
Only Smartscape nodes can be filtered using segments, meaning that Smartscape edges can't be filtered.