An example article for commands.
Loads data from the specified resource.
fetch dataObject [, from:<from>] [, to:<to>] [, timeframe:<timeframe>] [, samplingRatio:<samplingRatio>] [, scanLimitGBytes:<scanLimitGBytes>]
TBD. Details about the syntax description here. (Included as snippet)
| Parameter | Type | Description | Required |
|---|---|---|---|
dataObject | data object | The data object to fetch data for. | Required |
from | timestamp, long, duration, string | The start of the timeframe (if no explicit timeframe is specified). A duration is interpreted as an offset from | Optional |
to | timestamp, long, duration, string | The end of the timeframe (if no explicit timeframe is specified). A duration is interpreted as an offset from | Optional |
timeframe | timeframe, string | The desired timeframe (if not specified, global timeframe is used). | Optional |
samplingRatio | double, long | The desired sampling ratio. | Optional |
scanLimitGBytes | long | The maximum number of gigabytes that shall be scanned during loading data. | Optional |
Here is an example of the fetch command in its simplest form.
fetch logs
All duration literals valid for the duration data type are applicable for the from and to parameters.
This example with relative time ranges uses DQL's time literals to query logs from the last 25 minutes. The duration is automatically interpreted as an offset of the now() function.
fetch logs, from:-25m
Using both the from and to parameters allows you to adjust the start and end timestamp of the query. The duration is automatically interpreted as an offset of the now() function.
fetch logs, from:-24h, to:-2h
You can also use absolute time ranges with the timeframe parameter.
fetch logs, timeframe:"2021-10-20T00:00:00Z/2021-10-28T12:00:00Z"
To improve query performance, you can apply sampling within the initial fetch pipeline stage. Sampling is available for logs, spans, user.sessions, and user.events. Sampling happens vertically across the data, resulting in the selection of a subset of records, according to the specified, optional samplingRatio parameter.
The applicable value ranges for sampling depend on the data object:
logs and spans: 1 (default, no sampling), 10, 100, 1000, 10000, and 100000.user.sessions and user.events: 1 (default, no sampling), 10, 100, 1000, and 10000.Depending on the specified value, 1/<samplingRatio> of available raw records are returned.
The selected samplingRatio is reported in the query result for each record through dt.system.sampling_ratio, which is a hidden field. To see the hidden field, you need to select it via the fields command.
Sampling is applied when data is ingested, so the same samplingRatio returns the same records across repeated queries. All the following commands work on the sampled set of input data, which yields imprecise aggregates.
Furthermore, result sets may vary greatly with different samplingRatio values. This is the nature of sampling, as a high sampling ratio is more likely to leave out low-frequency records. For example, if you had one ERROR log among millions of INFO logs, filter loglevel == "ERROR" would very likely return an empty result set for any sampled data.
In this example, an estimation of the occurrences of ERROR Logs happens across the last 7 days. As a final step, an interpolation of the countIf() aggregation is added by multiplying and overriding the aggregation result with the selected value of the sampling parameter.
fetch logs, from:now()-7d, samplingRatio:100| summarize c = countIf(loglevel == "ERROR")| fieldsAdd c = c*100
The optional scanLimitGBytes parameter controls the amount of uncompressed data to be read by the fetch stage. The default value is 500GB unless specified otherwise. If set to -1, all data available in the query time range is analyzed.