An example article for commands.
Loads data from the specified resource.
fetch dataObject [, from:<from>] [, to:<to>] [, timeframe:<timeframe>] [, samplingRatio:<samplingRatio>] [, scanLimitGBytes:<scanLimitGBytes>]
TBD. Details about the syntax description here. (Included as snippet)
Parameter
Type
Description
Required
dataObject
data object
The data object to fetch data for.
required
from
timestamp, long, duration, string
The start of the timeframe (if no explicit timeframe is specified). A duration is interpreted as an offset from now()
.
optional
to
timestamp, long, duration, string
The end of the timeframe (if no explicit timeframe is specified). A duration is interpreted as an offset from now()
.
optional
timeframe
timeframe, string
The desired timeframe (if not specified, global timeframe is used).
optional
samplingRatio
double, long
The desired sampling ratio.
optional
scanLimitGBytes
long
The maximum number of gigabytes that shall be scanned during loading data.
optional
Here is an example of the fetch
command in its simplest form.
fetch logs
All duration literals valid for the duration
data type are applicable for the from
and to
parameters.
This example with relative time ranges uses DQL's time literals to query logs from the last 25 minutes. The duration is automatically interpreted as an offset of the now()
function.
fetch logs, from:-25m
Using both the from
and to
parameters allows you to adjust the start and end timestamp of the query. The duration is automatically interpreted as an offset of the now()
function.
fetch logs, from:-24h, to:-2h
You can also use absolute time ranges with the timeframe
parameter.
fetch logs, timeframe:"2021-10-20T00:00:00Z/2021-10-28T12:00:00Z"
Currently, to improve query performance, sampling is applicable for Log data within the initial fetch
pipeline stage. Sampling happens vertically across the data, resulting in the selection of a subset of Log records, according to the specified, optional samplingRatio
parameter.
The applicable value ranges for sampling are:
Depending on the specified value, 1/<samplingRatio>
of available raw Log records are returned.
The selected samplingRatio
is reported in the query result for each record through dt.system.sampling_ratio
, which is a hidden field. To see the hidden field, you need to select it via the fields
command.
Sampling is non-deterministic, and will return a different result set with each query run. Also, all the following commands will work based on the sampled set of input data, yielding unprecise aggregates.
Furthermore, result sets may vary greatly with different samplingRatio
values. This is the nature of sampling, as a high sampling ratio is more likely to leave out low-frequency logs. For example, if you had one ERROR
log among millions of INFO
logs, filter loglevel == "ERROR"
would very likely return an empty result set for any sampled data.
In this example, an estimation of the occurrences of ERROR Logs happens across the last 7 days. As a final step, an interpolation of the countIf()
aggregation is added by multiplying and overriding the aggregation result with the selected value of the sampling parameter.
fetch logs, from:now()-7d, samplingRatio:100| summarize c = countIf(loglevel == "ERROR")| fieldsAdd c = c*100
The optional scanLimitGBytes
parameter controls the amount of uncompressed data to be read by the fetch
stage. The default value is 500GB
unless specified otherwise. If set to -1
, all data available in the query time range is analyzed.