Try it free

DPL JSON Data

  • Latest Dynatrace
  • Reference

The DPL JSON matchers parse JSON data according to RFC 8259. Use JSON_OBJECT for objects, JSON_ARRAY for arrays, and JSON_VALUE for standalone values.

Overview

JSON_VALUE matches any JSON element, including objects and arrays. JSON_OBJECT and JSON_ARRAY match only their own JSON type and return null for anything else.

input (string)JSON_OBJECT (record)JSON_ARRAY (array)JSON_VALUE

{"status":503}

status: 503

null

status: 503 (record)

[80,443]

null

[80, 443]

[80, 443] (array)

503

null

null

503 (long)

"web-01"

null

null

web-01 (string)

true

null

null

true (boolean)

See how to use in DQL
data record(input = """{"status":503}"""),
record(input = """[80,443]"""),
record(input = """503"""),
record(input = """"web-01""""),
record(input = """true""")
| parse input, "JSON_OBJECT:jsonObject"
| parse input, "JSON_ARRAY:jsonArray"
| parse input, "JSON_VALUE:jsonValue"
| fields input, jsonObject, jsonArray, jsonValue

JSON_OBJECT, JSON

Matches JSON objects (structures of name-value pairs enclosed in curly brackets) according to RFC 8259.

Use the following expression to parse the value:

JSON_OBJECT:result

For example:

Descriptioninput (string)result (record)

Flat object

{"status":503,"cached":false}

status: 503 cached: false

Nested object

{"http":{"status":503}}

http: {status: 503}

Object with array member

{"ports":[80,443]}

ports: [80, 443]

Mixed member types

{"host":"web-01","up":true,"load":0.75,"note":null}

host: web-01 up: true load: 0.75 note: null

Multi-word member name

{"client ip":"203.0.113.42"}

client ip: 203.0.113.42

Empty object

{}

{}

JSON array

[1,2,3]

null

Whitespace

␣

null

See how to use in DQL
data record(input = """{"status":503,"cached":false}"""),
record(input = """{"http":{"status":503}}"""),
record(input = """{"ports":[80,443]}"""),
record(input = """{"host":"web-01","up":true,"load":0.75,"note":null}"""),
record(input = """{"client ip":"203.0.113.42"}"""),
record(input = """{}"""),
record(input = """[1,2,3]"""),
record(input = """ """)
| parse input, "JSON_OBJECT:result"

Syntax

JSON_OBJECT{ matcher_expression:member, ... }( parameter = value, ... ):result

Configuration

ParameterTypeDescription

charset

string

Character set name enclosed in single or double quotes (for example charset="ISO-8859-1"). Default: none.

locale

string

String specifying IETF BCP 47 language tag enclosed in single or double quotes. For more information, see the list. Default: English.

greedy

string

Name of a record field that collects every member not named in the member list. Default: none.

flat

boolean

When true, explicitly specified members are exported as separate fields instead of nested fields of one record. A top-level JSON_OBJECT with flat=true can't have an export name. Default: false.

maxlen

long

Maximum byte size of a JSON object. Allows parsing large JSON objects exceeding the default. Maximum: 512000000. Default: 128000.

strict

boolean

When false, allows parsing objects not following JSON specification. Unquoted JSON names and string values consisting of one word can be parsed. Default: true.

Returns

The data type of the extracted value is record. Each member value carries the type of its JSON value: long, double, string, boolean, array, or record.

Basic examples

Example 1: Parse selected members

To extract only certain members, list them as a comma-separated sequence of matcher-name pairs enclosed in curly braces ({}). Enclose a member name that consists of multiple words in single or double quotes, and append a colon (:) followed by a single-word export name.

Given the following input:

input (string)
{
"client ip": "203.0.113.42",
"status": 503,
"request time": "2024-11-05 13:18:57 -0600",
"cached": false,
"path": "/api/v2/checkout"
}

Use the following expression:

JSON_OBJECT{
IPADDR:'client ip':client_ip, // Parse `client ip` as an IP address, exporting it as `client_ip`
INT:status, // Parse `status` as an integer
TIMESTAMP('yyyy-MM-dd HH:mm:ss Z'):'request time':request_time // Parse `request time`, exporting it as `request_time`
}:result
result (record)

client_ip: 203.0.113.42 status: 503 request_time: 2024-11-05T19:18:57.000Z

Members that aren't listed—cached and path—are dropped.

See how to use in DQL
data record(input = """{"client ip":"203.0.113.42","status":503,"request time":"2024-11-05 13:18:57 -0600","cached":false,"path":"/api/v2/checkout"}""")
| parse input, "JSON_OBJECT{ IPADDR:'client ip':client_ip, INT:status, TIMESTAMP('yyyy-MM-dd HH:mm:ss Z'):'request time':request_time }:result"
| fields result
Example 2: Parse or drop unselected members using greedy

Use the greedy parameter to collect every member you didn't name into a nested record, and assign the export name to null to drop a member you did name.

Given the same input as in example 1:

input (string)
{
"client ip": "203.0.113.42",
"status": 503,
"request time": "2024-11-05 13:18:57 -0600",
"cached": false,
"path": "/api/v2/checkout"
}

The following expression collects cached and path under other members:

JSON_OBJECT{
IPADDR:'client ip':client_ip,
INT:status,
TIMESTAMP('yyyy-MM-dd HH:mm:ss Z'):'request time':request_time
}(greedy='other members'):result
result (record)

client_ip: 203.0.113.42 status: 503 request_time: 2024-11-05T19:18:57.000Z other members: {cached: false path: /api/v2/checkout}

See how to use in DQL
data record(input = """{"client ip":"203.0.113.42","status":503,"request time":"2024-11-05 13:18:57 -0600","cached":false,"path":"/api/v2/checkout"}""")
| parse input, "JSON_OBJECT{ IPADDR:'client ip':client_ip, INT:status, TIMESTAMP('yyyy-MM-dd HH:mm:ss Z'):'request time':request_time }(greedy='other members'):result"
| fields result

Exporting a member as null excludes it from the result. The member is still consumed, so it doesn't reach the greedy field either:

JSON_OBJECT{
IPADDR:'client ip':null, // Match `client ip`, but keep it out of the result
INT:status
}(greedy='other members'):result
result (record)

status: 503 other members: {request time: 2024-11-05 13:18:57 -0600 cached: false path: /api/v2/checkout}

See how to use in DQL
data record(input = """{"client ip":"203.0.113.42","status":503,"request time":"2024-11-05 13:18:57 -0600","cached":false,"path":"/api/v2/checkout"}""")
| parse input, "JSON_OBJECT{ IPADDR:'client ip':null, INT:status }(greedy='other members'):result"
| fields result
Example 3: Parse members as separate fields using flat

By default, the extracted members form one nested record. Set flat=true to export each member as a field of its own. A top-level JSON_OBJECT with flat=true can't have an export name, because there is no record to name.

Given the same input as in example 1:

input (string)
{
"client ip": "203.0.113.42",
"status": 503,
"request time": "2024-11-05 13:18:57 -0600",
"cached": false,
"path": "/api/v2/checkout"
}

Use the following expression:

JSON_OBJECT{ IPADDR:'client ip':client_ip, INT:status }(flat=true)
client_ip (ip)status (long)

203.0.113.42

503

See how to use in DQL
data record(input = """{"client ip":"203.0.113.42","status":503,"request time":"2024-11-05 13:18:57 -0600","cached":false,"path":"/api/v2/checkout"}""")
| parse input, "JSON_OBJECT{ IPADDR:'client ip':client_ip, INT:status }(flat=true)"
| fields client_ip, status
Example 4: Parse inputs with mandatory members

A member that's listed in the member list but missing from the input is set to null in the resulting record. Append the + quantifier to make a member mandatory: when a mandatory member is missing, the whole JSON_OBJECT match fails and the field is null.

Use the following expression, in which service and status are mandatory and message is optional:

JSON_OBJECT{ STRING+:service, INT+:status, STRING:message }:result
Descriptioninput (string)result (record)

All mandatory members present

{"service":"checkout","status":503,"message":"upstream timeout"}

service: checkout status: 503 message: upstream timeout

Mandatory member status missing

{"service":"checkout","message":"upstream timeout"}

null

See how to use in DQL
data record(input = """{"service":"checkout","status":503,"message":"upstream timeout"}"""),
record(input = """{"service":"checkout","message":"upstream timeout"}""")
| parse input, "JSON_OBJECT{ STRING+:service, INT+:status, STRING:message }:result"
| fields input, result
Example 5: Parse arrays and nested objects as members

Parse an array member whose elements share one type with the syntax conversion_type[]:member_name. Append one square-bracket pair per dimension to match a multidimensional array. Parse an object member by nesting a JSON_OBJECT inside the member list.

Given the following input:

input (string)
{
"service": "checkout",
"upstream": { "host": "web-01", "port": 8080 },
"latencies": [12, 48, 7, 350],
"retry windows": [[1, 2], [4, 8]]
}

Use the following expression:

JSON_OBJECT{
STRING:service,
JSON_OBJECT{ STRING:host, INT:port }:upstream, // Parse the nested `upstream` object
LONG[]:latencies, // Parse `latencies` as an array of longs
LONG[][]:'retry windows':retry_windows // Parse `retry windows` as a two-dimensional array
}:result
result (record)

service: checkout upstream: {host: web-01 port: 8080} latencies: [12, 48, 7, 350] retry_windows: [[1, 2], [4, 8]]

See how to use in DQL
data record(input = """{"service":"checkout","upstream":{"host":"web-01","port":8080},"latencies":[12,48,7,350],"retry windows":[[1,2],[4,8]]}""")
| parse input, "JSON_OBJECT{ STRING:service, JSON_OBJECT{ STRING:host, INT:port }:upstream, LONG[]:latencies, LONG[][]:'retry windows':retry_windows }:result"
| fields result
Example 6: Parse non-standard JSON using strict

By default, JSON_OBJECT treats members strictly according to RFC 8259. Setting strict=false relaxes validation, so unquoted JSON names and string values consisting of one word can be parsed.

Given the following input, in which the name service and the value EU are unquoted:

input (string)
{
service: "checkout",
"status" : 503,
region : EU
}

Use the following expression:

JSON_OBJECT{ STRING:service, INT:status, STRING:region }(strict=false):result
result (record)

service: checkout status: 503 region: EU

See how to use in DQL
data record(input = """{service: "checkout", "status" : 503, region : EU}""")
| parse input, "JSON_OBJECT{ STRING:service, INT:status, STRING:region }(strict=false):result"
| fields result

Practical examples

Example 1: Parse a repeated member from every array element

To read the same member from every object in a nested array, mirror the input structure with JSON_ARRAY{ JSON_OBJECT{ ... } } and use array projection ([field][][subfield]) to gather the values into a flat array.

Given the following input:

input (string)
{
"flights": [
{ "bookingNumber": "BK001", "origin": "VIE", "destination": "LHR" },
{ "bookingNumber": "BK002", "origin": "MUC", "destination": "JFK" },
{ "bookingNumber": "BK003", "origin": "ZRH", "destination": "CDG" }
]
}

Use the following expression:

JSON_OBJECT{ // Parse the top-level object
JSON_ARRAY{ // Parse the `flights` array
JSON_OBJECT{ STRING:bookingNumber } // Capture `bookingNumber` from each element
}:flights
}:result

Then apply an iterative expression on the parsed record to collect the values:

| fieldsAdd bookingNumbers = result[flights][][bookingNumber]

result[flights][][bookingNumber] is an iterative expression. The [] operator tells DQL to walk every element of the array to its left instead of picking one by index, and the field name that follows is read from each element in turn. So result[flights] selects the array, [] iterates over its records, and [bookingNumber] takes that member from each one.

Because the expression isn't wrapped in an iterative function such as iSum() or iMax(), DQL implicitly applies iCollectArray(), gathering the per-element results into a new array. This keeps the array intact—no expand is needed, and the record isn't split into one row per element.

bookingNumbers (array)

[BK001, BK002, BK003]

See how to use in DQL
data record(input = """{"flights":[{"bookingNumber":"BK001","origin":"VIE","destination":"LHR"},{"bookingNumber":"BK002","origin":"MUC","destination":"JFK"},{"bookingNumber":"BK003","origin":"ZRH","destination":"CDG"}]}""")
| parse input, "JSON_OBJECT{ JSON_ARRAY{ JSON_OBJECT{ STRING:bookingNumber } }:flights }:result"
| fieldsAdd bookingNumbers = result[flights][][bookingNumber]
| fields bookingNumbers

The same result can be reached without modeling the JSON structure, by iterating over the input with ARRAY and skipping ahead to each "bookingNumber" occurrence:

DATA ARRAY{
'"bookingNumber":' DQS:i // Skip to the next `bookingNumber` and capture its value
((DATA >> '"bookingNumber')|DATA) // Consume up to the following occurrence, or to the end
}{1,}:bookingNumbers
bookingNumbers (array)

[BK001, BK002, BK003]

See how to use in DQL
data record(input = """{"flights":[{"bookingNumber":"BK001","origin":"VIE","destination":"LHR"},{"bookingNumber":"BK002","origin":"MUC","destination":"JFK"},{"bookingNumber":"BK003","origin":"ZRH","destination":"CDG"}]}""")
| parse input, """DATA ARRAY{'"bookingNumber":' DQS:i ((DATA >> '"bookingNumber')|DATA) }{1,}:bookingNumbers"""
| fields bookingNumbers

The second expression is shorter, but it scans text instead of parsing the JSON structure, so it collects bookingNumber wherever it occurs in the input—including from members outside the flights array.

Example 2: Parse a list of JSON objects

Input data often carries many records as elements of one array. Parsing such input with JSON_ARRAY isn't practical because:

  • You want each element as a separate record, but JSON_ARRAY returns the whole array as one field of one record.
  • The number of elements is likely to exceed the capacity of JSON_ARRAY, even with the maxlen parameter raised.

Instead, treat the input as a list of JSON objects separated by commas and ignore the enclosing square brackets. Because the pattern has to match repeatedly within one input, use parseAll rather than parse, which matches only once.

Use JSON object semantic validation to define the array element object. This avoids unmatched objects near the border of chunks of input data.

Given the following input:

input (string)
[
{ "orderId": "A-1001", "total": 42.5, "items": 2 },
{ "orderId": "A-1002", "total": 18.75, "items": 1 },
{ "orderId": "A-1003", "total": 99.9, "items": 5 }
]

Use the following expression, which makes orderId mandatory and captures the remaining members under details:

'['? // Optional opening bracket of the enclosing array
JSON_OBJECT{ STRING+:orderId }(greedy='details'):order
','? // Optional separator between elements
']'? // Optional closing bracket of the enclosing array
orders (array)

[orderId: A-1001 details: {total: 42.5 items: 2}, orderId: A-1002 details: {total: 18.75 items: 1}, orderId: A-1003 details: {total: 99.9 items: 5}]

See how to use in DQL
data record(input = """[{"orderId":"A-1001","total":42.5,"items":2},{"orderId":"A-1002","total":18.75,"items":1},{"orderId":"A-1003","total":99.9,"items":5}]""")
| fieldsAdd orders = parseAll(input, "'['? JSON_OBJECT{ STRING+:orderId }(greedy='details'):order ','? ']'?")
| fields orders

JSON_ARRAY

Matches JSON arrays according to RFC 8259. Arrays can contain members of different types and can appear outside a JSON object.

Use the following expression to parse the value:

JSON_ARRAY:result

For example:

Descriptioninput (string)result (array)

Number elements

[80,443,8080]

[80, 443, 8080]

String elements

["GET","POST"]

[GET, POST]

Mixed element types

[1,null,"x",true]

[1, null, x, true]

Nested arrays

[[1,2],[3,4]]

[[1, 2], [3, 4]]

Array of objects

[{"id":1},{"id":2}]

[id: 1, id: 2]

Empty array

[]

[]

JSON object

{"a":1}

null

Whitespace

␣

null

See how to use in DQL
data record(input = """[80,443,8080]"""),
record(input = """["GET","POST"]"""),
record(input = """[1,null,"x",true]"""),
record(input = """[[1,2],[3,4]]"""),
record(input = """[{"id":1},{"id":2}]"""),
record(input = """[]"""),
record(input = """{"a":1}"""),
record(input = """ """)
| parse input, "JSON_ARRAY:result"

Syntax

JSON_ARRAY{ matcher_expression }( parameter = value, ... ):result

Configuration

ParameterTypeDescription

charset

string

Character set name enclosed in single or double quotes (for example charset="ISO-8859-1"). Default: none.

locale

string

String specifying IETF BCP 47 language tag enclosed in single or double quotes. For more information, see the list. Default: English.

strict

boolean

When false, allows parsing arrays not following JSON specification. Unquoted JSON names and string values consisting of one word can be parsed. Default: true.

maxlen

long

Maximum byte size of an array. Allows parsing large JSON arrays exceeding the default. Maximum: 512000000. Default: 128000.

When conversion of an element fails, the entire output is set to null.

Returns

The data type of the extracted value is array. Each element carries the type of its JSON value: long, double, string, boolean, array, or record.

Basic examples

Example 1: Parse elements as a single type

Numbers are often encoded as JSON strings. Without a conversion type they stay string, so numeric array functions can't use them—arraySum returns 0. Name a conversion type inside the braces to convert every element.

Given the following input:

["12.5", "48.25", "7.0"]

Use the following expression:

JSON_ARRAY{ DOUBLE }:result
result (array)arraySum(result)

[12.5, 48.25, 7]

67.75

See how to use in DQL
data record(input = """["12.5","48.25","7.0"]""")
| parse input, "JSON_ARRAY{ DOUBLE }:result"
| fieldsAdd sum = arraySum(result)
| fields result, sum
Example 2: Parse non-standard JSON using strict

By default, JSON_ARRAY treats elements strictly according to RFC 8259. Setting strict=false relaxes validation, so unquoted JSON names and string values consisting of one word can be parsed. Every element type an array can hold is still recognized.

Given the following input, in which the member name host and the values web-01 and up are unquoted:

[{host: web-01}, [1,2], up, 503, 0.75, true, null]

Use the following expression:

JSON_ARRAY(strict=false):result
patternresult (array)

JSON_ARRAY(strict=false):result

[{host: web-01}, [1, 2], up, 503, 0.75, true, null]

JSON_ARRAY:result

null

See how to use in DQL
data record(input = """[{host: web-01}, [1,2], up, 503, 0.75, true, null]""")
| parse input, "JSON_ARRAY(strict=false):result"
| fields result

Practical example

Example: Parse an embedded array from a log line

A JSON array often appears as one field inside an otherwise unstructured log line. Combine JSON_ARRAY with the literals and matchers that describe the rest of the line, and give it a conversion type so the elements arrive as numbers that array functions can work with.

Given the following input:

2024-11-05T13:18:57Z WARN checkout retry latencies=[12,48,7,350] upstream=web-01

Use the following expression, in which LD skips the surrounding text:

LD ' retry latencies=' JSON_ARRAY{ LONG }:latencies LD // Convert the array elements to longs
latencies (array)

[12, 48, 7, 350]

Because the elements are long, array functions can be applied to latencies directly—arrayMax(latencies) returns 350 and arraySize(latencies) returns 4.

See how to use in DQL
data record(input = """2024-11-05T13:18:57Z WARN checkout retry latencies=[12,48,7,350] upstream=web-01""")
| parse input, "LD ' retry latencies=' JSON_ARRAY{ LONG }:latencies LD"
| fieldsAdd slowest = arrayMax(latencies), attempts = arraySize(latencies)
| fields latencies, slowest, attempts

JSON_VALUE

Matches JSON elements—array, string, number, boolean, or null—that are not enclosed in a JSON object. This is allowed by JSON Grammar.

Use the following expression to parse the value:

JSON_VALUE:result

For example:

Descriptioninput (string)result

Integer number

503

503 (long)

Fractional number

0.75

0.75 (double)

String

"web-01"

web-01 (string)

Boolean

true

true (boolean)

Null literal

null

null

Array

[80,443]

[80, 443] (array)

Object

{"status":503}

status: 503 (record)

Whitespace

␣

null

See how to use in DQL
data record(input = """503"""),
record(input = """0.75"""),
record(input = """"web-01""""),
record(input = """true"""),
record(input = """null"""),
record(input = """[80,443]"""),
record(input = """{"status":503}"""),
record(input = """ """)
| parse input, "JSON_VALUE:result"

Syntax

JSON_VALUE{ matcher_expression }( parameter = value, ... ):result

Configuration

ParameterTypeDescription

charset

string

Character set name enclosed in single or double quotes (for example charset="ISO-8859-1"). Default: none.

locale

string

String specifying IETF BCP 47 language tag enclosed in single or double quotes. For more information, see the list. Default: English.

strict

boolean

When false, allows parsing values not following JSON specification. Unquoted JSON names and string values consisting of one word can be parsed. Default: true.

maxlen

long

Maximum byte size of a value. Allows parsing large JSON values exceeding the default. Maximum: 512000000. Default: 128000.

Returns

The data type of the extracted value follows the JSON type of the matched value: long, double, string, boolean, array, or record.

Basic example

Example: Parse values with automatic and explicit conversion

JSON_VALUE derives the type from the JSON value, so a JSON number already arrives as a number. Name a conversion type inside the braces when the JSON type isn't the type you need—for example, to turn a JSON string into an ip.

Given the following input, which holds a status code and an IP address on separate lines:

input (string)
503
"203.0.113.42"

Use the following expression:

JSON_VALUE:status EOL JSON_VALUE{ IPADDR }:client_ip
status (long)client_ip (ip)

503

203.0.113.42

See how to use in DQL
data record(input = "503\n\"203.0.113.42\"")
| parse input, "JSON_VALUE:status EOL JSON_VALUE{ IPADDR }:client_ip"
| fields status, client_ip

Practical example

Example: Parse all member names

When the JSON structure is unknown or changes between records, collect the member names instead of the values. Iterate over the members with ARRAY, capture each name with DQS, and use JSON_VALUE to skip past the value.

JSON_VALUE is what makes the loop work—it consumes a value of any JSON type. A matcher such as DQS would only skip string values and stop at the first number, boolean, array, or object.

Given the following input:

input (string)
{
"tenant_id": "tenant-01",
"component": "STREAM",
"retries": 3,
"billable": false,
"tags": ["prod", "eu"],
"upstream": { "host": "web-01" }
}

Use the following expression:

'{' ARRAY{ // Match the opening brace, then loop over each member
SPACE? // Skip optional whitespace before the name
DQS:k // Capture the quoted member name into `k`
':' JSON_VALUE // Match the separator and skip a value of any type
','? // Skip the optional trailing comma
}*:keys // Repeat for all members, collecting the names into `keys`
keys (array)

[tenant_id, component, retries, billable, tags, upstream]

See how to use in DQL
data record(input = """{"tenant_id":"tenant-01","component":"STREAM","retries":3,"billable":false,"tags":["prod","eu"],"upstream":{"host":"web-01"}}""")
| parse input, "'{' ARRAY{ SPACE? DQS:k ':' JSON_VALUE ','? }*:keys"
| fields keys
Related tags
Dynatrace Platform