This page documents the DPL matchers for parsing time and date values. The fixed-format matchers—ISO8601, HTTPDATE, and JSONTIMESTAMP—each parse a specific timestamp format, shown below. TIMESTAMP (and its alias TIME) parses arbitrary formats using a configurable conversion pattern.
Matches an ISO 8601 timestamp in the form yyyy-MM-ddTHH:mm:ssZ, where the zone is Z or a numeric UTC offset (for example +01:00). For fractional-second timestamps, see JSONTIMESTAMP.
Use the following expression to parse ISO 8601 timestamps:
ISO8601:result
For example:
input (string) | result (timestamp) |
|---|---|
|
|
data record(input = "2019-01-01T13:23:45Z")| parse input, "ISO8601:result"
The data type of the extracted value is timestamp.
Matches a timestamp in the common HTTP log format dd/MMM/yyyy:HH:mm:ss Z (for example 26/Dec/2018:02:59:40 +0100).
Use the following expression to parse HTTP date timestamps:
HTTPDATE:result
For example:
input (string) | result (timestamp) |
|---|---|
|
|
data record(input = "26/Dec/2018:02:59:40 +0100")| parse input, "HTTPDATE:result"
The data type of the extracted value is timestamp.
Matches a timestamp with fractional-second precision in the form yyyy-MM-ddTHH:mm:ss.SSSZ, where the zone may be Z, a numeric offset, or a named timezone abbreviation (for example PST).
Use the following expression to parse JSON timestamps:
JSONTIMESTAMP:result
For example:
input (string) | result (timestamp) |
|---|---|
|
|
data record(input = "2019-01-01T01:01:01.123PST")| parse input, "JSONTIMESTAMP:result"
The data type of the extracted value is timestamp.
Allows parsing time and date fields in any format with millisecond precision, using a configurable conversion pattern.
Use the following expression to parse time and date values, where the conversion pattern is specified as a parameter:
TIMESTAMP('<format>'):result
For example, parsing various time formats with their corresponding conversion patterns:
| Description | input (string) | pattern | result (timestamp) |
|---|---|---|---|
Variable-length fields, timezone via parameter |
|
|
|
Fixed-length fields, no timezone |
|
|
|
Trailing content after the timestamp |
|
|
|
Missing seconds field |
|
|
|
Numeric timezone offset |
|
|
|
Day and month names, milliseconds, named timezone |
|
|
|
Full month name, literal text in pattern |
|
|
|
Epoch seconds |
|
|
|
Epoch milliseconds |
|
|
|
Epoch seconds with millisecond fraction |
|
|
|
Epoch seconds with microsecond fraction |
|
|
|
data record(input = "2019 1 23 1:35:47", pattern = "TIMESTAMP('yyyy M d H:m:s', tz='PST'):result")| parse input, "TIMESTAMP('yyyy M d H:m:s', tz='PST'):result"| append [data record(input = "2019-01-23 01:35:47", pattern = "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result"]| append [data record(input = "2019-01-23 01:35:47 [INFO] service started", pattern = "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result"]| append [data record(input = "2019-01-23 01:35", pattern = "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss'):result"]| append [data record(input = "2019 1 23 1:35:47 +0200", pattern = "TIMESTAMP('yyyy M d H:m:s Z'):result")| parse input, "TIMESTAMP('yyyy M d H:m:s Z'):result"]| append [data record(input = "Wed, Jan 1 2019 1:35:47.236 CET", pattern = "TIMESTAMP('EEE, MMM d yyyy H:m:s.SSS Z'):result")| parse input, "TIMESTAMP('EEE, MMM d yyyy H:m:s.SSS Z'):result"]| append [data record(input = "January 16th 2020, 23:56:10.933", pattern = """TIMESTAMP("MMMM d'th' yyyy, HH:mm:ss.S", locale='en'):result""")| parse input, """TIMESTAMP("MMMM d'th' yyyy, HH:mm:ss.S", locale='en'):result"""]| append [data record(input = "1576590440", pattern = "TIMESTAMP('s'):result")| parse input, "TIMESTAMP('s'):result"]| append [data record(input = "1576590440679", pattern = "TIMESTAMP('S'):result")| parse input, "TIMESTAMP('S'):result"]| append [data record(input = "1576590440.679", pattern = "TIMESTAMP('s.S'):result")| parse input, "TIMESTAMP('s.S'):result"]| append [data record(input = "1576590440.678599", pattern = "TIMESTAMP('s.SSSSSS'):result")| parse input, "TIMESTAMP('s.SSSSSS'):result"]
Specify the conversion pattern as the first argument, with optional parameters after it: TIMESTAMP('<format>', param=value):result.
| parameter | type | Description |
|---|---|---|
|
| Conversion pattern in single or double quotes. Default: |
|
| Timezone name per IANA Time Zone Database in single quotes. Default: user properties. |
|
| IETF BCP 47 language tag in single or double quotes (see the list for more information). Allows parsing locale-specific month and day names. Default: English. |
|
| Character set name in single or double quotes (for example |
The data type of the extracted value is timestamp.
Parsing date and time mean correctly assigning value to a timestamp - information describing a point in time. Log processing keeps timestamps similarly to Unix time (or epoch time) values - defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970.
Time value is always associated with geographical location, expressed usually as timezone. Hence at parsing the conversion from original time zone to UTC must happen (or otherwise the resulting time will have incorrect value when converted to UTC).
When timezone is present in the time field then TIMESTAMP can use it in conversion. In case it is not present you can specify timezone manually.
| Letter | Date or time component | Presentation | Example |
|---|---|---|---|
G | Era marker | Text | case insensitive AD or BC |
y | Year | Year | 2012; 96; 0015 |
Y | Week year | Year | 2009 |
M | Month in year | Month | July; Jul; 07, 7 |
w | Week in year | Numeric | 27 |
W | Week in month | Numeric | 2 |
D | Day in year | Numeric | 189 |
d | Day in month | Numeric | 10 |
F | Day of week in month | Numeric | 3 |
E | Day name in week | Text | Tue; Tuesday |
u | Unnecessary numeric | Unnecessary | 1 |
a | am/pm marker | Text | case insensitive am or pm |
H | Hour in day (0–23) | Numeric | 0 |
k | Hour in day (1–24) | Numeric | 24 |
K | Hour in AM/PM (0–11) | Numeric | 3 |
h | Hour in AM/PM (1–12) | Numeric | 1 |
m | Minute in hour | Numeric | 30 |
s | Second in minute | Numeric | 51 |
S | Milliseconds | Milliseconds | 2019-01-01 00:00:00.957 |
f | Fractional second | Fractional_second | 2019-01-01 00:00:00.250338976 |
z, Z | Time zone | Timezone | GMT+02:00; EET |
Time parsing is backed by the Java Calendar class. Depending on user Locale settings the Calendar may be Gregorian or locale-specific. Time and Date pattern behavior may be specific to the Calendar instance.
Pattern letters are usually repeated, as their number determines the exact presentation:
If the number of pattern letters is 4 or more, the full name of a field is expected by the parser. Otherwise, the abbreviated name is expected. For instance pattern "EE" expects the abbreviated name of the day in a week, such as "Tue".
Digits 0 - 9, leading zeroes and spaces are allowed. Depending on the number of letters in pattern specification, the behavior of parser is as follows:
Numeric data is allowed only. If the calendar is Gregorian then:
y - matches variable-length years, relative to 20'th century. When the year value is less than 32 then the date is adjusted to 21'st century, otherwise to 20'th century.yy - matches two-digit years, relative to 20'th century. When the year value is less than 32 then the date is adjusted to 21'st century, otherwise to 20'th century.yyy - matches variable-length years. The year is interpreted literally regardless of the number of digits. Therefore using the pattern MM-dd-yyy, a date "01-11-12" parses to Jan 11'th, 12 AD.yyyy - matches four-digit years. The year is interpreted literally.If the calendar is not Gregorian and the number of pattern letters is 4 or more, a calendar specific long form is used. Otherwise, calendar specific short form is used.
Patterns with 2 and 4 parsing letters (yy and yyyy respectively) are treated as fixed-length parsers. Hence pattern yy will parse successfully only 2 digit long years and fail for any other length.
Patterns with any other length are treated as variable length, which accepts any length of years. For instance pattern y parses successfully both "2" and "1256". Hence variable-length time units placed consecutively without non-numeric separators in-between, are impossible to parse correctly.
If the number of pattern letters is 3 or more, the month is interpreted as text, otherwise as numeric:
MMM-dd-yyyy parses "Jan-11-2012" to Jan 11'th, 2012.MMMM-dd-yyyy parses "January-11-2012" to Jan 11'th, 2012.Intended for skipping numeric parts of time and date, which do not contribute to timestamp computation. For example the number of the day in a week. These parts of the timestamp will be parsed as follows, but are ignored in the computation of timestamp value.
The number of milliseconds. Accepts numeric values up to 9 digits. The values exceeding 999 are divided by 10, 100, 1000 or 1000000 respectively to the number of digits. The remainder of the division is used as a fractional part representing milliseconds, and the quotient is added to the main timestamp.
The single letter 'S' matches variable-length value up to 9 digits. The pattern with up to 9 letters of 'S' matches values up to the respective number of digits.
Parsing time and date using the following pattern:
TIMESTAMP('yyyy-MM-dd HH:mm:ss.S', tz='UTC'):result
Millisecond values exceeding 999 overflow into the next time unit (seconds, minutes, hours, or days):
input (string) | result (timestamp) |
|---|---|
|
|
|
|
|
|
|
|
|
|
data record(input = "2019-01-01 00:00:00.999"),record(input = "2019-01-01 00:00:00.1000"),record(input = "2019-01-01 00:00:00.60000"),record(input = "2019-01-01 00:00:00.3600000"),record(input = "2019-01-01 00:00:00.86400000")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss.S', tz='UTC'):result"
The fraction of a second. Single 'f' letter matches numeric values up to 9 digits.
When used with TIMESTAMP then only up to 3 most significant digits from the value are used.
Parsing following date-time string using the following pattern:
TIMESTAMP('yyyy-MM-dd HH:mm:ss.f', tz='UTC'):result
Only the 3 most significant digits of the fractional part are used in the result:
input (string) | result (timestamp) |
|---|---|
|
|
|
|
|
|
|
|
data record(input = "2019-01-01 00:00:00.999"),record(input = "2019-01-01 00:00:00.1222"),record(input = "2019-01-01 00:00:00.3335"),record(input = "2019-01-01 00:00:00.44456789")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss.f', tz='UTC'):result"
Parses time zone expressed as timezone full name or abbreviation in English (see https://www.timeanddate.com/time/zones/)
Parsing the following date-time string to UTC timezone using the following pattern:
TIMESTAMP('yyyy-MM-dd HH:mm:ss', timezone='UTC'):result
The input string is parsed and converted to UTC:
input (string) | result (timestamp) |
|---|---|
|
|
data record(input = "2019-01-05 13:14:25")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss', timezone='UTC'):result"
Given the following input:
2019-01-23 01:35:47
Use the following pattern to parse a timestamp with a custom format and explicit timezone:
TIMESTAMP('yyyy-MM-dd HH:mm:ss', timezone='UTC'):result
result (timestamp) |
|---|
|
data record(input = "2019-01-23 01:35:47")| parse input, "TIMESTAMP('yyyy-MM-dd HH:mm:ss', timezone='UTC'):result"
Given the following input with German day and month names (in German, day abbreviations include a trailing period, for example Do. for Thursday):
Do., 24 Mai 2018 14:30:34 CET
Use the following pattern to extract the timestamp using the German locale:
TIMESTAMP('E, d MMM yyyy HH:mm:ss Z', locale='de'):result
result (timestamp) |
|---|
|
data record(input = "Do., 24 Mai 2018 14:30:34 CET")| parse input, "TIMESTAMP('E, d MMM yyyy HH:mm:ss Z', locale='de'):result"
Parses a Unix timestamp—the number of seconds or milliseconds elapsed since 1970-01-01 00:00:00 UTC—using single-character format specifiers with TIMESTAMP.
Use the following format specifiers depending on your epoch format:
| specifier | matches |
|---|---|
| Integer seconds since epoch ( |
| Integer milliseconds since epoch ( |
| Decimal seconds with millisecond fraction ( |
| Decimal seconds with microsecond fraction ( |
For example, parsing Unix epoch values using each specifier:
input (string) | pattern | result (timestamp) |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
data record(input = "1576590440", pattern = "TIMESTAMP('s'):result")| parse input, "TIMESTAMP('s'):result"| append [data record(input = "1576590440679", pattern = "TIMESTAMP('S'):result")| parse input, "TIMESTAMP('S'):result"]| append [data record(input = "1576590440.679", pattern = "TIMESTAMP('s.S'):result")| parse input, "TIMESTAMP('s.S'):result"]| append [data record(input = "1576590440.678599", pattern = "TIMESTAMP('s.SSSSSS'):result")| parse input, "TIMESTAMP('s.SSSSSS'):result"]