If your ingested logs don’t look as expected, you can check if a particular log record contains warnings regarding issues that occurred for that log in the log ingest and processing pipeline. Look for a dt.ingest.warnings attribute in Notebooks. It lists warnings about issues that affected a particular log record.
Many of the warnings are likely related to Log ingestion limits.
To view what warnings occurred, their counts, and the source, run the following DQL query:
fetch logs| filter isNotNull(dt.ingest.warnings)| summarize count(), by:{dt.ingest.warnings, dt.openpipeline.source, log.source}| limit 100
You can also find logs with a specific warning by using the matchesValue function.
fetch logs| filter matchesValue(dt.ingest.warnings, "common_attr_corrected")| limit 100
Note that dt.ingest.warnings is an array type-—-a filter like dt.ingest.warnings == "common_attr_corrected" won't work.
Examples of possible warnings:
| Warning | Description |
|---|---|
content_trimmed | The content was trimmed after being received bythe API because it exceeded the event content maximum byte size limit. |
content_trimmed_pipe | The content was trimmed after processing rules were applied because it exceeded the event content maximum byte size limit. |
attr_count_trimmed | The number of attributes was trimmed after being received by the API because it exceeded the maximum number of attributes in a single event. |
attr_count_trimmed_pipe | The number of attributes was trimmed after processing rules were applied because it exceeded the maximum number of attributes in a single event. |
attr_key_trimmed | At least one attribute key was trimmed because it exceeded the key maximum byte size limit. |
attr_val_count_trimmed | At least one multi-value attribute had the number of values trimmed, after being received by the API, because it exceeded the maximum number of attributes in a single event. |
attr_val_count_trimmed_pipe | After applying processing rules, at least one multi-value attribute had its value number trimmed because it exceeded the maximum number of attributes. |
attr_val_size_trimmed | At least one attribute value size was trimmed after being received by the API because it exceeded the value maximum byte size limit. |
attr_val_size_trimmed_pipe | At least one attribute value size was trimmed after processing rules were applied because it exceeded the value maximum byte size limit. |
timestamp_corrected | The timestamp was too far in the future and was corrected to the current time. |
common_attr_corrected | At least one of the following attributes was corrected: |
processing_batch_timeout | Batch timeout occurred while executing log processing rules. |
processing_transformer_timeout | Execution timeout occurred in one of the processing transformers while executing log processing rules. |
processing_transformer_error | Execution error occurred in one of the processing transformers while executing log processing rules. |
processing_transformer_throttled | Execution throttled in one of the processing transformers while executing log processing rules. |
processing_output_record_conversion_error | Output conversion error occurred for some records while executing log processing rules. |
processing_prepare_input_error | “Prepare input error” occurred in one of the enabled log processing rules. |