OpenPipeline provides out-of-the-box processors with tested log parsers for common technologies, platforms, and logging frameworks. These processors are called technology bundles and let you process logs quickly, without creating custom processors.
Technology bundles normalize diverse log formats into consistent attributes aligned with the Semantic Dictionary, automatically extract key fields, and enable immediate DQL enrichment, searching, and alerting for faster troubleshooting, reliable analytics, and improved data quality.
This page lists the available technology bundles and their processors.
You can enable technology bundles as processors in any pipeline. In OpenPipeline, some technology bundles are enabled by default in the pre-processing stage and are applied to every pipeline, so you don't need to manually configure them.
To explore the details of every processor, go to
Settings > Process and contextualize > OpenPipeline > Logs and add a new processor. This requires at least one OpenPipeline pipeline. If you don't have one yet, follow the steps to set up a processing pipeline.
The following table lists the technology bundles available in OpenPipeline.
Use OpenPipeline to check the library and validate outputs.
| Category | Technology | Processors |
|---|---|---|
AWS | Amazon API Gateway | Default, Request Id, WebSocket, WebSocket CLF, WebSocket JSON, WebSocket XML, WebSocket CSV |
AWS | Amazon Aurora | Default, DA, Loglevel |
AWS | Amazon CloudFront | Json Access, Plain Text Access, Access Additional |
AWS | Amazon EKS | Control Plane Scheduler TEXT (CloudWatch ingestion), Control Plane API Server TEXT (CloudWatch ingestion), Control Plane Controller Manager TEXT (CloudWatch ingestion), Control Plane Kube Controller Manager TEXT (CloudWatch ingestion), Control Plane Authenticator TEXT (CloudWatch ingestion) |
AWS | Amazon Elasticache | Engine Logs (CloudWatch), Slow Logs (CloudWatch) |
AWS | Amazon MSK Broker | TEXT (Firehose ingestion), TEXT (CloudWatch ingestion) |
AWS | Amazon MSK Connect | TEXT (Firehose ingestion), TEXT (CloudWatch ingestion) |
AWS | Amazon OpenSearch Service | Index Slow TEXT (CloudWatch ingestion), Application TEXT (CloudWatch ingestion), Search Slow TEXT (CloudWatch ingestion), Audit JSON (CloudWatch ingestion) |
AWS | Amazon Relational Database Service (RDS) | Default, DA, DB2 diag.log, DB2 notify.log, Loglevel, MySQL and MariaDB - Error, MySQL and MariaDB - Slowquery, Oracle Audit, Oracle Listener, PostgreSQL Default, SQL Server Agent, SQL Server Error |
AWS | Amazon Route 53 | Public DNS TEXT (CloudWatch ingestion) |
AWS | Amazon Simple Notification Service (SNS) | Default, DA, Loglevel |
AWS | Amazon Virtual Private Cloud Flow Default | Default |
AWS | AWS App Runner | Default, Loglevel |
AWS | AWS CloudTrail | Management |
AWS | AWS Common | Default |
AWS | AWS Lambda | Default, Loglevel, Text Format, Json Format, Start End Format |
AWS | AWS Transit Gateway | Default Flow |
AWS | AWS WAF | Default, Http Request, Http Request Headers Inserted, Labels, Challenge Response, Captcha Response, Loglevel |
Applications | .NET | Serilog, Log4net, Microsoft Logging Extension, Nlog, Structured Message, Exception Stack Trace |
Applications | Go | Default, Record-based Log (JSON or KVP) |
Applications | Java | Spring Boot, Slf4j, Log4j, Logback, Util Logging, Default, Generic, Exception Stack Trace |
Applications | NodeJS | Log4js Default, Morgan Tiny, Morgan Short, Morgan Dev, Morgan Common, Morgan Combined, Pino, Pino with severity level as string, Pino Pretty Format, Winston, Exception Stack Trace |
Applications | PHP | Monolog, Laminas |
Applications | Python | Default, Exception Stack Trace |
Applications | Ruby | Default, Exception Stack Trace |
Azure | Azure Application Gateway Logs | Access Log, WAF Firewall Log |
Azure | Azure Entra ID Audit Logs | MS Entra ID Audit |
Azure | Azure Function App Logs | Application Logs, Exception Logs |
Azure | Azure Kubernetes Service (AKS) Logs | Kubernetes Audit, cloud-controller-manager, klog, Health Check Failure |
Azure | Azure Load Balancer Logs | Health Event |
Azure | Azure Services | Azure Common, Azure Activity Logs, Azure Operational Logs, Azure Container Service, Azure Function App, Azure API Management, Azure MariaDB, Azure MySQL, Azure MySQL Audit Logs, Azure MySQL Slow Logs, Azure PostgreSQL, Azure MSSQL, MSSQL Message Composition, MSSQL Loglevel |
Databases | Cassandra | Logback, Dropped Messages, Keyspace Column Family |
Databases | Elasticsearch | Default, Slow Log Index, Slow Log Search |
Databases | PostgreSQL | Default, Query Logs, Cleanup Query Logs, Duration Logs, Json Logs, Json Query Logs |
Databases | Redis | Docker, Server, Loglevel |
Dev Tools | JFrog Artifactory | Generic, Request, Access, Access Audit, Access Security Audit |
Message Brokers | Apache Kafka | Slf4j, Partition |
Message Brokers | RabbitMQ | Default, Separators Format, Json |
Servers | Apache HTTP | Access Logs, Error Logs |
Servers | Apache Tomcat | Initial Log, Default |
Servers | HAProxy | HTTP, HTTPS, TCP, Error, Default |
Servers | JBoss | HTTP, Generic |
Servers | Microsoft IIS | Default, NCSA, W3C Custom Format, Duration to nanoseconds conversion |
Servers | Nginx | Access Log, Error Log |
Syslog | Syslog | Cisco Switch C9300, 5424, 3164, Generic, Additional Fields, Loglevel |
Built-in processors are a classic pipeline concept. Their OpenPipeline equivalent is the technology bundle. Their functionality is similar, but technology bundles extract different fields. See Field name changes from classic pipeline for field-level differences.
If classic pipeline is your default for log processing, you can still apply technology bundles by creating a custom OpenPipeline pipeline. See Process logs with technology bundle parsers.
The table below shows the equivalent technology bundle for each classic processor.
| Classic processor | Equivalent technology bundle |
|---|---|
[Built-in] web_server:nginx:access_log | Servers > Nginx > Nginx Access Log processor |
[Built-in] web_server:nginx:error_log | Servers > Nginx > Nginx Error Log processor |
[Built-in] db:cassandra:logback | Databases > Cassandra > Cassandra Logback processor |
[Built-in] db:cassandra:dropped_messages | Databases > Cassandra > Cassandra Dropped Messages processor |
[Built-in] db:cassandra:keyspace_column_family | Databases > Cassandra > Cassandra Keyspace Column Family processor |
[Built-in] db:elasticsearch:default_log | Databases > Elasticsearch > Elasticsearch Default processor |
[Built-in] db:elasticsearch:slow_log_index | Databases > Elasticsearch > Elasticsearch Slow Log Index processor |
[Built-in] db:elasticsearch:slow_log_search | Databases > Elasticsearch > Elasticsearch Slow Log Search |
[Built-in] load_balancer:haproxy:http | Servers > HAProxy > HAProxy HTTP processor |
[Built-in] load_balancer:haproxy:tcp | Servers > HAProxy > HAProxy TCP processor |
[Built-in] load_balancer:haproxy:error | Servers > HAProxy > HAProxy Error processor |
[Built-in] load_balancer:haproxy:default | Servers > HAProxy > HAProxy Default processor |
[Built-in] cloud:azure:common | Azure > Azure Services |
[Built-in] cloud:aws:common | AWS > AWS Common |
[Built-in] cloud:aws:apigateway | AWS > Amazon API Gateway |
[Built-in] cloud:aws:lambda | AWS > AWS Lambda |
[Built-in] cloud:aws:rds | AWS > Amazon Relational Database Service (RDS) |
[Built-in] cloud:aws:rdsaurora | AWS > Amazon Aurora |
[Built-in] cloud:aws:sns | AWS > Amazon Simple Notification Service (SNS) |
[Built-in] cloud:aws:apprunner | AWS > AWS App Runner |
[Built-in] cloud:aws:cloudtrail | AWS > AWS CloudTrail |