Configuration

Streemlined is configured through application.yml. This page covers all available configuration options.

Configuration File Precedence

Streemlined loads configuration from these locations (in this order):

Configuration files loaded in order from the system property micronaut.config.files or the environment variable MICRONAUT_CONFIG_FILES.
Environment Variables
Java System Properties

Stick to one configuration source to avoid precedence confusion.

# With Java System Property
java -jar streemlined.jar -Dmicronaut.config.files=/path/to/application.yml

# With Environment Variable
export MICRONAUT_CONFIG_FILES=/path/to/application.yml
java -jar streemlined.jar

services:
  streemlined:
    image: streemlined:latest
    ports:
      - "8080:8080"
    environment:
      LICENSE_TOKEN: <your-license-token>
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS: broker1:9092,broker2:9092
      DATABASES_DEFAULT_CONNECTION_URL: jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
      DATABASES_DEFAULT_CONNECTION_USER: avnadmin
      DATABASES_DEFAULT_CONNECTION_PASSWORD: password
      

services:
  streemlined:
    image: streemlined:latest
    ports:
      - "8080:8080"
    environment:
      MICRONAUT_CONFIG_FILES: /config/application.yml
    volumes:
      - ./application.yml:/config/application.yml

Environment Variables

You can override any configuration using environment variables. Variable naming convention:

Replace . with _
Replace - with _
Use uppercase

services:
  streemlined:
    image: streemlined:latest
    ports:
      - "8080:8080"
    environment:
      LICENSE_TOKEN: <your-license-token>
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS: broker1:9092,broker2:9092
      DATABASES_DEFAULT_CONNECTION_URL: jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
      DATABASES_DEFAULT_CONNECTION_USER: avnadmin
      DATABASES_DEFAULT_CONNECTION_PASSWORD: password
      PLUGINS_PATH: /libs
    volumes:
      - ./libs:/libs

export LICENSE_TOKEN=<your-license-token>
export CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker1:9092,broker2:9092
export DATABASES_DEFAULT_CONNECTION_URL=jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
export DATABASES_DEFAULT_CONNECTION_USER=avnadmin
export DATABASES_DEFAULT_CONNECTION_PASSWORD=password
export PLUGINS_PATH=/libs

java -jar streemlined.jar

License

A license token is required to run Streemlined. Configure it under the license key:

license:
  token: <your-license-token>

LICENSE_TOKEN=<your-license-token>

Kafka Clusters

Streemlined supports connecting to multiple Kafka clusters simultaneously. Each cluster is configured as a named entry under the clusters key, and is referenced by name in your pipeline nodes.

clusters:
  my-cluster:
    bootstrap.servers: localhost:9092
    
    # Authentication (optional)
    security.protocol: SASL_PLAINTEXT
    sasl.mechanism: PLAIN
    sasl.jaas.config: >
      org.apache.kafka.common.security.plain.PlainLoginModule required
      username="admin"
      password="secret";

    # Schema Registry (optional)
    registry:
      schema.registry.url: http://localhost:8081
      basic.auth.credentials.source: USER_INFO
      basic.auth.user.info: username:password

CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=localhost:9092

# Authentication (optional)
CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_PLAINTEXT
CLUSTERS_MY-CLUSTER_SASL_MECHANISM=PLAIN
CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="secret";'

# Schema Registry (optional)
CLUSTERS_MY-CLUSTER_REGISTRY_SCHEMA_REGISTRY_URL=http://localhost:8081
CLUSTERS_MY-CLUSTER_REGISTRY_BASIC_AUTH_CREDENTIALS_SOURCE=USER_INFO
CLUSTERS_MY-CLUSTER_REGISTRY_BASIC_AUTH_USER_INFO=username:password

Multiple Clusters

You can configure as many clusters as you need. Each Kafka Consumer or Producer node in your pipeline references a cluster by name.

application.yml

clusters:
  us-east:
    bootstrap.servers: broker-us.example.com:9092
    registry:
      schema.registry.url: http://schema-registry-us.example.com:8081

  eu-west:
    bootstrap.servers: broker-eu.example.com:9092
    security.protocol: SASL_SSL
    sasl.mechanism: SCRAM-SHA-256
    sasl.jaas.config: >
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="admin"
      password="secret";
    registry:
      schema.registry.url: https://schema-registry-eu.example.com:8081
      basic.auth.credentials.source: USER_INFO
      basic.auth.user.info: admin:secret

Security Configuration

No Auth
SASL/PLAIN
SASL/SCRAM
SSL

clusters:
  my-cluster:
    bootstrap.servers: localhost:9092

CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=localhost:9092

clusters:
  my-cluster:
    bootstrap.servers: broker:9092
    security.protocol: SASL_PLAINTEXT
    sasl.mechanism: PLAIN
    sasl.jaas.config: >
      org.apache.kafka.common.security.plain.PlainLoginModule required
      username="user"
      password="pass";

CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9092
CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_PLAINTEXT
CLUSTERS_MY-CLUSTER_SASL_MECHANISM=PLAIN
CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username="user" password="pass";'

clusters:
  my-cluster:
    bootstrap.servers: broker:9092
    security.protocol: SASL_SSL
    sasl.mechanism: SCRAM-SHA-256
    sasl.jaas.config: >
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="user"
      password="pass";

CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9092
CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_SSL
CLUSTERS_MY-CLUSTER_SASL_MECHANISM=SCRAM-SHA-256
CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass";'

clusters:
  my-cluster:
    bootstrap.servers: broker:9093
    security.protocol: SSL
    ssl.truststore.location: /path/to/truststore.jks
    ssl.truststore.password: truststore-pass
    ssl.keystore.location: /path/to/keystore.jks
    ssl.keystore.password: keystore-pass

CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9093
CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SSL
CLUSTERS_MY-CLUSTER_SSL_TRUSTSTORE_LOCATION=/path/to/truststore.jks
CLUSTERS_MY-CLUSTER_SSL_TRUSTSTORE_PASSWORD=truststore-pass
CLUSTERS_MY-CLUSTER_SSL_KEYSTORE_LOCATION=/path/to/keystore.jks
CLUSTERS_MY-CLUSTER_SSL_KEYSTORE_PASSWORD=keystore-pass

Schema Registry

Each cluster can optionally have its own Schema Registry for Avro or Protobuf schemas, configured under the registry key within the cluster:

application.yml

clusters:
  my-cluster:
    bootstrap.servers: broker:9092
    registry:
      schema.registry.url: http://localhost:8081
      basic.auth.credentials.source: USER_INFO
      basic.auth.user.info: username:password

Additional Schema Registry configuration options are available, such as SSL settings and schema caching. See the Confluent Schema Registry client configuration documentation for a complete list of properties.

Database Connections

Configure named database connections under the databases key:

databases:
  default:
    connection.url: jdbc:postgresql://localhost:5432/mydb
    connection.user: postgres
    connection.password: secret
  
  analytics:
    connection.url: jdbc:postgresql://analytics-host:5432/analytics
    connection.user: analytics_user
    connection.password: analytics_pass

DATABASES_DEFAULT_CONNECTION_URL=jdbc:postgresql://localhost:5432/mydb
DATABASES_DEFAULT_CONNECTION_USER=postgres
DATABASES_DEFAULT_CONNECTION_PASSWORD=secret

DATABASES_ANALYTICS_CONNECTION_URL=jdbc:postgresql://analytics-host:5432/analytics
DATABASES_ANALYTICS_CONNECTION_USER=analytics_user
DATABASES_ANALYTICS_CONNECTION_PASSWORD=analytics_pass

Reference these connections by name in your JDBC Sink nodes.

Supported Databases

Streemlined packages the Aiven JDBC Connector for Apache Kafka by default. Tested databases include:

PostgreSQL
MySQL / MariaDB
SQL Server
Oracle
Snowflake
SQLite

AI Assistant

The AI assistant requires a Language Model to be configured. The following providers are available:

Anthropic
Mistral AI

Need another LLM?
Most Language Models can be supported with a simple update of Streemlined. Full list
If you need a specific provider, please ask.

langchain4j:
  anthropic:
    api-key: <your-anthropic-api-key>
    model-name: claude-opus-4-6
#  mistral-ai:
#    api-key: <your-mistral-api-key>
#    model-name: mistral-medium-latest

LANGCHAIN4J_ANTHROPIC_API_KEY=<your-anthropic-api-key>
LANGCHAIN4J_ANTHROPIC_MODEL_NAME=claude-opus-4-6

Metrics and Monitoring

Streemlined exposes Prometheus metrics by default:

micronaut:
  metrics:
    export:
      prometheus:
        enabled: true
        step: PT1M
        descriptions: true

MICRONAUT_METRICS_EXPORT_PROMETHEUS_ENABLED=true
MICRONAUT_METRICS_EXPORT_PROMETHEUS_STEP=PT1M
MICRONAUT_METRICS_EXPORT_PROMETHEUS_DESCRIPTIONS=true

Need a different metrics reporter? Micronaut Micrometer supports many alternatives including Datadog, CloudWatch, Graphite, InfluxDB, StatsD, New Relic, Dynatrace, Wavefront, Azure Monitor, Stackdriver, and more. Contact support to request an additional registry. See the Micronaut Micrometer documentation for the full list of available registries.

Available Endpoints

Endpoint	Description
`/health`	Health check status
`/prometheus`	Prometheus metrics
`/metrics`	Micrometer endpoint for humans and debugging, not for tooling or long-term integration.

Logging

Configure logging levels via application properties:

logger:
  levels:
    io.streemlined: INFO
    org.apache.kafka: WARN

LOGGER_LEVELS_IO_STREEMLINED=INFO
LOGGER_LEVELS_ORG_APACHE_KAFKA=WARN

Log Levels

Level	Use Case
`TRACE`	Only if requested by Support
`DEBUG`	Development and troubleshooting
`INFO`	Normal operation
`WARN`	Production (recommended)
`ERROR`	Minimal logging

Get Started

Build Pipelines

Test Pipelines

Deploy Pipelines

Configuration File Precedence

Environment Variables

License

Kafka Clusters

Multiple Clusters

Security Configuration

Schema Registry

Database Connections

Supported Databases

AI Assistant

Metrics and Monitoring

Available Endpoints

Logging

Log Levels

Next Steps

Quickstart

Nodes Reference

​Configuration File Precedence

​Environment Variables

​License

​Kafka Clusters

​Multiple Clusters

​Security Configuration

​Schema Registry

​Database Connections

​Supported Databases

​AI Assistant

​Metrics and Monitoring

​Available Endpoints

​Logging

​Log Levels

​Next Steps

Quickstart

Nodes Reference

Configuration File Precedence

Environment Variables

License

Kafka Clusters

Multiple Clusters

Security Configuration

Schema Registry

Database Connections

Supported Databases

AI Assistant

Metrics and Monitoring

Available Endpoints

Logging

Log Levels

Next Steps