> ## Documentation Index
> Fetch the complete documentation index at: https://docs.streemlined.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Configuration

> Configure Kafka connections, databases, AI assistant (Anthropic), and runtime settings

Streemlined is configured through `application.yml`. This page covers all available configuration options.

## Configuration File Precedence

Streemlined loads configuration from these locations (in this order):

1. Configuration files loaded in order from the system property `micronaut.config.files` or the environment variable `MICRONAUT_CONFIG_FILES`.
2. Environment Variables
3. Java System Properties

<Tip>
  Stick to one configuration source to avoid precedence confusion.
</Tip>

<CodeGroup>
  ```bash JAR File theme={null}
  # With Java System Property
  java -jar streemlined.jar -Dmicronaut.config.files=/path/to/application.yml

  # With Environment Variable
  export MICRONAUT_CONFIG_FILES=/path/to/application.yml
  java -jar streemlined.jar
  ```

  ```yaml docker compose (with environment variables) theme={null}
  services:
    streemlined:
      image: streemlined:latest
      ports:
        - "8080:8080"
      environment:
        LICENSE_TOKEN: <your-license-token>
        CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS: broker1:9092,broker2:9092
        DATABASES_DEFAULT_CONNECTION_URL: jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
        DATABASES_DEFAULT_CONNECTION_USER: avnadmin
        DATABASES_DEFAULT_CONNECTION_PASSWORD: password
        
  ```

  ```yaml docker compose (with volume mount) theme={null}
  services:
    streemlined:
      image: streemlined:latest
      ports:
        - "8080:8080"
      environment:
        MICRONAUT_CONFIG_FILES: /config/application.yml
      volumes:
        - ./application.yml:/config/application.yml
  ```
</CodeGroup>

## Environment Variables

You can override any configuration using environment variables.

Variable naming convention:

* Replace `.` with `_`
* Replace `-` with `_`
* Use uppercase

<CodeGroup>
  ```yaml docker compose theme={null}
  services:
    streemlined:
      image: streemlined:latest
      ports:
        - "8080:8080"
      environment:
        LICENSE_TOKEN: <your-license-token>
        CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS: broker1:9092,broker2:9092
        DATABASES_DEFAULT_CONNECTION_URL: jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
        DATABASES_DEFAULT_CONNECTION_USER: avnadmin
        DATABASES_DEFAULT_CONNECTION_PASSWORD: password
        PLUGINS_PATH: /libs
      volumes:
        - ./libs:/libs
  ```

  ```bash JAR file theme={null}
  export LICENSE_TOKEN=<your-license-token>
  export CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker1:9092,broker2:9092
  export DATABASES_DEFAULT_CONNECTION_URL=jdbc:postgresql://pg-xxyyzz-streemlined-nnnn.c.aivencloud.com:13645/defaultdb?ssl=require
  export DATABASES_DEFAULT_CONNECTION_USER=avnadmin
  export DATABASES_DEFAULT_CONNECTION_PASSWORD=password
  export PLUGINS_PATH=/libs

  java -jar streemlined.jar
  ```
</CodeGroup>

***

## License

A license token is required to run Streemlined. Configure it under the `license` key:

<CodeGroup>
  ```yaml application.yml theme={null}
  license:
    token: <your-license-token>
  ```

  ```bash Environment Variables theme={null}
  LICENSE_TOKEN=<your-license-token>
  ```
</CodeGroup>

***

## Kafka Clusters

Streemlined supports connecting to **multiple Kafka clusters** simultaneously. Each cluster is configured as a named entry under the `clusters` key, and is referenced by name in your pipeline nodes.

<CodeGroup>
  ```yaml application.yml theme={null}
  clusters:
    my-cluster:
      bootstrap.servers: localhost:9092
      
      # Authentication (optional)
      security.protocol: SASL_PLAINTEXT
      sasl.mechanism: PLAIN
      sasl.jaas.config: >
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="secret";

      # Schema Registry (optional)
      registry:
        schema.registry.url: http://localhost:8081
        basic.auth.credentials.source: USER_INFO
        basic.auth.user.info: username:password
  ```

  ```bash Environment Variables theme={null}
  CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=localhost:9092

  # Authentication (optional)
  CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_PLAINTEXT
  CLUSTERS_MY-CLUSTER_SASL_MECHANISM=PLAIN
  CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="secret";'

  # Schema Registry (optional)
  CLUSTERS_MY-CLUSTER_REGISTRY_SCHEMA_REGISTRY_URL=http://localhost:8081
  CLUSTERS_MY-CLUSTER_REGISTRY_BASIC_AUTH_CREDENTIALS_SOURCE=USER_INFO
  CLUSTERS_MY-CLUSTER_REGISTRY_BASIC_AUTH_USER_INFO=username:password
  ```
</CodeGroup>

### Multiple Clusters

You can configure as many clusters as you need. Each Kafka Consumer or Producer node in your pipeline references a cluster by name.

```yaml application.yml theme={null}
clusters:
  us-east:
    bootstrap.servers: broker-us.example.com:9092
    registry:
      schema.registry.url: http://schema-registry-us.example.com:8081

  eu-west:
    bootstrap.servers: broker-eu.example.com:9092
    security.protocol: SASL_SSL
    sasl.mechanism: SCRAM-SHA-256
    sasl.jaas.config: >
      org.apache.kafka.common.security.scram.ScramLoginModule required
      username="admin"
      password="secret";
    registry:
      schema.registry.url: https://schema-registry-eu.example.com:8081
      basic.auth.credentials.source: USER_INFO
      basic.auth.user.info: admin:secret
```

### Security Configuration

<Tabs>
  <Tab title="No Auth">
    <CodeGroup>
      ```yaml application.yml theme={null}
      clusters:
        my-cluster:
          bootstrap.servers: localhost:9092
      ```

      ```bash Environment Variables theme={null}
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=localhost:9092
      ```
    </CodeGroup>
  </Tab>

  <Tab title="SASL/PLAIN">
    <CodeGroup>
      ```yaml application.yml theme={null}
      clusters:
        my-cluster:
          bootstrap.servers: broker:9092
          security.protocol: SASL_PLAINTEXT
          sasl.mechanism: PLAIN
          sasl.jaas.config: >
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="user"
            password="pass";
      ```

      ```bash Environment Variables theme={null}
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9092
      CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_PLAINTEXT
      CLUSTERS_MY-CLUSTER_SASL_MECHANISM=PLAIN
      CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.plain.PlainLoginModule required username="user" password="pass";'
      ```
    </CodeGroup>
  </Tab>

  <Tab title="SASL/SCRAM">
    <CodeGroup>
      ```yaml application.yml theme={null}
      clusters:
        my-cluster:
          bootstrap.servers: broker:9092
          security.protocol: SASL_SSL
          sasl.mechanism: SCRAM-SHA-256
          sasl.jaas.config: >
            org.apache.kafka.common.security.scram.ScramLoginModule required
            username="user"
            password="pass";
      ```

      ```bash Environment Variables theme={null}
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9092
      CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SASL_SSL
      CLUSTERS_MY-CLUSTER_SASL_MECHANISM=SCRAM-SHA-256
      CLUSTERS_MY-CLUSTER_SASL_JAAS_CONFIG='org.apache.kafka.common.security.scram.ScramLoginModule required username="user" password="pass";'
      ```
    </CodeGroup>
  </Tab>

  <Tab title="SSL">
    <CodeGroup>
      ```yaml application.yml theme={null}
      clusters:
        my-cluster:
          bootstrap.servers: broker:9093
          security.protocol: SSL
          ssl.truststore.location: /path/to/truststore.jks
          ssl.truststore.password: truststore-pass
          ssl.keystore.location: /path/to/keystore.jks
          ssl.keystore.password: keystore-pass
      ```

      ```bash Environment Variables theme={null}
      CLUSTERS_MY-CLUSTER_BOOTSTRAP_SERVERS=broker:9093
      CLUSTERS_MY-CLUSTER_SECURITY_PROTOCOL=SSL
      CLUSTERS_MY-CLUSTER_SSL_TRUSTSTORE_LOCATION=/path/to/truststore.jks
      CLUSTERS_MY-CLUSTER_SSL_TRUSTSTORE_PASSWORD=truststore-pass
      CLUSTERS_MY-CLUSTER_SSL_KEYSTORE_LOCATION=/path/to/keystore.jks
      CLUSTERS_MY-CLUSTER_SSL_KEYSTORE_PASSWORD=keystore-pass
      ```
    </CodeGroup>
  </Tab>
</Tabs>

### Schema Registry

Each cluster can optionally have its own Schema Registry for Avro or Protobuf schemas, configured under the `registry` key within the cluster:

```yaml application.yml theme={null}
clusters:
  my-cluster:
    bootstrap.servers: broker:9092
    registry:
      schema.registry.url: http://localhost:8081
      basic.auth.credentials.source: USER_INFO
      basic.auth.user.info: username:password
```

<Tip>
  Additional Schema Registry configuration options are available, such as SSL settings and schema caching. See the [Confluent Schema Registry client configuration documentation](https://docs.confluent.io/platform/current/schema-registry/sr-client-configs.html) for a complete list of properties.
</Tip>

***

## Database Connections

Configure named database connections under the `databases` key:

<CodeGroup>
  ```yaml application.yml theme={null}
  databases:
    default:
      connection.url: jdbc:postgresql://localhost:5432/mydb
      connection.user: postgres
      connection.password: secret
    
    analytics:
      connection.url: jdbc:postgresql://analytics-host:5432/analytics
      connection.user: analytics_user
      connection.password: analytics_pass
  ```

  ```bash Environment Variables theme={null}
  DATABASES_DEFAULT_CONNECTION_URL=jdbc:postgresql://localhost:5432/mydb
  DATABASES_DEFAULT_CONNECTION_USER=postgres
  DATABASES_DEFAULT_CONNECTION_PASSWORD=secret

  DATABASES_ANALYTICS_CONNECTION_URL=jdbc:postgresql://analytics-host:5432/analytics
  DATABASES_ANALYTICS_CONNECTION_USER=analytics_user
  DATABASES_ANALYTICS_CONNECTION_PASSWORD=analytics_pass
  ```
</CodeGroup>

Reference these connections by name in your JDBC Sink nodes.

### Supported Databases

Streemlined packages the [Aiven JDBC Connector for Apache Kafka](https://github.com/Aiven-Open/jdbc-connector-for-apache-kafka) by default. Tested databases include:

* PostgreSQL
* MySQL / MariaDB
* SQL Server
* Oracle
* Snowflake
* SQLite

***

## AI Assistant

The AI assistant requires a Language Model to be configured. The following providers are available:

* Anthropic
* Mistral AI

<Tip>
  **Need another LLM?**\
  Most Language Models can be supported with a simple update of Streemlined. [Full list](https://docs.langchain4j.dev/integrations/language-models/)\
  If you need a specific provider, please ask.
</Tip>

<CodeGroup>
  ```yaml application.yml theme={null}
  langchain4j:
    anthropic:
      api-key: <your-anthropic-api-key>
      model-name: claude-opus-4-6
  #  mistral-ai:
  #    api-key: <your-mistral-api-key>
  #    model-name: mistral-medium-latest
  ```

  ```bash Environment Variables theme={null}
  LANGCHAIN4J_ANTHROPIC_API_KEY=<your-anthropic-api-key>
  LANGCHAIN4J_ANTHROPIC_MODEL_NAME=claude-opus-4-6
  ```
</CodeGroup>

***

## Metrics and Monitoring

Streemlined exposes Prometheus metrics by default:

<CodeGroup>
  ```yaml application.yml theme={null}
  micronaut:
    metrics:
      export:
        prometheus:
          enabled: true
          step: PT1M
          descriptions: true
  ```

  ```bash Environment Variables theme={null}
  MICRONAUT_METRICS_EXPORT_PROMETHEUS_ENABLED=true
  MICRONAUT_METRICS_EXPORT_PROMETHEUS_STEP=PT1M
  MICRONAUT_METRICS_EXPORT_PROMETHEUS_DESCRIPTIONS=true
  ```
</CodeGroup>

<Tip>
  Need a different metrics reporter? Micronaut Micrometer supports many alternatives including **Datadog**, **CloudWatch**, **Graphite**, **InfluxDB**, **StatsD**, **New Relic**, **Dynatrace**, **Wavefront**, **Azure Monitor**, **Stackdriver**, and more. Contact support to request an additional registry. See the [Micronaut Micrometer documentation](https://micronaut-projects.github.io/micronaut-micrometer/latest/guide/) for the full list of available registries.
</Tip>

### Available Endpoints

| Endpoint      | Description                                                                             |
| ------------- | --------------------------------------------------------------------------------------- |
| `/health`     | Health check status                                                                     |
| `/prometheus` | Prometheus metrics                                                                      |
| `/metrics`    | Micrometer endpoint for humans and debugging, not for tooling or long-term integration. |

***

## Logging

Configure logging levels via application properties:

<CodeGroup>
  ```yaml application.yml theme={null}
  logger:
    levels:
      io.streemlined: INFO
      org.apache.kafka: WARN
  ```

  ```bash Environment Variables theme={null}
  LOGGER_LEVELS_IO_STREEMLINED=INFO
  LOGGER_LEVELS_ORG_APACHE_KAFKA=WARN
  ```
</CodeGroup>

### Log Levels

| Level   | Use Case                        |
| ------- | ------------------------------- |
| `TRACE` | Only if requested by Support    |
| `DEBUG` | Development and troubleshooting |
| `INFO`  | Normal operation                |
| `WARN`  | Production (recommended)        |
| `ERROR` | Minimal logging                 |

***

## Next Steps

<CardGroup cols={2}>
  <Card title="Quickstart" icon="rocket" href="/quickstart">
    Build your first pipeline
  </Card>

  <Card title="Nodes Reference" icon="diagram-project" href="/nodes">
    Explore all available nodes
  </Card>
</CardGroup>