Quickstart

Step 1: Configuration file

Create an application.yml file with your license, Kafka clusters, and databases:

license:
  token: <your-license-token>

clusters:
  my-cluster:
    bootstrap.servers: localhost:9092

license:
  token: <your-license-token>

clusters:
  my-cluster:
    bootstrap.servers: localhost:9092
    security.protocol: SASL_PLAINTEXT
    sasl.mechanism: PLAIN
    sasl.jaas.config: | 
      org.apache.kafka.common.security.plain.PlainLoginModule required 
      username="admin" 
      password="secret";
    registry:
      # Full list of available Schema Registry properties
      # https://docs.confluent.io/platform/current/schema-registry/sr-client-configs.html
      schema.registry.url: http://localhost:8081
      basic.auth.credentials.source: USER_INFO 
      basic.auth.user.info: username:password

databases:
  default: 
    # Full list of available JDBC connection properties
    # https://docs.confluent.io/kafka-connectors/jdbc/current/sink-connector/sink_config_options.html#connection
    connection.url: jdbc:postgresql://localhost:5432/mydb
    connection.user: postgres
    connection.password: secret

Step 2: Run

Compose
Docker

Create a docker-compose.yml file

docker-compose.yml

services:
  streemlined:
    image: streemlined/streemlined:latest
    ports:
      - "8080:8080"
    volumes:
      - ./application.yml:/home/app/application.yml

command

docker compose up -d

command

docker run -p 8080:8080 -v ./application.yml:/app/application.yml streemlined/streemlined:latest

Open your browser and navigate to http://localhost:8080. You’re ready to build your first pipeline!

Step 3 (optional): Persist pipelines

By default, pipelines are stored inside the container. If the container is removed, your work is lost. Mount a local directory or use a Docker volume to persist your pipelines across restarts:

Local Directory
Docker Volume

docker-compose.yml

services:
  streemlined:
    image: streemlined/streemlined:latest
    ports:
      - "8080:8080"
    volumes:
      - ./application.yml:/home/app/application.yml
      - ./storage:/home/app/storage

docker-compose.yml

services:
  streemlined:
    image: streemlined/streemlined:latest
    ports:
      - "8080:8080"
    volumes:
      - ./application.yml:/home/app/application.yml
      - storage:/home/app/storage
volumes:
  storage:

The default Streemlined image comes pre-packaged with two Sink Connectors:

JDBC Connector for Apache Kafka — Transfer data from Kafka topics to relational databases
HTTP Connector for Apache Kafka — Send data from Kafka topics to HTTP endpoints

Need additional connectors? See Adding Connector Plugins to extend Streemlined with more Kafka Connect connectors.

What’s Next?

Core Concepts

Understand pipelines, nodes, and JSONata

Visual Editor

Master the pipeline editor

Nodes Reference

Explore all available nodes

Get Started

Build Pipelines

Test Pipelines

Deploy Pipelines

Step 1: Configuration file

Step 2: Run

Step 3 (optional): Persist pipelines

What’s Next?

Core Concepts

Visual Editor

Nodes Reference

​Step 1: Configuration file

​Step 2: Run

​Step 3 (optional): Persist pipelines

​What’s Next?

Core Concepts

Visual Editor

Nodes Reference

Step 1: Configuration file

Step 2: Run

Step 3 (optional): Persist pipelines

What’s Next?