Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.streemlined.io/llms.txt

Use this file to discover all available pages before exploring further.

The AI Pipeline Builder is Streemlined’s assisted editing experience: you describe pipelines in plain language and the assistant updates the graph in the runner using built-in discovery and editor tools.

​
Open the assistant

In the visual editor, the left node palette uses tabs (Sources, Processors, Flow, Sinks, AI). Choose AI to open AI Pipeline Builder. You’ll see:
  • A short description: Describe the pipeline you want and AI will build it for you.
  • An empty-state example prompt such as reading from Kafka, enriching from a database, and writing to an analytics topic.
  • Clear session (trash icon) when there is history—this wipes the persisted conversation for the current pipeline name only.
Send a prompt with Send, or press Enter (same hint as under the textarea: Enter sends, Shift+Enter inserts a new line). Paste multi-line snippets when useful before sending. Optional microphone dictation appears when your browser supports the Web Speech API; follow the prompts if voice packs must load. While the runner is answering, the transcript may briefly show indicators when tools ran; history is polled so the conversation stays in sync with the server copy.

​
Prerequisites

The assistant finds real cluster names, topics, databases, and tables through your Streemlined configuration—not invented names—so discovery works only when infra is wired as in Configuration:
  • clusters: Kafka clusters available to discovery tools
  • databases: JDBC connections for listing tables and inferring schemas
Without these, prompts that depend on Kafka or JDBC metadata will stall or produce errors until you fix application.yml (and connectivity). The AI assistant also needs valid Anthropic credentials and related settings. Configure them as described in AI assistant (Anthropic).
If you automate Streemlined from scripts or downstream services, the same assistant is reachable over HTTP:
  • POST /ai/generate with JSON body { "prompt": "your request" }
  • GET /ai/messages/{memoryId} and DELETE /ai/messages/{memoryId} for inspecting or clearing persisted chat
memoryId in the runner matches the current pipeline name.

​
Behavior and constraints

Conversation memory is stored per pipeline name for up to roughly the last fifty messages. If you rename the pipeline, the assistant treats it as a new conversation id until you rename back or reuse the old id. While the pipeline is running, the AI panel inputs are disabled—you must stop before continuing with the assistant. While a generation is in progress, dragging new nodes from the palette onto the canvas is blocked (you’ll see a toast). Wait for the turn to finish, or keep editing after it completes.

​
What the assistant can do

Behind the scenes, the model drives tools connected to Streemlined’s editor:
AreaWhat it can help with
KafkaList configured clusters, search topics by substring, fetch topic schemas (Kafka Connect JSON)
DatabasesList configured JDBC sources, search tables/views, inspect table/query schemas
GraphInspect stage kinds and field hints, read the current diagram and validation issues, add or remove stages and links, update stage configs
RuntimeStart or stop the current pipeline from the orchestrator context and inspect status
Replies aim for concise natural language—it does not dump entire pipeline JSON unless you integrate via API and parse tool outputs yourself.

​
Configure the AI assistant

Shipped defaults live in kanal/src/main/resources/application.yml. For property names, environment-variable overrides, and UI behavior (limits, HTTP endpoints), see AI assistant (Anthropic) and the Prerequisites section above (including the HTTP tip).