Haystack

deepset's Haystack, through OpenAIChatGenerator.

Connect

pip install haystack-ai

generator = OpenAIChatGenerator(
    model="fast",
    api_base_url=URL,
    api_key=Secret.from_token(KEY),
    streaming_callback=lambda chunk: chunks.append(chunk.content or ""),
)
prompt = ChatPromptBuilder(
    template=[ChatMessage.from_user("{{ question }}")],
    required_variables=["question"],
)
pipe = Pipeline()
pipe.add_component("prompt", prompt)
pipe.add_component("llm", generator)
pipe.connect("prompt.prompt", "llm.messages")

Keep /v1 in api_base_url. api_key takes a Secret; pass Secret.from_token(...) explicitly so the plugboard key is used rather than OPENAI_API_KEY.

What works

Text in pipelines, streaming with streaming_callback, and tool calls with Tool (send the result back with ChatMessage.from_tool(..., origin=call)).

Limits

• Streamed answers carry no token counts unless you request them: add generation_kwargs={"stream_options": {"include_usage": True}}.