LangChain Python (ChatAnthropic)

ChatAnthropic from langchain-anthropic, on plugboard's Messages API. For the OpenAI-shaped client, see LangChain Python.

Connect

base_url is the plugboard origin, without /v1:

def chat_model(model, **overrides):
    from langchain_anthropic import ChatAnthropic

    settings = {
        "model": model,
        "base_url": os.environ["PLUGBOARD_BASE_URL"].removesuffix("/v1"),
        "api_key": os.environ["PLUGBOARD_KEY"],
        "max_tokens": 1024,
        **NO_RETRIES,
    }
    settings.update(overrides)
    return ChatAnthropic(**settings)

Tool calls

llm = chat_model(model).bind_tools([GET_WEATHER])
first = llm.invoke([HumanMessage(PROMPT)])
call = first.tool_calls[0]
second = llm.invoke([HumanMessage(PROMPT), first, ToolMessage(content=json.dumps(WEATHER), tool_call_id=call["id"])])

Streaming

for chunk in chat_model(model).stream(PROMPT):
    text += text_of(chunk.content)
    # The last chunk (the message_delta) carries the stop reason in response_metadata.
    if chunk.response_metadata.get("stop_reason"):
        finish = finish_reason(chunk.response_metadata["stop_reason"])
    if chunk.usage_metadata:
        usage = dict(chunk.usage_metadata)

The stop reason of a streamed answer is on the last chunk, in response_metadata["stop_reason"].

Structured output

output = chat_model(model).with_structured_output(Meal, method="function_calling").invoke(PROMPT)

Limits

• ChatAnthropic has no model-list call. GET /v1/models with your key works.
• Errors carry error.type and no error.code.