Skip to content

DSPy ​

DSPy reaches models through LiteLLM; the openai/ prefix sends them to Chat Completions.

Connect ​

py
def lm_for(model: str) -> dspy.LM:
    return dspy.LM(f"openai/{model}", api_base=os.environ["PLUGBOARD_BASE_URL"], api_key=os.environ["PLUGBOARD_KEY"], cache=False)

Put api_base on each dspy.LM, not in the model string. A second model needs its own dspy.LM.

Predict and ChainOfThought ​

py
if variant == "predict":
    out = dspy.Predict("question -> answer")(question="Reply with exactly: pong")
else:
    out = dspy.ChainOfThought("question -> answer")(question="Reply with exactly: pong")

Typed output ​

py
class Estimate(dspy.Signature):
    """Estimate the meal's nutrition."""

    question: str = dspy.InputField()
    answer: Meal = dspy.OutputField()

out = dspy.Predict(Estimate)(question=PROMPT)

DSPy asks for its own text format first and falls back to a JSON schema when the text doesn't parse.

Limits ​

  • LiteLLM fetches its price map from GitHub on import. Set LITELLM_LOCAL_MODEL_COST_MAP=True to skip it.