Braintrust Python SDKs

Braintrust is a platform for evaluating and shipping AI products. Learn more at braintrust.dev and in the docs.

This repository contains Braintrust's Python SDKs and integrations, including:

• The main braintrust SDK package in ./py
• Built-in integrations under py/src/braintrust/integrations and related compatibility packages under ./integrations
• Self-contained uv-project examples for every integration in ./examples
• Tests and local development tooling for Python SDK development

Quickstart

Install the main SDK and scorer package:

# uv
uv add braintrust autoevals
# pip
pip install braintrust autoevals

Create tutorial_eval.py:

from autoevals import LevenshteinScorer
from braintrust import Eval

Eval(
    "Say Hi Bot",
    data=lambda: [
        {"input": "Foo", "expected": "Hi Foo"},
        {"input": "Bar", "expected": "Hello Bar"},
    ],
    task=lambda input: "Hi " + input,
    scores=[LevenshteinScorer],
)

Run it:

BRAINTRUST_API_KEY=<YOUR_API_KEY> braintrust eval tutorial_eval.py

Packages

Package	Purpose	PyPI	Docs
braintrust	Core Python SDK for logging, tracing, evals, CLI workflows, and built-in integrations.		py/README.md

Integrations included in braintrust

Integration	Auto-instrumented	Min version
OpenAI	Yes	openai>=1.71.0
Anthropic	Yes	anthropic>=0.48.0
Cohere	Yes	cohere>=5.0.0
Mistral	Yes	mistralai>=1.12.4
LiteLLM	Yes	litellm>=1.74.0
OpenRouter	Yes	openrouter>=0.6.0
Vercel AI SDK for Python	Yes	ai>=0.4.0
Google GenAI	Yes	google-genai>=1.30.0
Bedrock Runtime	Yes	boto3>=1.34.116
HuggingFace Hub	Yes	huggingface-hub>=0.32.0
Hugging Face Transformers	Yes	transformers>=4.42.0
Google ADK	Yes	google-adk>=1.14.1
Pydantic AI	Yes	pydantic_ai>=1.10.0
LangChain	Yes	langchain-core>=0.3.28
LlamaIndex	Yes	llama-index-core>=0.13.0
Instructor	Yes	instructor>=1.11.0
DSPy	Yes	dspy>=2.6.0
OpenAI Agents	Yes	openai-agents>=0.0.19
Claude Agent SDK	Yes	claude_agent_sdk>=0.1.10
Cursor SDK	Yes	cursor-sdk>=1.0.25
AutoGen	Yes	autogen-agentchat>=0.7.0
CrewAI	Yes	crewai>=1.13.0
Strands	Yes	strands-agents>=1.20.0
Agno	Yes	agno>=2.1.0
AgentScope	Yes	agentscope>=1.0.0
LiveKit Agents	Yes	livekit-agents>=1.3.1
Pipecat	Yes	pipecat-ai>=1.3.0
Temporal	Yes	temporalio>=1.19.0
Harbor	No (Harbor plugin)	harbor>=0.16.0
pytest plugin	No	pytest>=8

Documentation

• Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python
• Release notes: https://www.braintrust.dev/docs/reference/release-notes

License

Apache-2.0