Python for JS Devs · LLM APIs

Calling an LLM API from FastAPI

The payoff. Everything so far — type hints, Pydantic, async, DI, the blocking trap — comes together in one LLM-backed endpoint.

You'll use Anthropic's official Python SDK (anthropic). The same shape applies to OpenAI's SDK. We default to Claude Opus 4.8 (model id claude-opus-4-8), the current most capable model.

1 · Install & the smallest call

uv add anthropic

Set your key as an env var (the SDK reads ANTHROPIC_API_KEY automatically — never hardcode it):

# .env
ANTHROPIC_API_KEY=sk-ant-...

const client = new Anthropic();
const msg = await client.messages
  .create({ model, max_tokens, messages });

import anthropic
client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY

msg = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}],
)

text = next(b.text for b in msg.content if b.type == "text")

2 · Use the ASYNC client in FastAPI (Lesson 17 pays off)

An HTTP call to an LLM is I/O that takes seconds. In an async def route you must use the async client and await it — a blocking call here would freeze the event loop for every other request.

import anthropic
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()
client = anthropic.AsyncAnthropic()   # the async client

class ChatIn(BaseModel):
    message: str

class ChatOut(BaseModel):
    reply: str

@app.post("/chat", response_model=ChatOut)
async def chat(body: ChatIn):
    msg = await client.messages.create(     # awaited → loop stays free
        model="claude-opus-4-8",
        max_tokens=1024,
        messages=[{"role": "user", "content": body.message}],
    )
    reply = next(b.text for b in msg.content if b.type == "text")
    return ChatOut(reply=reply)

Every earlier lesson is visible here: Pydantic models validate in/out (12, 13), the async client cooperates with the loop (10, 17), and the API key comes from typed settings (16).

3 · Config & errors, the right way

try:
    msg = await client.messages.create(...)
except anthropic.RateLimitError:
    raise HTTPException(status_code=429, detail="Slow down")
except anthropic.APIStatusError as e:
    raise HTTPException(status_code=502, detail="LLM upstream error")

Check yourself