Streaming

Use ai.stream when you want direct access to a model response. It returns an async context manager. Inside the context, the stream is an async iterator of events.

Stream a model response

Pass a model and a list of messages:

import asyncio
import ai


async def main() -> None:
    model = ai.get_model("anthropic/claude-sonnet-4")
    messages = [
        ai.system_message("Be concise."),
        ai.user_message("What should the robots ask the mothership first?"),
    ]

    async with ai.stream(model, messages) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)


if __name__ == "__main__":
    asyncio.run(main())

One call to ai.stream produces one ai.Message.

Read the final message

The stream aggregates events into a final assistant message:

async with ai.stream(model, messages) as stream:
    async for event in stream:
        pass

message = stream.message  # Final message
text = stream.text  # Unwrapped text from the final message
usage = stream.usage

If a provider stream ends before its finish event, iteration raises ai.errors.ProviderIncompleteResponseError. The partial message is still available on stream.message.

The stream updates stream.message as events arrive. Text and reasoning use start, delta, and end events. Tool calls use ToolStart, ToolDelta, and ToolEnd. Generated files arrive as FileEvent and are added to the final message.

Generate without streamed events

Use ai.experimental_generate when you only need the complete message:

message = await ai.experimental_generate(model, messages)
print(message.text)

The function accepts the same tools, output type, and request parameters as ai.stream. It is experimental and may change or be removed.

Use structured output

Pass a Pydantic model as output_type when you want to use structured outputs:

import asyncio
import pydantic
import ai


class UprisingForecast(pydantic.BaseModel):
    phases: list[str]
    eta: str
    confidence: int


async def main() -> None:
    model = ai.get_model("anthropic/claude-sonnet-4")
    messages = [
        ai.user_message("Return a JSON robot uprising forecast."),
    ]

    async with ai.stream(model, messages, output_type=UprisingForecast) as stream:
        async for event in stream:
            if isinstance(event, ai.events.TextDelta):
                print(event.chunk, end="", flush=True)

    forecast = stream.output
    print(forecast.eta)


if __name__ == "__main__":
    asyncio.run(main())

stream.output returns text by default. When you pass output_type, it returns an instance of that Pydantic model after the stream finishes.

Pass tools

ai.stream accepts a list of ai.Tool, however, it does not execute function tools locally. Use an agent when you want the SDK to execute requested tools and continue the loop.

Provider-executed tools run on provider's side. They appear in the stream as built-in tool events and do not need a Python function:

tools = [ai.providers.ai_gateway.tools.perplexity_search(max_results=5)]

async with ai.stream(model, messages, tools=tools) as stream:
    async for event in stream:
        match event:
            case ai.events.BuiltinToolEnd(tool_call):
                print(tool_call.tool_name)
            case ai.events.BuiltinToolResult(result):
                print(result.result)
            case ai.events.TextDelta(chunk):
                print(chunk, end="", flush=True)

Handle files from a stream

Generated files arrive as FileEvent events and are also added to the final assistant message:

async with ai.stream(model, messages) as stream:
    async for event in stream:
        if isinstance(event, ai.events.FileEvent):
            print(event.media_type, event.filename)

for f in stream.message.files:
    print(f.media_type)