Getting Started
Reference

ai

Reference for top-level ai exports.

ai is the application-facing namespace. This page mirrors the public names re-exported from ai.__all__.

import ai

model = ai.get_model("anthropic/claude-sonnet-4")
messages = [ai.user_message("Hello")]

Linked pages

These top-level exports have their own pages under ai.

  • stream: Call a model and iterate response events. The same page documents Stream.
  • experimental_generate: Call a model and return a buffered Message.
  • get_model: Resolve a model id into a Model.
  • get_provider: Resolve and configure a provider.
  • @ai.tool: Define executable tools from Python functions.
  • Agent: Run the default agent loop.

Module aliases

These module aliases are also re-exported from ai.

  • errors: Error classes and HTTP error helpers.
  • events: Stream, agent, tool, and hook events.
  • messages: Message and message part models.
  • models: Model namespace used by the top-level model exports on this page.
  • mcp: MCP tool loading helpers.
  • ops: Dedicated model operations.
  • providers: Provider classes and provider-specific namespaces.
  • testing: Scripted model and agent testing.
  • tools: Model-facing tool schema types.
  • ui: UI adapter namespaces.
  • util: Async utility helpers.
  • experimental_telemetry: Tracing and observability APIs.

Models

Model

Model identifies what to call. Providers own credentials, clients, endpoints, model listing, and wire translation.

model = ai.Model(id="gpt-5", provider=provider)
model.id
model.provider
model.protocol
model.with_protocol(protocol)

Model is a lightweight reference. It does not own network state.

Fields:

  • id: Provider model id.
  • provider: Provider instance.
  • protocol: Optional provider protocol override.

Methods:

  • with_protocol(protocol): Return a copy that uses a specific provider protocol.

probe

probe asks the model provider to verify that a model exists and is reachable.

await ai.probe(model)

Providers

Provider

Provider is the base class for provider instances. Providers own credentials, clients, endpoints, model listing, and wire translation.

provider.name
provider.base_url
provider.api_key
provider.headers
provider.protocol
await provider.list_models()
await provider.probe(model)

Subclasses implement provider-specific configuration and clients. Application code usually gets a provider with get_provider.

ProviderProtocol

ProviderProtocol translates messages, tools, params, and dedicated model operations to provider wire formats.

Provider instances use a protocol for language-model calls and ai.ops.

protocol.stream(
    client,
    model,
    messages,
    tools=tools,
    params=params,
    provider=provider.name,
)
await protocol.generate(
    client,
    model,
    messages,
    tools=tools,
    output_type=output_type,
    params=params,
    provider=provider.name,
)

Request Params

Model params are top-level ai types.

Use InferenceRequestParams with stream and Agent.run.

params = ai.InferenceRequestParams().with_temperature(0)
async with ai.stream(model, messages, params=params) as stream:
    ...

Request params:

  • InferenceRequestParams: Inference request options.
  • ProviderServiceParams: Provider service tier options.
  • ReasoningParams: Provider reasoning or thinking options.
  • OutputParams: Output token, include, verbosity, and reasoning summary options.
  • CacheParams: Prompt cache options.
  • ContextManagementParams: Server-side context management options.
  • TokenThreshold: Token count used as a trigger threshold.

Sampling params:

  • TemperatureSamplerParams
  • TopKSamplerParams
  • TopPSamplerParams
  • MinPSamplerParams
  • RepetitionPenaltyParams
  • SeedSamplerParams
  • RandomSeed
  • RANDOM
  • DEFAULT
  • UNSET
  • ModelProviderDefault
  • Unset

Tool calling params:

  • ToolCallingParams
  • ToolChoiceMode
  • ToolSelection
  • ToolRef

Routing params:

  • RoutingParams
  • RoutingTarget
  • RoutingTargetChain
  • GeoRegion
  • CloudRegion
  • ProviderRankingStrategy
  • GLOBAL

Messages

Message builders create Message values.

ai.message("Hello", role="user")
ai.system_message("You are concise.")
ai.user_message("Hello", ai.file_part(data, media_type="image/png"))
ai.assistant_message("Hi")
ai.tool_message(tool_call_id="tc_1", result="done", tool_name="lookup")

Message builder exports:

  • message
  • system_message
  • user_message
  • assistant_message
  • tool_message

Part builders create message part values.

ai.text_part("hello")
ai.file_part(data, media_type="image/png", filename="image.png")
ai.thinking("reasoning text")
ai.content_output("caption", ai.file_part(png_bytes, media_type="image/png"))
ai.tool_result_part("tc_1", result={"ok": True}, tool_name="lookup")

Part builder exports:

  • text_part
  • file_part
  • thinking
  • content_output
  • tool_result_part

Tools and Agents

Agent

Agent runs the default agent loop.

agent = ai.Agent(tools=[contact_mothership])

Arguments:

  • tools: Optional AgentTool values from tool and schema-only Tool declarations for provider-executed tools.

AgentTool

AgentTool binds a model-facing Tool declaration to an executable Python function.

tool.name
tool.tool
tool.fn
tool.validator
tool.require_approval

Pass AgentTool values to Agent(tools=[...]).

Context

Custom loops use Context to resolve model tool calls and ToolRunner to run them.

context.model
context.messages
context.tools
context.output_type
context.params

Useful methods:

  • keep_running(): Return True while the last message still needs work.
  • resolve(tool_call): Convert model tool call parts into executable ToolCall objects.
  • add(message): Append messages to history.

ToolCall

ToolCall is the executable runtime object produced by Context.resolve.

tool_call.id
tool_call.name
tool_call.fn
tool_call.kwargs
result = await tool_call()

ToolRunner

ToolRunner schedules tool calls and collects their result messages.

async with ai.ToolRunner() as runner:
    runner.schedule(tool_call)
    async for result in runner.events():
        ...
    message = runner.get_tool_message()

Use add_result(result) when a custom loop executes a tool itself but still wants the runner to aggregate the result message.

Streaming tool aliases

Async-iterable tools can yield partial output while they run.

  • StreamingTextTool: Concatenate yielded strings.
  • StreamingStatusTool[T]: Treat intermediate yields as status updates and the last yielded value as the final result.
  • SubAgentTool: Forward nested agent events and use the nested final text as model input.

Tool result helpers

ai.tool_result(tool_call_id="tc_1", tool_name="lookup", result={"ok": True})
ai.deferred_tool_result(hook_part, tool_call_id="tc_1", tool_name="lookup")

Exports:

  • tool_result: Create a ToolCallResult.
  • deferred_tool_result: Create a deferred hook placeholder result.

Hooks

Hooks let an agent pause while another process or UI supplies a decision.

approval = await ai.hook(
    "approve_contact_mothership",
    payload=ai.tools.ToolApproval,
    metadata={"tool": "contact_mothership"},
)

Hook exports:

  • hook: Emit a deferred hook event and wait for a matching resolution.
  • resolve_hook: Resolve a live or future hook.
  • defer_hook: Mark a serialized deferred hook as aborted.
  • cancel_hook: Cancel a live hook by label.
  • HookRegistry: Store live and pre-registered hook resolutions.
  • get_hook_registry: Return the current hook registry.
  • HookDeferredException: Signal that a hook was deferred for a later run.

yield_from

yield_from forwards values from an async iterable through the current agent runtime and returns the aggregator's model-facing result. See ai.agents for the advanced API.

Errors

Top-level error exports are documented in ai.errors.

PreviousReferenceNextstream