feat: #2228 Add tool origin tracking to ToolCallItem and ToolCallOutputItem#2242
feat: #2228 Add tool origin tracking to ToolCallItem and ToolCallOutputItem#2242habema wants to merge 2 commits intoopenai:mainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
seratch
left a comment
seratch
left a comment
There was a problem hiding this comment.
Our current top priority is to merge HITL enhancement, which changes runner internals a lot. So, we may ask you to resolve the conflicts after the big PR is merged.
src/agents/agent.py
Outdated
|
|
||
| return run_result.final_output | ||
|
|
||
| # Set origin tracking on the FunctionTool created by @function_tool |
There was a problem hiding this comment.
it seems the code comment here is incorrect
There was a problem hiding this comment.
Changed to Set origin tracking on run_agent (the FunctionTool returned by @function_tool) for more clarity
src/agents/tool.py
Outdated
| mcp_server_name: str | None = None | ||
| """The name of the MCP server. Only set when type is MCP.""" | ||
|
|
||
| agent_as_tool_name: str | None = None |
There was a problem hiding this comment.
This object may receive feature requests to have more properties, so simply having the reference to the agent object may be fine. Same for mcp_server_name too
There was a problem hiding this comment.
Makes sense. I've changed the implementation so that the objects are stored. I'm not too sure about the custom __repr__, below is an example of what ToolOrigin would look like with and without it. Let me know what you think.
Without a __repr__ method, ToolOrigin would look like:
- MCP
ToolOrigin(type=<ToolOriginType.MCP: 'mcp'>, mcp_server=<tests.mcp.helpers.FakeMCPServer object at 0x...>, agent_as_tool=None)
- Agent as tool:
ToolOrigin(type=<ToolOriginType.AGENT_AS_TOOL: 'agent_as_tool'>, mcp_server=None, agent_as_tool=Agent(name='test_agent', handoff_description=None, tools=[], mcp_servers=[], mcp_config={}, instructions='Test agent', prompt=None, handoffs=[], model=<tests.fake_model.FakeModel object at 0x...>, model_settings=ModelSettings(...), input_guardrails=[], output_guardrails=[], output_type=None, hooks=None, tool_use_behavior='run_llm_again', reset_tool_choice=True))
- Function:
ToolOrigin(type=<ToolOriginType.FUNCTION: 'function'>, mcp_server=None, agent_as_tool=None)
With the custom __repr__:
ToolOrigin(type='mcp', mcp_server_name='test_server')ToolOrigin(type='agent_as_tool', agent_as_tool_name='test_agent')ToolOrigin(type='function')
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8b2d12a8b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
github-actions
bot
added
dependencies
documentation
habema
force-pushed
the
feat/track-tool-origin
branch
from
1f0af9c to
8cbf13f
Compare
- Add ToolOriginType enum and ToolOrigin dataclass - Add _tool_origin field to FunctionTool - Set tool_origin for MCP tools and agent-as-tool - Extract and set tool_origin in ToolCallItem and ToolCallOutputItem creation - Add comprehensive tests for tool origin tracking
habema
force-pushed
the
feat/track-tool-origin
branch
from
8cbf13f to
edbec4e
Compare
|
Merged and updated to match new runner internals, and codex comment addressed. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e1b635702c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| tool_origin = _get_tool_origin_info(tool_run.function_tool) | ||
| run_item = ToolCallOutputItem( | ||
| output=result, | ||
| raw_item=ItemHelpers.tool_call_output_item(tool_run.tool_call, result), | ||
| agent=agent, |
There was a problem hiding this comment.
This path only sets tool_origin when run_single_tool returns a plain value, but approval rejections return a prebuilt FunctionToolResult (run_item=function_rejection_item(...)) and go through the isinstance(result, FunctionToolResult) branch without enrichment, so rejected function calls emit ToolCallOutputItem with tool_origin=None. In runs that depend on approval workflows, this makes output-item origin metadata inconsistent with successful calls and breaks origin-based tracing/auditing.
Useful? React with 👍 / 👎.
| description: str | None = None | ||
| """Optional tool description if known at item creation time.""" | ||
|
|
||
| tool_origin: ToolOrigin | None = field(default=None, repr=False) |
There was a problem hiding this comment.
tool_origin was added to ToolCallItem/ToolCallOutputItem, but run-state persistence does not include it (RunState._serialize_item/_deserialize_items only round-trip core fields like raw_item, agent, output, and description). Any interrupted run that is serialized and resumed will silently drop origin metadata from restored items, so the new tracing signal is lost exactly in resume flows.
Useful? React with 👍 / 👎.
2 participants
Adds
tool_originfield toToolCallItemandToolCallOutputItemto track where function tools came from (regular functions, MCP servers, or agent-as-tool). This helps with debugging and tracing which MCP server handled a call when using multiple servers.The
tool_originis aToolOriginobject that contains:type: The origin type (FUNCTION,MCP, orAGENT_AS_TOOL)mcp_server_name: The MCP server name (only set for MCP tools)agent_as_tool_name: The agent name (only set for agent-as-tool)New tests added
Closes #2228