Skip to content

Support openai-agents#4437

Merged
antonpirker merged 95 commits into
masterfrom
antonpirker/openai-agents-integration
Jun 24, 2025
Merged

Support openai-agents#4437
antonpirker merged 95 commits into
masterfrom
antonpirker/openai-agents-integration

Conversation

@antonpirker

@antonpirker antonpirker commented Jun 3, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

Add support for AI agents projects using openai-agents (https://pypi.org/project/openai-agents/)

Docs PR is here: getsentry/sentry-docs#14113

This integration:

  • records tracing data of agent invocation, tool execution, ai client requests to LLMs, and handoffs to other agents.
  • captures input and output to/from LLMs if set_default_pii=True.
  • is mostly compatible to the OpenTelememetry gen_ai semantic conventions. (input and output is not compatible because Sentry does not have Span events. This information is stored in arrays on the Span attributes.
  • Captures errors that happen during agent execution (like problems during interaction with the LLM.

This integration does not:

  • Capture errors during function tool exection because this is very hard to patch (see comment in the code)

Example span tree in Sentry.io:
Screenshot 2025-06-24 at 12 15 17

@codecov

codecov Bot commented Jun 3, 2025 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 83.33333% with 77 lines in your changes missing coverage. Please review.

Project coverage is 80.64%. Comparing base (ad2bbff) to head (4e4f39c).
Report is 2 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
sentry_sdk/integrations/openai_agents/utils.py 72.56% 23 Missing and 8 partials ⚠️
...dk/integrations/openai_agents/patches/agent_run.py 80.00% 8 Missing and 5 partials ⚠️
...ry_sdk/integrations/openai_agents/patches/tools.py 78.12% 6 Missing and 1 partial ⚠️
...k/integrations/openai_agents/spans/execute_tool.py 75.00% 3 Missing and 3 partials ⚠️
...y_sdk/integrations/openai_agents/patches/models.py 83.33% 4 Missing ⚠️
...k/integrations/openai_agents/spans/invoke_agent.py 77.77% 3 Missing and 1 partial ⚠️
...y_sdk/integrations/openai_agents/patches/runner.py 86.95% 2 Missing and 1 partial ⚠️
..._sdk/integrations/openai_agents/spans/ai_client.py 82.35% 3 Missing ⚠️
sentry_sdk/integrations/openai_agents/__init__.py 90.00% 2 Missing ⚠️
...integrations/openai_agents/spans/agent_workflow.py 77.77% 2 Missing ⚠️
... and 1 more
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #4437      +/-   ##
==========================================
+ Coverage   80.57%   80.64%   +0.06%     
==========================================
  Files         142      156      +14     
  Lines       16057    16475     +418     
  Branches     2748     2805      +57     
==========================================
+ Hits        12938    13286     +348     
- Misses       2251     2304      +53     
- Partials      868      885      +17
Files with missing lines Coverage Δ
sentry_sdk/consts.py 94.63% <100.00%> (+1.18%) ⬆️
sentry_sdk/integrations/__init__.py 77.01% <ø> (ø)
sentry_sdk/integrations/openai_agents/consts.py 100.00% <100.00%> (ø)
...sdk/integrations/openai_agents/patches/__init__.py 100.00% <100.00%> (ø)
...y_sdk/integrations/openai_agents/spans/__init__.py 100.00% <100.00%> (ø)
sentry_sdk/integrations/openai_agents/__init__.py 90.00% <90.00%> (ø)
...integrations/openai_agents/spans/agent_workflow.py 77.77% <77.77%> (ø)
...ry_sdk/integrations/openai_agents/spans/handoff.py 77.77% <77.77%> (ø)
...y_sdk/integrations/openai_agents/patches/runner.py 86.95% <86.95%> (ø)
..._sdk/integrations/openai_agents/spans/ai_client.py 82.35% <82.35%> (ø)
... and 6 more

... and 2 files with indirect coverage changes

@antonpirker
antonpirker marked this pull request as ready for review June 23, 2025 12:00
@antonpirker
antonpirker requested a review from a team as a code owner June 23, 2025 12:00
sentrivana
sentrivana reviewed Jun 23, 2025
View reviewed changes
Comment thread sentry_sdk/integrations/openai_agents/utils.py Outdated
Comment thread sentry_sdk/integrations/openai_agents/utils.py
Comment thread sentry_sdk/integrations/openai_agents/consts.py
Comment thread sentry_sdk/integrations/openai_agents/patches/models.py
Comment thread sentry_sdk/integrations/openai_agents/patches/models.py
Comment thread sentry_sdk/integrations/openai_agents/patches/agent_run.py Outdated
Comment thread sentry_sdk/integrations/openai_agents/patches/agent_run.py Outdated
Comment thread sentry_sdk/integrations/openai_agents/patches/agent_run.py Outdated
Comment thread sentry_sdk/integrations/openai_agents/patches/agent_run.py Outdated
Comment thread sentry_sdk/integrations/openai_agents/patches/agent_run.py Outdated
@antonpirker
antonpirker requested a review from sentrivana June 24, 2025 10:16
sentrivana
sentrivana approved these changes Jun 24, 2025
View reviewed changes

@sentrivana sentrivana left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

But let the record show I don't like having the spans stuff in separate files 😢

@antonpirker
antonpirker enabled auto-merge (squash) June 24, 2025 12:24
@antonpirker

antonpirker commented Jun 24, 2025

Copy link
Copy Markdown
Contributor Author

Thanks Ivana!

@antonpirker
antonpirker merged commit 4a0e5ed into master Jun 24, 2025
136 checks passed
@antonpirker
antonpirker deleted the antonpirker/openai-agents-integration branch June 24, 2025 12:42
@szokeasaurusrex szokeasaurusrex mentioned this pull request Jun 30, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sentrivana sentrivana sentrivana approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@antonpirker @sentrivana