Skip to content

fix(langchain): Stop double-counting usage in multi-candidate responses - #7064

Merged
alexander-alderman-webb merged 7 commits into
masterfrom
webb/langchain/stop-double-counting-candidate-tokens
Aug 10, 2026
Merged

fix(langchain): Stop double-counting usage in multi-candidate responses#7064
alexander-alderman-webb merged 7 commits into
masterfrom
webb/langchain/stop-double-counting-candidate-tokens

Conversation

@alexander-alderman-webb

@alexander-alderman-webb alexander-alderman-webb commented Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Description

Take token usage from the first candidate object in each generation, instead of summing token usage from each candidate.

The inner list in LLMResult.generations contains candidate generations (multiple output choices). Candidates do not have independent token usage, but langchain hooks duplicate the token usage on each candidate.

Add a test using a google-genai response, as ChatOpenAI models do not enter the else: branch below:

sentry-python/sentry_sdk/integrations/langchain.py

Lines 769 to 774 in f670a1b

def _record_token_usage(span: "Union[Span, StreamedSpan]", response: "Any") -> None:
token_usage = _get_token_usage(response)
if token_usage:
input_tokens, output_tokens, total_tokens = _extract_tokens(token_usage)
else:
input_tokens, output_tokens, total_tokens = _extract_tokens_from_generations(

Issues

Reminders

@github-actions

github-actions Bot commented Aug 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Codecov Results 📊

✅ 99560 passed | ⏭️ 6671 skipped | Total: 106231 | Pass Rate: 93.72% | Execution Time: 348m 44s

📊 Comparison with Base Branch

Metric Change
Total Tests 📈 +72
Passed Tests 📈 +24
Failed Tests —
Skipped Tests 📈 +48

All tests are passing successfully.

✅ Patch coverage is 88.89%. Project has 2501 uncovered lines.
❌ Project coverage is 90%. Comparing base (base) to head (head).

Files with missing lines (1)
File Patch % Lines
sentry_sdk/integrations/langchain.py 88.89% ⚠️ 1 Missing and 1 partials
Coverage diff 
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
- Coverage    90.01%    90.00%    -0.01%
==========================================
  Files          193       193         —
  Lines        25020     25021        +1
  Branches      9032      9032         —
==========================================
+ Hits         22521     22520        -1
- Misses        2499      2501        +2
- Partials      1434      1438        +4

Generated by Codecov Action

@alexander-alderman-webb
alexander-alderman-webb marked this pull request as ready for review August 6, 2026 13:38
@alexander-alderman-webb
alexander-alderman-webb requested a review from a team as a code owner August 6, 2026 13:38
@alexander-alderman-webb
alexander-alderman-webb enabled auto-merge (squash) August 10, 2026 09:16
@alexander-alderman-webb
alexander-alderman-webb enabled auto-merge (squash) August 10, 2026 09:19
@alexander-alderman-webb
alexander-alderman-webb enabled auto-merge (squash) August 10, 2026 09:20
@alexander-alderman-webb
alexander-alderman-webb merged commit 488f971 into master Aug 10, 2026
145 checks passed
@alexander-alderman-webb
alexander-alderman-webb deleted the webb/langchain/stop-double-counting-candidate-tokens branch August 10, 2026 09:23
@sentry-release-bot sentry-release-bot Bot mentioned this pull request Aug 10, 2026
Closed
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sentrivana sentrivana sentrivana approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alexander-alderman-webb @sentrivana