fix(litellm): preserve reasoning.summary when passing to LiteLLM #2144
+59
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, only `reasoning.effort` was extracted from ModelSettings,
causing `reasoning.summary` to be silently discarded. This prevented
users from receiving reasoning summaries when using GPT models through
LiteLLM.
Now we build a dict with both `effort` and `summary` fields:
`{"effort": "medium", "summary": "auto"}` which LiteLLM supports
since ~v1.78.
Fixes: BerriAI/litellm#17428
|
This was a good catch, this fixes GPT 5.1. But it breaks Anthropic, since they only expect a string. You will likely need to make this an edge case only for the openai provider. |
Only pass reasoning_effort as dict when summary is explicitly set.
When only effort is provided, pass it as a string for backward
compatibility with all providers (including Anthropic).
- Dict format: {"effort": "medium", "summary": "auto"} - OpenAI only
- String format: "medium" - all providers
Contributor
Author
Good catch! Fixed - now we only use dict format when summary (OpenAI only) is explicitly set. If only effort is provided, we pass it as a string for backward compatibility with all providers. |
Contributor
Author
You're welcome! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Previously, only
reasoning.effortwas extracted fromModelSettings.reasoning, causingreasoning.summaryto be discarded. This prevented users from receiving reasoning summaries when using GPT models through LiteLLM.The fix builds a dict with both
effortandsummaryfields ({"effort": "medium", "summary": "auto"}) which LiteLLM supports since ~v1.78.Changes:
Before:
After:
Test plan
test_reasoning_effort_prefers_model_settingsto expect dict formattest_reasoning_summary_is_preservedto verify both fields are passedIssue
Fixes BerriAI/litellm#17428
Checks
make lintandmake format