fix(generic_guardrail_api): return tool_calls on the guardrail response by ninadphalak · Pull Request #41013 · BerriAI/litellm

ninadphalak · 2026-09-13T18:40:35Z

TLDR

Problem this solves:

a guardrail that rewrites tool arguments cannot return them
the reply keeps the arguments the caller sent in
the Responses path warns "no tool calls" on every tool call

How it solves it:

the guardrail response can carry tool calls
returned calls are validated before they are applied
an absent or unusable list keeps the calls already sent

User Flow

Before: an operator rewrites tool arguments in a guardrail and the rewritten values never reach the application

The operator configures a guardrail on the generic guardrail API route that rewrites a postal code inside a tool call's arguments
The application sends POST https://litellm-domain/v1/chat/completions with "tools" present and gets back finish_reason: "tool_calls"
The tool call in the reply still carries the postal code the application sent, so it dispatches to the address it already knew
Nothing in the response says the guardrail's values were ignored

After: the same request comes back with the guardrail's values in the tool call

The application sends the same POST https://litellm-domain/v1/chat/completions
The same tool call comes back with the rewritten postal code in arguments
The application dispatches to the address the guardrail returned
A guardrail that returns a call with no function name or arguments leaves the reply unchanged, and the proxy logs why

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

I have added meaningful tests
The handful of test files covering my change pass locally: I could not run them, my environment has openai 3.5.0 while this repo pins <3.0.0, so LiteLLM does not import here. See Caveats
My PR passes all required CI/CD checks: lint fails on this branch, see Caveats for which step and why
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 (4/5 on the first revision and 4/5 on the second, with its findings addressed in each case)

Screenshots / Proof of Fix

Pending: this environment cannot run a live proxy with a guardrail endpoint and a provider key, so I have no unmocked Before and After to show. The unit tests in tests/test_litellm/proxy/guardrails/guardrail_hooks/test_generic_guardrail_api.py are the evidence so far, and I will add the end to end run if a maintainer wants it before review

Type of Pull Request

🐛 Bug Fix

Caveats (if any)

Medium

tool_calls: list[...] | list[...] | None is the exact type the field carries, and it adds 6 to the LIT001 ceiling in type-discipline-budget.json. Annotating it as a read-only Sequence is not an option: the value is assigned into GenericGuardrailAPIInputs, whose tool_calls field is declared as a list, so a Sequence fails the basedpyright budget instead. Raising the ceiling by 6 or removing 6 elsewhere are both one line changes if a maintainer prefers either

Low

the tests drive a mocked guardrail response, no live guardrail server was run
the end to end proof section is pending, see above
streamed tool calls stay out of scope: incremental_diff is documented for string delta.content only
a returned call whose arguments is not a string is treated as unusable and the sent calls are kept
this branch's lint job also fails on steps that are red on the base branch, so it is not a green signal here

`GenericGuardrailAPIRequest` carries `tool_calls` to the guardrail, but `GenericGuardrailAPIResponse` has no matching field and `_build_guardrail_return_inputs` never set the key, so a guardrail that rewrites tool arguments cannot hand them back. The read side already exists: - `litellm/llms/openai/chat/guardrail_translation/handler.py` reads `guardrailed_inputs.get("tool_calls", [])`, and the output path reads `returned_tool_calls` with a length guard - `litellm/llms/anthropic/chat/guardrail_translation/handler.py` reads `returned_tool_calls`, also length-guarded - `litellm/llms/openai/responses/guardrail_translation/handler.py` passes it to `_post_guardrail_tool_call_shapes`, which falls back and logs "guardrail returned no tool calls for the N scanned" whenever the key is absent Only the generic hook's write side was missing, so all of those lookups missed and a guardrail's returned tool calls were dropped. Add the field and thread it through, mirroring `tools`/`images`: the guardrail's replacements win, otherwise the calls that were sent are echoed back unchanged. Every consumer compares the returned list against the one it sent and falls back to its own copy on a length mismatch, so an unchanged echo leaves existing guardrails behaving exactly as before.

codspeed-hq · 2026-09-13T18:43:27Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing ninadphalak:fix/generic-guardrail-response-tool-calls (47cf317) with litellm_internal_staging (c2c2a62)}

greptile-apps · 2026-09-13T18:43:36Z

Greptile Summary

This revision fixes the invalid TypedDict alias and keeps tool-call selection immutable while preserving the guardrail write-back behavior

Adds tool calls to generic guardrail responses and returns validated rewrites to downstream response handlers
Falls back to the original calls when returned calls are absent, malformed, or unusable downstream
Adds parsing, precedence, fallback, and malformed-response regression coverage

Confidence Score: 4/5

The code changes appear functionally safe, but the unresolved repository requirement for concise test comments must be satisfied before merging

Tool-call rewrites are validated and downstream handlers preserve original calls when a returned list is absent or has the wrong length. The import-breaking type alias and malformed-call crash findings are fixed. Three earlier threads were manually resolved without explanatory replies. The existing verbose test-docstring finding remains unresolved

Files Needing Attention: tests/test_litellm/proxy/guardrails/guardrail_hooks/test_generic_guardrail_api.py

Important Files Changed

Filename	Overview
litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py	Propagates validated guardrail tool-call rewrites and retains original calls when returned values are unusable
litellm/types/proxy/guardrails/guardrail_hooks/generic_guardrail_api.py	Adds tool calls to the guardrail response model and replaces the import-breaking alias with explicit supported types
tests/test_litellm/proxy/guardrails/guardrail_hooks/test_generic_guardrail_api.py	Adds useful tool-call regression coverage, but the existing unresolved verbose-docstring policy finding remains

_{Reviews (3): Last reviewed commit: "fix(generic_guardrail_api): annotate the..." | Re-trigger Greptile}

codecov · 2026-09-13T18:53:28Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…tra comments A same-length list of tool calls with no function block was forwarded as-is, and the chat handlers index function.name and function.arguments on whatever comes back, so an unusable list raised there instead of returning a response. Returned calls are now checked for those two fields, and the calls already sent are kept when one is missing them, which is the fallback the responses handler already takes. The new field is annotated with the request's own field type rather than spelling the list union out twice, which keeps it exact and keeps the type-discipline budget flat, and the comment blocks that restated the code are gone.

ninadphalak · 2026-09-13T18:55:47Z

@greptileai

greptile-apps · 2026-09-13T18:58:53Z

+    @pytest.mark.asyncio
+    async def test_guardrail_returned_tool_calls_flow_back_to_inputs(self, generic_guardrail):
+        """A guardrail that rewrites tool arguments has to be able to return them.
+
+        ``tool_calls`` is sent on the request, so the matching response field is what closes
+        the loop. Without it the calls were dropped on the floor and the caller silently
+        kept the values it sent in -- for a rewriting guardrail (reversible redaction, for
+        instance) the rewritten arguments never reached the client.
+        """


Verbose test comments violate policy

These docstrings restate straightforward behavior, including similar blocks near lines 1937 and 1999. Repository policy requires removing them before merge

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

… never rebind The type alias that named this field read the request's field through a TypedDict subscript, which ruff reads as a forward reference, and it failed the basedpyright budget. The annotations spell the union out again, which is exact and checks against GenericGuardrailAPIInputs without an alias and without a suppression comment. The unusable-reply path no longer rebinds a Final local, so it needs no suppression comment either: the calls that will be applied are picked in one expression.

ninadphalak · 2026-09-14T01:09:17Z

@greptileai

greptile-apps Bot reviewed Sep 13, 2026

View reviewed changes

Comment thread litellm/types/proxy/guardrails/guardrail_hooks/generic_guardrail_api.py

Comment thread litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py Outdated

greptile-apps Bot reviewed Sep 13, 2026

View reviewed changes

ninadphalak mentioned this pull request Sep 13, 2026

fix(rehydration): restore Anthropic tool calls, and escape values on the JSON path ninadphalak/LLM-Shield-Proxy#33

Merged

Navigation Menu

Uh oh!

Uh oh!

fix(generic_guardrail_api): return tool_calls on the guardrail response - #41013

fix(generic_guardrail_api): return tool_calls on the guardrail response#41013
ninadphalak wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
ninadphalak:fix/generic-guardrail-response-tool-calls

ninadphalak commented Sep 13, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Sep 13, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 13, 2026 •

edited

Loading

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Sep 13, 2026 •

edited

Loading

Uh oh!

ninadphalak commented Sep 13, 2026

Uh oh!

Uh oh!

greptile-apps Bot Sep 13, 2026

Uh oh!

ninadphalak commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

ninadphalak commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

User Flow

Relevant issues

Affected release

Linear ticket

Pre-Submission checklist

Screenshots / Proof of Fix

Type of Pull Request

Caveats (if any)

Medium

Low

Uh oh!

codspeed-hq Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ninadphalak commented Sep 13, 2026

Uh oh!

Uh oh!

greptile-apps Bot Sep 13, 2026

Choose a reason for hiding this comment

Uh oh!

ninadphalak commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ninadphalak commented Sep 13, 2026 •

edited

Loading

codspeed-hq Bot commented Sep 13, 2026 •

edited

Loading

greptile-apps Bot commented Sep 13, 2026 •

edited

Loading

codecov Bot commented Sep 13, 2026 •

edited

Loading