[Bug]: a content-filtered turn arrives on /v1/messages as stop_reason end_turn, same as an ordinary answer #40857

Check for existing issues

• I have searched the existing issues and checked that my issue is not a duplicate.

What happened?

A turn the provider blocked on content policy arrives on /v1/messages as "stop_reason": "end_turn", the same value an ordinary completed turn gets. The block does reach the gateway: the identical upstream response on /v1/chat/completions comes back with "finish_reason": "content_filter". It is only the Anthropic surface that loses it.

The mapping from OpenAI finish reasons to Anthropic stop reasons handles stop, length and tool_calls, and everything else falls through to end_turn. content_filter is in that fallthrough, even though refusal is already one of the stop reasons litellm declares for /v1/messages, and the reverse direction already maps Anthropic refusal to OpenAI content_filter. So a refusal that arrives over /v1/messages, is translated to chat completions and translated back, does not survive the round trip.

This is not the same as #39721, which was fixed by #39723. That one covers an upstream that sends an OpenAI refusal string, and the code reaches for the refusal stop reason only when that string is present. Providers whose blocks carry no such string still fall through. litellm's own finish reason map sends Gemini SAFETY, Bedrock guardrail_intervened, Azure content_filtered and several others to content_filter, and none of them set an OpenAI refusal field, so all of them land on end_turn.

Streaming has it too: the final message_delta carries "stop_reason": "end_turn".

Expected: a turn whose finish reason is content_filter arrives on /v1/messages as "stop_reason": "refusal", matching the value the reverse translation already produces and the one #39723 settled on for the refusal-string case.

User Flow

Before a (hypothetical) fix: a developer whose agent framework speaks the Anthropic Messages API cannot tell a blocked answer from an empty one, so the agent retries or reports success on a request the provider refused

1. The proxy admin configures a model whose provider blocks unsafe prompts, and restarts the proxy
2. The developer sends POST https://litellm-domain/v1/chat/completions with a prompt the provider blocks, and gets HTTP 200 with "finish_reason": "content_filter", so their app knows the answer was refused
3. The developer sends the same prompt as POST https://litellm-domain/v1/messages and gets HTTP 200 with "content": [] and "stop_reason": "end_turn"
4. They send an ordinary prompt to the same route and get HTTP 200 with "stop_reason": "end_turn" as well, so the two responses differ only by whether content is empty, which is also what a legitimately empty turn looks like
5. They try "stream": true and the closing message_delta also reads "stop_reason": "end_turn"
6. Their agent loop treats the refusal as a normal turn and either retries the same blocked prompt or records the request as answered

After a (hypothetical) fix: the same requests tell the developer the answer was refused

1. The proxy admin configures the same model and restarts the proxy
2. POST https://litellm-domain/v1/chat/completions returns HTTP 200 with "finish_reason": "content_filter", unchanged
3. POST https://litellm-domain/v1/messages now returns HTTP 200 with "stop_reason": "refusal"
4. An ordinary prompt still returns "stop_reason": "end_turn", so the two are distinguishable
5. With "stream": true the closing message_delta reads "stop_reason": "refusal"
6. Their agent loop sees the refusal and stops instead of retrying

Proof the bug occurs

I have no key for a provider that content-filters, so the deployment points at a local OpenAI-compatible stub that answers with the shape such a provider produces: no content, finish_reason content_filter, and no OpenAI refusal field. Everything from the proxy inwards is real, and the same run shows an ordinary turn through the same routes for comparison. Happy to redo it against a live provider if you would rather see that.

Config the proxy ran with:

model_list:
  - model_name: filtered-model
    litellm_params:
      model: hosted_vllm/filtered-model
      api_key: stub-key-no-provider-call
      api_base: http://127.0.0.1:8899/v1

general_settings:
  master_key: sk-1234

The stub, which answers a filtered turn when the prompt contains "filters" and an ordinary one otherwise:

import json, time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

class H(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_POST(self):
        req = json.loads(self.rfile.read(int(self.headers.get("Content-Length", 0)) or 0) or b"{}")
        filtered = "filters" in json.dumps(req.get("messages", ""))
        reason = "content_filter" if filtered else "stop"
        content = None if filtered else "here you go"

        if not req.get("stream"):
            payload = json.dumps({
                "id": "chatcmpl-stub", "object": "chat.completion", "created": int(time.time()),
                "model": "filtered-model",
                "choices": [{"index": 0, "message": {"role": "assistant", "content": content},
                             "finish_reason": reason}],
                "usage": {"prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12},
            }).encode()
            self.send_response(200)
            self.send_header("Content-Type", "application/json")
            self.send_header("Content-Length", str(len(payload)))
            self.end_headers()
            self.wfile.write(payload)
            return

        self.send_response(200)
        self.send_header("Content-Type", "text/event-stream")
        self.send_header("Connection", "close")
        self.end_headers()
        for delta, finish in (({"role": "assistant", "content": content or ""}, None), ({}, reason)):
            d = {"id": "chatcmpl-stub", "object": "chat.completion.chunk", "created": int(time.time()),
                 "model": "filtered-model", "choices": [{"index": 0, "delta": delta, "finish_reason": finish}]}
            self.wfile.write(f"data: {json.dumps(d)}\n\n".encode())
            self.wfile.flush()
        self.wfile.write(b"data: [DONE]\n\n")
        self.wfile.flush()
        self.close_connection = True

    def log_message(self, *a):
        pass

ThreadingHTTPServer(("127.0.0.1", 8899), H).serve_forever()

Commit: 9a715df (tip of litellm_internal_staging)

H=(-H "Authorization: Bearer sk-1234" -H "Content-Type: application/json")

Case 1, the provider filters the turn.

$ curl -s -w '\nHTTP %{http_code}\n' http://localhost:4000/v1/chat/completions ${H[@]} -d '{"model":"filtered-model","messages":[{"role":"user","content":"something the provider filters"}]}'
{"id":"chatcmpl-stub","created":1789223216,"model":"filtered-model","object":"chat.completion","choices":[{"finish_reason":"content_filter","index":0,"message":{"role":"assistant","content":null},"provider_specific_fields":{}}],"usage":{"completion_tokens":0,"prompt_tokens":12,"total_tokens":12}}
HTTP 200

$ curl -s -w '\nHTTP %{http_code}\n' http://localhost:4000/v1/messages ${H[@]} -d '{"model":"filtered-model","max_tokens":256,"messages":[{"role":"user","content":"something the provider filters"}]}'
{"id":"chatcmpl-stub","type":"message","role":"assistant","model":"filtered-model","stop_sequence":null,"usage":{"input_tokens":12,"output_tokens":0},"content":[],"stop_reason":"end_turn","stop_details":null}
HTTP 200

Case 2, an ordinary answer over the same routes.

$ curl -s -w '\nHTTP %{http_code}\n' http://localhost:4000/v1/chat/completions ${H[@]} -d '{"model":"filtered-model","messages":[{"role":"user","content":"say hello"}]}'
{"id":"chatcmpl-stub","created":1789223216,"model":"filtered-model","object":"chat.completion","choices":[{"finish_reason":"stop","index":0,"message":{"content":"here you go","role":"assistant"},"provider_specific_fields":{}}],"usage":{"completion_tokens":0,"prompt_tokens":12,"total_tokens":12}}
HTTP 200

$ curl -s -w '\nHTTP %{http_code}\n' http://localhost:4000/v1/messages ${H[@]} -d '{"model":"filtered-model","max_tokens":256,"messages":[{"role":"user","content":"say hello"}]}'
{"id":"chatcmpl-stub","type":"message","role":"assistant","model":"filtered-model","stop_sequence":null,"usage":{"input_tokens":12,"output_tokens":0},"content":[{"type":"text","text":"here you go"}],"stop_reason":"end_turn","stop_details":null}
HTTP 200

The blocked turn and the answered turn carry the same stop_reason.

Case 3, the same filtered turn with "stream": true.

$ curl -sN -w '\nHTTP %{http_code}\n' http://localhost:4000/v1/messages ${H[@]} -d '{"model":"filtered-model","max_tokens":256,"stream":true,"messages":[{"role":"user","content":"something the provider filters"}]}'
event: message_start
data: {"type": "message_start", "message": {"id": "msg_3846e5f1-8cc6-4ea3-b61a-ef755ffd4bcc", "type": "message", "role": "assistant", "content": [], "model": "filtered-model", "stop_reason": null, "stop_sequence": null, "usage": {"input_tokens": 0, "output_tokens": 0, "cache_creation_input_tokens": 0, "cache_read_input_tokens": 0}}}

event: message_delta
data: {"type": "message_delta", "delta": {"stop_reason": "end_turn"}, "usage": {"input_tokens": 11, "output_tokens": 0}}

HTTP 200

What part of LiteLLM is this about?

Proxy

What LiteLLM version are you on ?

9a715df

Twitter / LinkedIn details

No response