Skip to content

Commit 129097d

Browse files
JunyiXu-nvcodego7250
authored andcommitted
[TRTLLM-6842][feat] Support Response API for general purpose (NVIDIA#9392)
Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
1 parent ebed71d commit 129097d

File tree

2 files changed

+1
-5
lines changed

2 files changed

+1
-5
lines changed

tensorrt_llm/serve/openai_server.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -994,7 +994,6 @@ async def create_stream_response(generator, request: ResponsesRequest, sampling_
994994
use_harmony=self.use_harmony,
995995
reasoning_parser=self.llm.args.reasoning_parser,
996996
tool_parser=self.tool_parser)
997-
998997
except asyncio.CancelledError:
999998
if promise is not None:
1000999
promise.abort()

tensorrt_llm/serve/responses_utils.py

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -60,16 +60,13 @@
6060
StreamingResponsesResponse,
6161
UCompletionRequest,
6262
UCompletionResponse)
63+
6364
from tensorrt_llm.serve.tool_parser.base_tool_parser import BaseToolParser
6465
from tensorrt_llm.serve.tool_parser.core_types import ToolCallItem
6566
from tensorrt_llm.serve.tool_parser.tool_parser_factory import ToolParserFactory
6667

6768
from .harmony_adapter import HarmonyAdapter, get_harmony_adapter
6869

69-
# yapf: enable
70-
71-
# yapf: enable
72-
7370
REASONING_EFFORT = {
7471
"high": ReasoningEffort.HIGH,
7572
"medium": ReasoningEffort.MEDIUM,

0 commit comments

Comments
 (0)