You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[tools.extract_sections]description = "Extracts meaningful sections each with title and starting line index from a document."parameters = "tools/extract_sections.json"strict = true# FUNCTIONS[functions.segment_sections]type = "chat"tools = ["extract_sections"][functions.segment_sections.variants.gpt_4o_mini]type = "chat_completion"model = "openai::gpt-4o-mini"
With caching enabled, sending the same request twice, the calls are still sent to openAI. I checked the usage counter on openAI dashboard, it's still incrementing. Shouldn't tensorzero cache the inference and return the stored inference response? Attached is the screenshot of the second request. Would appreciate if you guys could help me out.
Above is a cached response from openAI, not tensorzero.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi team, appreciate the work that has been put in Tensorzero. Posting here, since am not sure if this a bug.
Using TensorZero Gateway
2025.9.3Below is my code:
My config:
With caching enabled, sending the same request twice, the calls are still sent to
openAI. I checked the usage counter on openAI dashboard, it's still incrementing. Shouldn't tensorzero cache the inference and return the stored inference response? Attached is the screenshot of the second request. Would appreciate if you guys could help me out.Above is a cached response from openAI, not tensorzero.
Beta Was this translation helpful? Give feedback.
All reactions