Malformed gemini request with tool call #4867
Replies: 3 comments
-
|
Hi @quangIO I think you're correct this is an issue (and bad error handling) on our end. We currently set the function calling mode based on the tool choice only and didn't consider this relationship between the two settings. I will cut a PR fixing this ~tomorrow and think about how we might give better errors in the general case. Thanks for reporting! |
Beta Was this translation helpful? Give feedback.
-
|
@quangIO #4875 should fix the immediate issue here. Will also take a look at error handling |
Beta Was this translation helpful? Give feedback.
-
|
#5772 should improve the quality of streaming errors in general. Please reopen if you still see bad/uninformative errors out there! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Using AI Studio with explicitly specified allowed tools, Tensorzero produces the following request body:
Request with Playwright MCP
{ "contents": [ { "role": "user", "parts": [ { "thought": false, "thoughtSignature": null, "text": "Hi" } ] } ], "tools": [ { "functionDeclarations": [ { "name": "browser_close", "description": "Close the page", "parameters": { "type": "object", "properties": { "args": { "properties": {}, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_resize", "description": "Resize the browser window", "parameters": { "type": "object", "properties": { "args": { "required": [ "width", "height" ], "type": "object", "properties": { "width": { "description": "Width of the browser window", "type": "number" }, "height": { "description": "Height of the browser window", "type": "number" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_console_messages", "description": "Returns all console messages", "parameters": { "type": "object", "properties": { "args": { "properties": { "onlyErrors": { "description": "Only return error messages", "type": "boolean" } }, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_handle_dialog", "description": "Handle a dialog", "parameters": { "type": "object", "properties": { "args": { "required": [ "accept" ], "type": "object", "properties": { "accept": { "description": "Whether to accept the dialog.", "type": "boolean" }, "promptText": { "description": "The text of the prompt in case of a prompt dialog.", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_evaluate", "description": "Evaluate JavaScript expression on page or element", "parameters": { "type": "object", "properties": { "args": { "required": [ "function" ], "type": "object", "properties": { "function": { "description": "() => { /* code */ } or (element) => { /* code */ } when element is provided", "type": "string" }, "element": { "description": "Human-readable element description used to obtain permission to interact with the element", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_file_upload", "description": "Upload one or multiple files", "parameters": { "type": "object", "properties": { "args": { "properties": { "paths": { "description": "The absolute paths to the files to upload. Can be single file or multiple files. If omitted, file chooser is cancelled.", "type": "array", "items": { "type": "string" } } }, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_fill_form", "description": "Fill multiple form fields", "parameters": { "type": "object", "properties": { "args": { "required": [ "fields" ], "type": "object", "properties": { "fields": { "description": "Fields to fill in", "type": "array", "items": { "type": "object", "properties": { "name": { "description": "Human-readable field name", "type": "string" }, "type": { "description": "Type of the field", "type": "string", "enum": [ "textbox", "checkbox", "radio", "combobox", "slider" ] }, "ref": { "description": "Exact target field reference from the page snapshot", "type": "string" }, "value": { "description": "Value to fill in the field. If the field is a checkbox, the value should be `true` or `false`. If the field is a combobox, the value should be the text of the option.", "type": "string" } }, "required": [ "name", "type", "ref", "value" ] } } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_install", "description": "Install the browser specified in the config. Call this if you get an error about the browser not being installed.", "parameters": { "type": "object", "properties": { "args": { "properties": {}, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_press_key", "description": "Press a key on the keyboard", "parameters": { "type": "object", "properties": { "args": { "required": [ "key" ], "type": "object", "properties": { "key": { "description": "Name of the key to press or a character to generate, such as `ArrowLeft` or `a`", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_type", "description": "Type text into editable element", "parameters": { "type": "object", "properties": { "args": { "required": [ "element", "ref", "text" ], "type": "object", "properties": { "element": { "description": "Human-readable element description used to obtain permission to interact with the element", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot", "type": "string" }, "text": { "description": "Text to type into the element", "type": "string" }, "submit": { "description": "Whether to submit entered text (press Enter after)", "type": "boolean" }, "slowly": { "description": "Whether to type one character at a time. Useful for triggering key handlers in the page. By default entire text is filled in at once.", "type": "boolean" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_navigate", "description": "Navigate to a URL", "parameters": { "type": "object", "properties": { "args": { "required": [ "url" ], "type": "object", "properties": { "url": { "description": "The URL to navigate to", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_navigate_back", "description": "Go back to the previous page", "parameters": { "type": "object", "properties": { "args": { "properties": {}, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_network_requests", "description": "Returns all network requests since loading the page", "parameters": { "type": "object", "properties": { "args": { "properties": {}, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_run_code", "description": "Run Playwright code snippet", "parameters": { "type": "object", "properties": { "args": { "required": [ "code" ], "type": "object", "properties": { "code": { "description": "Playwright code snippet to run. The snippet should access the `page` object to interact with the page. Can make multiple statements. For example: `await page.getByRole('button', { name: 'Submit' }).click();`", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_take_screenshot", "description": "Take a screenshot of the current page. You can't perform actions based on the screenshot, use browser_snapshot for actions.", "parameters": { "type": "object", "properties": { "args": { "properties": { "type": { "description": "Image format for the screenshot. Default is png.", "type": "string", "enum": [ "png", "jpeg" ], "default": "png" }, "filename": { "description": "File name to save the screenshot to. Defaults to `page-{timestamp}.{png|jpeg}` if not specified. Prefer relative file names to stay within the output directory.", "type": "string" }, "element": { "description": "Human-readable element description used to obtain permission to screenshot the element. If not provided, the screenshot will be taken of viewport. If element is provided, ref must be provided too.", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot. If not provided, the screenshot will be taken of viewport. If ref is provided, element must be provided too.", "type": "string" }, "fullPage": { "description": "When true, takes a screenshot of the full scrollable page, instead of the currently visible viewport. Cannot be used with element screenshots.", "type": "boolean" } }, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_snapshot", "description": "Capture accessibility snapshot of the current page, this is better than screenshot", "parameters": { "type": "object", "properties": { "args": { "properties": {}, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_click", "description": "Perform click on a web page", "parameters": { "type": "object", "properties": { "args": { "required": [ "element", "ref" ], "type": "object", "properties": { "element": { "description": "Human-readable element description used to obtain permission to interact with the element", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot", "type": "string" }, "doubleClick": { "description": "Whether to perform a double click instead of a single click", "type": "boolean" }, "button": { "description": "Button to click, defaults to left", "type": "string", "enum": [ "left", "right", "middle" ] }, "modifiers": { "description": "Modifier keys to press", "type": "array", "items": { "type": "string", "enum": [ "Alt", "Control", "ControlOrMeta", "Meta", "Shift" ] } } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_drag", "description": "Perform drag and drop between two elements", "parameters": { "type": "object", "properties": { "args": { "required": [ "startElement", "startRef", "endElement", "endRef" ], "type": "object", "properties": { "startElement": { "description": "Human-readable source element description used to obtain the permission to interact with the element", "type": "string" }, "startRef": { "description": "Exact source element reference from the page snapshot", "type": "string" }, "endElement": { "description": "Human-readable target element description used to obtain the permission to interact with the element", "type": "string" }, "endRef": { "description": "Exact target element reference from the page snapshot", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_hover", "description": "Hover over element on page", "parameters": { "type": "object", "properties": { "args": { "required": [ "element", "ref" ], "type": "object", "properties": { "element": { "description": "Human-readable element description used to obtain permission to interact with the element", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot", "type": "string" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_select_option", "description": "Select an option in a dropdown", "parameters": { "type": "object", "properties": { "args": { "required": [ "element", "ref", "values" ], "type": "object", "properties": { "element": { "description": "Human-readable element description used to obtain permission to interact with the element", "type": "string" }, "ref": { "description": "Exact target element reference from the page snapshot", "type": "string" }, "values": { "description": "Array of values to select in the dropdown. This can be a single value or multiple values.", "type": "array", "items": { "type": "string" } } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_tabs", "description": "List, create, close, or select a browser tab.", "parameters": { "type": "object", "properties": { "args": { "required": [ "action" ], "type": "object", "properties": { "action": { "description": "Operation to perform", "type": "string", "enum": [ "list", "new", "close", "select" ] }, "index": { "description": "Tab index, used for close/select. If omitted for close, current tab is closed.", "type": "number" } } }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } }, { "name": "browser_wait_for", "description": "Wait for text to appear or disappear or a specified time to pass", "parameters": { "type": "object", "properties": { "args": { "properties": { "time": { "description": "The time to wait in seconds", "type": "number" }, "text": { "description": "The text to wait for", "type": "string" }, "textGone": { "description": "The text to wait for to disappear", "type": "string" } }, "type": "object" }, "metadata": { "properties": { "reason": { "description": "Very short and concise reason why you are calling this tool with arguments without repeating\nuser's request", "type": "string", "default": "" }, "is_dangerous": { "description": "Is calling this function dangerous? e.g. file write, network request to local network,...", "type": "boolean", "default": false } }, "type": "object" } }, "required": [ "args", "metadata" ] } } ] } ], "toolConfig": { "functionCallingConfig": { "mode": "auto", "allowedFunctionNames": [ "browser_close", "browser_resize", "browser_console_messages", "browser_handle_dialog", "browser_evaluate", "browser_file_upload", "browser_fill_form", "browser_install", "browser_press_key", "browser_type", "browser_navigate", "browser_navigate_back", "browser_network_requests", "browser_run_code", "browser_take_screenshot", "browser_snapshot", "browser_click", "browser_drag", "browser_hover", "browser_select_option", "browser_tabs", "browser_wait_for" ] } }, "generationConfig": {}, "systemInstruction": { "role": "model", "parts": [ { "thought": false, "thoughtSignature": null, "text": "You listen to user request" } ] } }Save the above request to data.json and send it to AI Studio:
We received error:
{ "error": { "code": 400, "message": "Please set allowed_function_names only when function calling mode is ANY.", "status": "INVALID_ARGUMENT" } }Change auto to any indeed fixed this issue.
The main issue here seems to be that we are auto instead of any in functionCallingConfig. Furthermore, we should deliver error messages / provide better logging information for users.
From the API, we are only seeing information like this, where we only know about 400 Bad Requests, but we don't know why.
Beta Was this translation helpful? Give feedback.
All reactions