@uinstinct if I understand the problem correctly, this happens when the tool call state ends up in an invalid place because the model hits max tokens or similar. This fix would only work because we use incremental JSON parsing to create parsedArgs. But I think if it ends up invalid we should throw an error, since e.g. the file being created will be truncated

I think in that case it should throw an error that the tool call args were invalid, basically a try again with smaller/more targeted output message

Yes, the error now shows the correct missing argument.

Although I think resolving the error would require increasing max tokens because the agent does not send the contents argument in any case. Should we modify the createnewfile tool to support creating files without contents? (and then maybe edittool can be used to add contents to the new file)

Although I think resolving the error would require increasing max tokens because the agent does not send the contents argument in any case. Should we modify the createnewfile tool to support creating files without contents? (and then maybe edittool can be used to add contents to the new file)

This makes sense. Since

• agree that it doesn't make sense to increase max tokens dynamically
• probably won't be reliable to create empty and then edit since feels like unexpected behavior and edit tool will have same issue unless model is aware why it happened

I think a good solution is to throw an error instructing the model to try again with smaller output.

If we take that route I guess the main question is how do we know if this was caused by max token limit being reached?

Here are 2 ideas

1. I think the best indicator of max tokens being the culprit is if the json in arguments is invalid. This is why the current approach of using parsedArgs (which incrementally parses) at least fixes the issue of saying the wrong arg is missing, but doesn't fix any underlying issue.

I think the best solution is here in callTool.ts

const args = safeParseToolCallArgs(toolCall);

We should not safe parse args here, we should detect invalid JSON and throw a sensible error which explains that this often happens when huge args are passed, try smaller ones.

2. We could also just add a note to getStringArg which says hey sometimes this happens when you tried to pass huge args, try a smaller value. This would probably fix most cases but feels less robust

Thoughts on just doing the first approach for now?

something to keep in mind is that the incremental args parsing is just so that we can display tool calls as they stream in, so all that should stay in place, but if they end up invalid it makes sense to throw

1. I think the best indicator of max tokens being the culprit is if the json in arguments is invalid. This is why the current approach of using parsedArgs (which incrementally parses) at least fixes the issue of saying the wrong arg is missing, but doesn't fix any underlying issue.

I think the best solution is here in callTool.ts

const args = safeParseToolCallArgs(toolCall);

We should not safe parse args here, we should detect invalid JSON and throw a sensible error which explains that this often happens when huge args are passed, try smaller ones.

Tried this now. In this approach, in most of the cases, the arguments from the model are valid JSON but have the missing parameter - which can only be detected during tool call. Hence, I think throwing errors while parsing would not be useful.

2. We could also just add a note to getStringArg which says hey sometimes this happens when you tried to pass huge args, try a smaller value. This would probably fix most cases but feels less robust

In this case, I think we should only narrow it down to a specific models where there is a chance of huge args being sent to the tool call. Otherwise, if the tool call argument is missing, the model should avoid retrying and add that first.