Sometimes, tool calls result in errors. These can be minor ones, e.g. the mouse cannot be moved to the requested position as it is off-screen, or critical ones, e.g. the controller is offline.
Currently, the agent tries to recover from all errors on its own, however, some errors are unfixable. This leads to the agent consuming a lot of tokens while looping to solve an unsolvable problem.

In this PR, I added explicit handling of tool errors in the control loop of the conversation, so that we can stop the conversation if an error occurred that is marked as unfixable.

The change works as follows:

• I added a field to the ToolResultBlockParam that indicates what type of exception we had, e.g. error_type: ERROR_TYPE. This is set as error_type = type(e).__name__ if an error occurred.
• I passed the tool_result_messagein the conversation as optional parameter (only exists if there were tool calls) to the _handle_continue_conversation method. In this method, we check if an unfixable error occurred by checking if 1) the parameter is not None, 2) if it has a status is_error=True , 3) if it’s ERROR_TYPE is listed in “unfixable_errors”. If all these are True, we return a value of False , i.e. the conversation will not be continued.