There is a breaking bug in the functionary_v1_v2_handler such that it is not able to handle some chat requests as mentioned here. This is a result of wrong logic in the handler.

The following optimizations were done:

• Fixed the wrong logic (certain if-else statements were wrong)
• Changed the logic when tool_choice="auto". In a while loop,
  1. Generate function name first
  2. Change the grammar according depending on whether the model chooses a function or not and generate the content
  3. Check whether the model wants to generate another turn by checking for the presence of \n<|from|>assistant
  4. Break out of the loop if stop token is generated

The second optimization allows for both text response and tool calls to be filled in 1 assistant turn.