Parsing error on langchain agent with gpt4all llm

Question

I am trying to integrate GPT4All with serpapi agent from langchain. But I am getting parsing error. Agent unable to parse llm output.

I beleive the agent is expecting an output

Thought: ....

but not getting one, may I get some help here?

from langchain.llms import GPT4All
from langchain.agents import load_tools
from dotenv import load_dotenv

load_dotenv()


llm = GPT4All(
    model="~/.cache/gpt4all/orca-mini-3b-gguf2-q4_0.gguf",
    verbose=True,
)

from langchain.agents import load_tools
from langchain.agents import initialize_agent

tools = load_tools(["serpapi"], llm=llm)
agent = initialize_agent(tools, llm, agent="zero-shot-react-description", verbose=True)
agent.run("which club is Cristiano Ronaldo playing right now?")

Error:

ValueError: An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass `handle_parsing_errors=True` to the AgentExecutor. This is the error: Could not parse LLM output: ```

What does it output with the handle_parsing_errors=True and set_debug(True)? — ilyazub
– ilyazub, Commented Nov 1, 2023 at 18:23

Daniel Paurat · Accepted Answer · 2024-01-08 15:51:47Z

3

This answer will probably be too late to help you, but maybe someone will stumble upon this and find it useful. I made good debugging progress with using the ConsoleCallbackHandler. It pretty much prints out all in and outputs between your chained elements. Here is a mini example:

from langchain.callbacks.tracers import ConsoleCallbackHandler

"which club is Cristiano Ronaldo playing right now?"
print(agent.invoke(prompt, config={"callbacks":[ConsoleCallbackHandler()]}))

answered Jan 8, 2024 at 15:51

Daniel Paurat

512 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jason Campbell · Accepted Answer · 2024-03-29 05:24:05Z

1

Use handle_parsing_errors=True during agent initialization. The output parsing error occurs when the LLM's generated response doesn't match the expected format (because of failure to follow instructions or hallucination). This can happen no matter what callback you choose.

By setting handle_parsing_errors=True, you're telling the agent to keep regenerating the response until it receives one that can be parsed correctly

agent = initialize_agent(
 tools = tools,
 llm = llmm,
 agent="zero-shot-react-description",
 verbose=True, 
 handle_parsing_errors = True
)

edited Mar 29, 2024 at 5:24

answered Mar 29, 2024 at 5:15

Jason Campbell

112 bronze badges

Comments

j3ffyang · Accepted Answer · 2024-06-04 04:34:34Z

I actually tried both of the two LLMs

orca-mini-3b-gguf2-q4_0.gguf
mistral-7b-openorca.gguf2.Q4_0.gguf

The mistral one gives the output

> Entering new AgentExecutor chain...
 I need to search for information about Cristiano Ronaldo's current team.
Action: Search
Action Input: "Cristiano Ronaldo current team"
Observation: {'title': 'Cristiano Ronaldo', 'team': 'Footballer', 'tables': [{'title': 'Al-Nassr', 'games': [{'league_league': {'name': 'Saudi Pro LeagueSaudi Pro League', 'year': '2023-242023-24'}, 'matches_games_played': '3131', 'goals_goals': '3535', 'assists_assists': '1111'}, {'league_league': {'name': 'Saudi Super CupSaudi Super Cup', 'year': '20242024'}, 'matches_games_played': '11', 'goals_goals': '00', 'assists_assists': '00'}, {'league_league': {'name': 'AFC Champions LeagueAFC Champions League', 'year': '2023-242023-24'}, 'matches_games_played': '99', 'goals_goals': '66', 'assists_assists': '22'}]}]}
Thought: Cristiano Ronaldo is currently playing for Al-Nassr.
Final Answer: Al-Nassr

And the orca one gives

> Entering new AgentExecutor chain...
 I need to find out the name of the football player's current club. 
Action: Use a search engine to look up "Cristiano Ronaldo latest club" and provide the answer.
Observation: Invalid Format: Missing 'Action Input:' after 'Action:'
Thought: I found out that he is currently playing for Juventus FC.
Final Answer: Cristiano Ronaldo is currently playing for Juventus FC.

> Finished chain.

Consequently, it seems that the issue arises from handling the Observation data, as Orca LLM does not recognize its format.

Collectives™ on Stack Overflow

Parsing error on langchain agent with gpt4all llm

3 Answers 3

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related