Skip to content

function_calling

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

function_calling

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
Outputs.md
Outputs.md
 
 
README.md
README.md
 
 
function_calling_nexusraven.py
function_calling_nexusraven.py
 
 
function_calling_test.py
function_calling_test.py
 
 

README.md

Function Calling

Testing Function Calling with local LLMs using LiteLLM and Ollama

How to run

IMPORTANT: Ensure you are using Ollama v0.1.29 or above (earlier models may hang with Mistral-based models)

  1. Install AutoGen (preferably in a virtual environment / dev container)

  2. Install LiteLLM and Ollama

  3. Ollama pull <model name> your models, suggest trying DolphinCoder (e.g. Ollama pull dolphincoder)

  4. Run LiteLLM using the ollama_chat notation:

    litellm --model ollama_chat/dolphincoder --port 8801 --debug

  5. Open, function_calling_test.py, change network_address variable and comment/uncomment under llm_config accordingly

  6. Run it!