Skip to content

feat(models): add llama.cpp local model configuration example#2960

Open
YuanyuanMa03 wants to merge 1 commit into
bytedance:mainfrom
YuanyuanMa03:feat/add-llamacpp-model-config
Open

feat(models): add llama.cpp local model configuration example#2960
YuanyuanMa03 wants to merge 1 commit into
bytedance:mainfrom
YuanyuanMa03:feat/add-llamacpp-model-config

Conversation

@YuanyuanMa03
Copy link
Copy Markdown
Contributor

Summary

Add llama.cpp local LLM inference configuration example to config.example.yaml

What

Added commented configuration example for llama.cpp local model server:

  • Uses OpenAI-compatible API via langchain_openai:ChatOpenAI
  • Default port: 8080 (llama.cpp standard)
  • Example uses Qwen3.6 35B model as referenced in the issue

Why

Issue #2931 requests configuration example for llama.cpp local deployment. Users deploying local models via llama.cpp can now easily configure DeerFlow without manually writing configuration.

How

Configuration follows existing OpenAI-compatible pattern:

  • base_url: http://localhost:8080/v1
  • api_key is optional (commented with explanation)
  • Includes Docker deployment note (host.docker.internal)
  • Documents server startup command and advanced options

Test plan

  • Configuration follows existing patterns in config.example.yaml
  • All required fields documented (name, display_name, use, model, base_url)
  • api_key handling properly explained
  • Docker deployment note included

Checklist

  • Code follows project style guidelines
  • Documentation updated (config.example.yaml)

Fixes #2931

Add configuration example for llama.cpp local LLM inference:
- Uses OpenAI-compatible API via langchain_openai:ChatOpenAI
- Default port: 8080, with Docker host.docker.internal note
- Example uses Qwen3.6 35B model as referenced in bytedance#2931
- Includes server startup command and advanced options
- api_key is optional but may be required by langchain_openai

Fixes bytedance#2931

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[runtime] 如何在config.yaml中配置LLama.cpp本地部署的qwen3.6-35b-a3b模型

1 participant