Configurable embedding function (custom / OpenAI-compatible endpoint) #3030
Replies: 1 comment
|
Another real-world use case, plus verified constraints from digging into this today on v13.24.5. Use case: CJK users get weak recall from both retrieval paths. My corpus is mostly Chinese. MiniLM-L6-v2 is English-centric, so the semantic path contributes little — and the FTS path is also broken for CJK in the server runtime right now (#3859). For non-English users, a configurable EF is the difference between search working and search being keyword-luck. Verified constraints (for whoever picks this up):
One addition to the proposal: beyond API providers (OpenAI-compatible endpoints), please also consider local embedding models — e.g. Happy to test any preview build on a mostly-Chinese corpus (~900 observations). |
Uh oh!
There was an error while loading. Please reload this page.
Problem
Current behavior
Two layers block customization:
Proposed solution
Add env-driven embedding config, e.g.:
When set, the worker should: add the provider package to the uvx --with list, pass embedding_function_name to chroma_create_collection, and ensure the same EF (with base/model/dims) is applied on read paths.
Workaround (current)
Patching worker-service.cjs (--with openai + --dotenv-path) and chroma-mcp's server.py (force the OpenAI EF on all collection ops). Works but is lost on every plugin update and uv cache clean.
Benefit
Much better multilingual + higher-quality recall, opt-in, no breaking change to the ONNX default.
All reactions