Repository navigation
Create DocEmbedder class along with RAGRetriever #5426
Description
Currently, the Document Embedder comes separately and outside of Feast for RAG use cases. We are in the process of reviewing and merging the RAGRetriever PR https://github.com/feast-dev/feast/pull/5405 which currently does not explicitly add the Document Embedder as part of the RAGRetriever. Having this functionality within feast will help users using feast for RAG use case as a one stop solution.
As part of this implementation, we should:
Add the document embeddings capacity to feast using HuggingFace doc embedders OR any other embedders.
The RAGRetriever uses the DocEmbedder object for adding it to offline / online store.
The DocEmbedder provides the choice to choose HuggingFace doc embedders or any other.
Metadata
Metadata
Assignees
Labels
No labels
Activity
patelchaitany commentedon Feb 3, 2026
Hey I like to work On this issue.
patelchaitany commentedon Feb 5, 2026
Hey I thought of Implementing this plan and i am looking for any feedback Like is there something which i missed or Which i can do the better.
Implementation Plan:
Document Embedder Class - A high-level orchestrator that coordinates the entire ingestion pipeline.
Chunker - Handles chunking for different types of data. Users can extend or override methods to handle data in their own way.
Logical Layer - Defines how to represent the data in the FeatureView schema. This is written by the Data Scientist or Data Engineer.
Embedding Class - Handles embedding generation. This is designed as a separate class so it can support different modalities (text, images, etc.) by simply extending the base class.
Storage - Write the processed data to both the offline and online store.
Here is the Diagram
I have some concern Regarding this - "The RAGRetriever uses the DocEmbedder object for adding it to offline / online store."
DocEmbedder and FeastRAGRetriever are independent classes. The FeastRAGRetriever provides retrieval functionality at query time. The DocEmbedder is only needed during data ingestion. Keeping them separate maintains clean code and clear responsibilities.
jyejare commentedon Feb 6, 2026
Hey @patelchaitany , Thanks for putting this all together. Initially, when I put this together (and I am open to have any modifications in the architecture), I saw DocEmbedder as a mediator between the raw documents and embeddings. So the mindmap I had in mind for RAGRetriever was like:
Raw Documents -> Chunking -> embeddings -> Offload to Store -> read embeddings based on the query -> Top-K retrievals.
Its like one shot architecture from raw docs to embeddings. But even then, I wanted to keep the Embedder class separate, which would take care of pulling the docs and saving them in the offline store.
So, I think we are on the same page, just only extra thing I had in mind was to plug DocEmbedder with RAGRetreiver as described in mind map.
Let me know your thoughts.
[feat] : feast-dev#5426 added the Docembedder class basic version.
feat: feast-dev#5426 added the Docembedder class basic version.
feat: feast-dev#5426 added the Docembedder class basic version.
feat: feast-dev#5426 added the Docembedder class basic version.
feat: feast-dev#5426 added the Docembedder class basic version.
feat: feast-dev#5426 added the Docembedder class basic version.