feat: Add versioning support for FAISS online store#6196
feat: Add versioning support for FAISS online store#6196rpathade wants to merge 1 commit intofeast-dev:masterfrom
Conversation
Signed-off-by: RutujaPathade <73137503+RutujaPathade@users.noreply.github.com>
rpathade
force-pushed
the
feat/faiss_online_store
branch
from
c11eedc to
aa79070
Compare
rpathade
changed the title
rpathade
requested review from
lokeshrangineni,
robhowley and
tokoko
and removed request for
a team
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| from feast.infra.online_stores.faiss_online_store import FaissOnlineStore | ||
| from feast.infra.online_stores.sqlite import SqliteOnlineStore | ||
|
|
||
| if isinstance(self, SqliteOnlineStore): | ||
| if isinstance(self, (SqliteOnlineStore, FaissOnlineStore)): |
There was a problem hiding this comment.
🔴 Importing optional faiss dependency in base OnlineStore breaks all online stores when faiss is not installed
The PR adds from feast.infra.online_stores.faiss_online_store import FaissOnlineStore at online_store.py:259 inside _check_versioned_read_support. This method is called from get_online_features (online_store.py:191) and get_online_features_async (online_store.py:315) for every online store (Redis, DynamoDB, Bigtable, etc.). Since faiss_online_store.py:5 has a top-level import faiss and faiss-cpu is an optional dependency (pyproject.toml:64: faiss = ["faiss-cpu>=1.7.0,<=1.10.0"]), this import will raise ModuleNotFoundError for any user who hasn't installed the faiss extra. The pre-existing SqliteOnlineStore import was safe because sqlite.py only uses stdlib/core imports. The fix should wrap this import in a try/except ImportError and handle the case where FaissOnlineStore is unavailable.
| from feast.infra.online_stores.faiss_online_store import FaissOnlineStore | |
| from feast.infra.online_stores.sqlite import SqliteOnlineStore | |
| if isinstance(self, SqliteOnlineStore): | |
| if isinstance(self, (SqliteOnlineStore, FaissOnlineStore)): | |
| try: | |
| from feast.infra.online_stores.faiss_online_store import FaissOnlineStore | |
| except ImportError: | |
| FaissOnlineStore = None | |
| from feast.infra.online_stores.sqlite import SqliteOnlineStore | |
| supported = [SqliteOnlineStore] | |
| if FaissOnlineStore is not None: | |
| supported.append(FaissOnlineStore) | |
| if isinstance(self, tuple(supported)): |
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
What this PR does / why we need it:
Closes #6173
Feature view versioning was introduced in #6101, but version-qualified feature references (e.g.
driver_stats@v2:trips_today) only worked with the SQLite online store. All other online stores raisedVersionedOnlineReadNotSupportedwhen a versioned ref was used.This PR adds versioned read/write support to the FAISS online store, following the same pattern as the SQLite reference implementation:
enable_online_feature_view_versioningis enabled,_table_idappends a_v{N}suffix to the in-memory key namespace (e.g.project_driver_stats_v2), routing materialization to the correct versioned index@v2) setprojection.version_tagon the feature view beforeonline_readis called;_table_idpicks this up and routes to the correct index.projection.version_tagtakes priority overcurrent_version_numberso explicit version requests are always honoured_index/_in_memory_storewith per-table_indices/_in_memory_storesdicts, so multiple feature views and versions can coexist in memory simultaneously_check_versioned_read_supportinonline_store.pynow allowsFaissOnlineStorethroughAll existing
online_read,online_write_batch,retrieve_online_documents,update, andteardownmethods are covered since every method resolves its namespace through_table_id.Which issue(s) this PR fixes:
Part of #2728
Checks
git commit -s)Testing Strategy
Added 11 unit tests covering
_table_idwith versioning disabled,projection.version_tagpriority overcurrent_version_number, version 0 edge case, no version info fallback, versioned write/read round trips, namespace isolation between versions, missing index handling, and teardown cleanup.Misc
"type"from mock config in tests to matchFaissOnlineStoreConfig's strict pydantic validation (no extra fields permitted)