feat: Add Chronon online and offline store integrations#6188
Draft
franciscojavierarceo wants to merge 4 commits into
Draft
feat: Add Chronon online and offline store integrations#6188franciscojavierarceo wants to merge 4 commits into
franciscojavierarceo wants to merge 4 commits into
Conversation
b15df79 to
47fdba8
Compare
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
47fdba8 to
a68a4ef
Compare
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds an in-tree Chronon bridge for Feast. Chronon remains the owner of feature computation, materialization, and online serving; Feast provides registry/source/store integration for historical retrieval and online reads.
The integration includes:
ChrononSourcefor Chronon materialization metadata, Join/GroupBy routing metadata, and source validationChrononOfflineStorefor reading Chronon-produced Parquet materializations with point-in-time historical retrievalChrononOnlineStoreforget_online_featuresreads through Chronon's feature serviceChrononProviderregistration as a passthrough provider alias for Chronon-backed reposexamples/chronondemo covering offline historical retrieval and online reads against Chronon's quickstart serviceNotable details
This keeps the Feast/Chronon boundary intentionally narrow:
created_timestamp_columnused to choose the latest duplicate event timestamp.airbnb/chrononcheckout instead of relying on a nonexistent submodule or top-levelchronon/directory.Testing
Ran locally:
sbt, checked outairbnb/chrononat6c0b8de9f0301521baf61a46ff3083c566fb4052, built the quickstart Mongo implementation jar, and built the Chronon service jar with Java 11 and Thrift 0.13.CHRONON_REPO=/Users/farceo/.codex/worktrees/ecbd/chronon JAVA_BIN=/opt/homebrew/opt/openjdk@11/bin/java CHRONON_SERVICE_PORT=19000 infra/scripts/chronon/start-local-chronon-service.sh(CHRONON_SERVICE_URL=http://127.0.0.1:19000; Chronon quickstart fetch returnedpurchase_price_sum_30d=1253andrefund_amt_sum_30d=1269)CHRONON_SERVICE_URL=http://127.0.0.1:19000 uv run python examples/chronon/run_demo.py(offline demo returnedcompleted_rides_7d=[18, 7]; online demo returned1253and1269)CHRONON_SERVICE_URL=http://127.0.0.1:19000 uv run pytest -c sdk/python/pytest.ini sdk/python/tests/integration/online_store/test_chronon_online_store_real_service.py --integration -q(1 passed)uv run pytest -c sdk/python/pytest.ini sdk/python/tests/unit/infra/online_stores/chronon_online_store sdk/python/tests/unit/infra/offline_stores/contrib/chronon_offline_store sdk/python/tests/integration/online_store/test_chronon_online_store.py sdk/python/tests/integration/offline_store/test_chronon_offline_store.py --integration -q(7 passed)uv run ruff format infra/scripts/chronon/chronon_service_launcher.py examples/chronon/run_demo.pyuv run ruff check infra/scripts/chronon/chronon_service_launcher.py examples/chronon/run_demo.pybash -n infra/scripts/chronon/start-local-chronon-service.sh infra/scripts/chronon/stop-local-chronon-service.shuv run python examples/chronon/run_demo.py --offline-onlyuv run pre-commit run detect-secrets --files examples/chronon/README.md infra/scripts/chronon/start-local-chronon-service.sh .github/workflows/pr_chronon_integration_tests.ymlgit diff --checkThe live Chronon stack was stopped after verification, and cleanup left no Chronon Java process, listener, or Docker containers running.
Follow-ups
Open questions for follow-up PRs: