@franciscojavierarceo

FAILED sdk/python/tests/unit/test_on_demand_python_transformation.py::TestOnDemandPythonTransformation::test_stored_writes - AssertionError: assert ['counter', '...put_datetime'] == ['conv_rate_p...put_datetime']
At index 0 diff: 'counter' != 'conv_rate_plus_acc'
Right contains 2 more items, first extra item: 'driver_id'
Full diff:
[

• 'conv_rate_plus_acc',
  'counter',
• 'current_datetime',
  'driver_id',
  'input_datetime',
  ]
  FAILED sdk/python/tests/unit/online_store/test_online_retrieval.py::test_milvus_stored_writes_with_explode - KeyError: 'Feature document_id not found in projection milvus_explode_feature_view'

lots of unit tests failing. Will need to check :/

So maybe I misunderstood, but ODFVs should be queryable from offline store. They should either execute the transformation at query time or be precomputed and stored already.

First, here's what the docs say:

https://docs.feast.dev/reference/beta-on-demand-feature-view

By setting write_to_online_store=True, transformations are applied during data ingestion, and the transformed features are stored in the online store. This can improve online feature retrieval performance by reducing computation during reads. Conversely, if write_to_online_store=False (the default if omitted), transformations are applied during feature retrieval.

Here's my thinking and experience using this new feature in prod. You call store.materialize an an ODFV, and it does get stored in online store (tested this ourselves).

ODFVs are precomputed and stored within the online store if write_to_online_store=True, or computed on request time otherwise. At least that's how their materialization is implemented within Feast code (we tested it in prod!). Offline stores are populated by offline jobs, outside of Feast e.g. Clickhouse, Spark, whatnot, which get_historical_features merely then slices off features from to materialize into online store e.g. materialize from Clickhouse into Redis.

@franciscojavierarceo write_to_online_store flag that enables materializing (precomputing) ODFVs has online, not offline store, within its semantical definition as well.

Lastly:

the logic I propose to change in the PR above within get_historical_features from an offline store:

elif view.projection and view.write_to_online_store:

has nothing to do with offline stores, but rather with online stores - it seems flawed to call it in offline_store.get_historical_features

sorry long text, but got to explain :D The test is missing materialize step, which loads ODFV values into online store, to be queried (think, cached).

Here's the gist. Hope it makes sense.

@franciscojavierarceo

@franciscojavierarceo Closing PR. Will open up a related fix only targeting get_historical_features, need to re-think this.