feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features by addenergyx · Pull Request #6617 · feast-dev/feast

addenergyx · 2026-07-19T17:46:22Z

What this PR does / why we need it

Adds an opt-in filter_by_created_timestamp: bool = False parameter to get_historical_features that enforces created_timestamp <= entity_timestamp in point-in-time joins.

Today created_timestamp_column is used only as a dedup tiebreaker, never as a filter, so a training row at entity timestamp T can be served a value that did not exist yet at T (backfills, late corrections). With filter_by_created_timestamp=True, retrieval reflects what was actually known as of each entity row's timestamp, i.e. what the online store would have served at that moment.

store.get_historical_features(
    entity_df=entity_df,
    features=features,
    filter_by_created_timestamp=True,  # default False: existing behavior
)

Design

Threaded through FeatureStore.get_historical_features using the same optional-kwargs pattern as start_date/end_date: only passed to the offline store when True, so the default path is untouched and third-party stores are unaffected unless a user explicitly opts in.
For SQL-template stores, the shared build_point_in_time_query gains a filter_by_created_timestamp template variable, and each template's __base join gains one guarded predicate:
```
{% if filter_by_created_timestamp and featureview.created_timestamp_column %}
AND subquery.created_timestamp <= entity_dataframe.entity_timestamp
{% endif %}
```
Everything else (event-timestamp window, TTL, dedup, ROW_NUMBER) is unchanged.
Feature views without a created_timestamp_column are unaffected.

Store coverage

Implemented: BigQuery, Redshift, Spark, Trino, Athena, Postgres, ClickHouse, Couchbase (SQL templates); DuckDB, MSSQL, Oracle (via the shared ibis point_in_time_join); Dask/file (pandas filter mirroring _filter_ttl); Hybrid (delegates, re-checking the resolved child store).

Stores declare support via a supports_filter_by_created_timestamp class attribute on OfflineStore (default False). A single guard in the passthrough provider raises NotImplementedError when the flag is set for a store that does not declare support, so no store can silently ignore it and new stores are covered automatically. Unsupported today: Snowflake (its ASOF JOIN dedups before the join and MATCH_CONDITION takes a single predicate, so the cutoff cannot be expressed without restructuring the query), Remote (needs protocol support to forward the flag), Ray, and MongoDB.

Notes

Rows with a NULL created_timestamp are excluded by the predicate in all implementations; the docstring notes the column should be non-null when the mode is used. (In the dask store, unmatched entity rows from the left join are still preserved.)
In the dask store, an entity row whose matching feature rows are all filtered out disappears from the result instead of surviving with null features. This matches the store's existing TTL filtering behavior and is a pre-existing trait of its incremental join, not something this PR changes.
Orthogonal to ttl: TTL bounds by event time, this bounds by known time.

Which issue(s) this PR fixes

Fixes #6615

Misc

Tests added in sdk/python/tests/unit/infra/offline_stores/test_filter_by_created_timestamp.py:

Template rendering: predicate present iff filter_by_created_timestamp=True and a created_timestamp_column is set; default rendering unchanged.
Dask and ibis (DuckDB) end-to-end: same event timestamp with two versions created before/after the entity timestamp; default serves the later-created value, filter_by_created_timestamp=True serves the version known at the entity timestamp.

ruff check/ruff format clean; mypy feast reports the same 17 pre-existing errors as master (none introduced).

…ical_features When a feature view has a created_timestamp_column, it is currently used only as a dedup tiebreaker in point-in-time joins, so retrieval can serve feature values whose created_timestamp is after the entity row's event timestamp (backfills, late corrections). This leaks future information into training data and makes training sets non-reproducible. Add an opt-in at_event_time flag (default False) to get_historical_features that adds a created_timestamp <= entity_timestamp predicate to the point-in-time join, so retrieval reflects what was known as of each entity row's timestamp. Implemented for the SQL template stores (BigQuery, Redshift, Spark, Trino, Athena, Postgres, ClickHouse, Couchbase), the ibis-based stores (DuckDB, MSSQL, Oracle) and the dask store. Snowflake, Remote and Ray raise NotImplementedError when the flag is set. Fixes feast-dev#6615 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

Adds a universal offline store integration test covering at_event_time (default returns backfilled values, at_event_time=True only returns values created at or before the entity timestamp, stores without support skip via NotImplementedError) and documents the flag on the point-in-time joins concept page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

…ress review feedback Rename the flag to filter_by_created_timestamp to match the repo's mechanical flag naming and sit alongside created_timestamp_column. Review fixes: - HybridOfflineStore now forwards optional kwargs to the delegated store instead of raising TypeError for supported underlying stores. - The dask filter now excludes feature rows with a null created timestamp (consistent with the SQL predicate) while still keeping unmatched entity rows from the left join. - The integration test only treats NotImplementedError from get_historical_features itself as an unsupported-store skip. - Add template-render tests for all SQL dialects and a dask null created-timestamp test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

…re capability flag Simplification pass over the feature, reviewed again by Codex: - Declare support via OfflineStore.supports_filter_by_created_timestamp (default False) and check it once in the passthrough provider via ensure_filter_by_created_timestamp_supported, replacing the three per-store NotImplementedError guards. Unsupported stores can no longer silently ignore the flag. - The hybrid store re-checks the resolved child store before delegating. - Standardize every supporting store on an explicit filter_by_created_timestamp parameter instead of kwargs.get lookups. - Document the flag in the OfflineStore.get_historical_features docstring alongside start_date/end_date. - Return the dask created-timestamp filter lazily so the mask fuses into the following _drop_duplicates persist instead of forcing an extra materialization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

franciscojavierarceo

The Dask implementation can drop entity rows. It performs the left merge first and then _filter_created_timestamp() removes rows whose matched feature has created_timestamp > entity_timestamp. If every candidate for an entity is too new, the merge produced no unmatched/null row to fall back to, so filtering removes that entity entirely. get_historical_features() should preserve entity-dataframe cardinality and return null features in this case. Please apply the cutoff in/before the join or reconstruct unmatched rows, and add a regression where all matching candidates are future-created.

ntkathole · 2026-07-30T06:32:21Z

            feature_table[timestamp_field] <= entity_table[event_timestamp_col],
        )

+        if filter_by_created_timestamp and created_timestamp_field:


This predicate casts the created timestamp to UTC but the existing event-timestamp comparison on the same join doesn't cast. Compare with the existing predicate at line 433-434 which does no cast.

Thanks @ntkathole, made a change so read_fv now normalizes the created timestamp on read, as it already did for the event timestamp, so the predicate no longer casts. Made it unconditional since deduplicate() reads that column regardless of the flag.

codecov-commenter · 2026-07-30T07:12:43Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 70.27027% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 46.74%. Comparing base (faf85e0) to head (3d2ea3e).

Files with missing lines	Patch %	Lines
sdk/python/feast/infra/offline_stores/ibis.py	33.33%	4 Missing ⚠️
sdk/python/feast/feature_store.py	33.33%	1 Missing and 1 partial ⚠️
...feast/infra/offline_stores/hybrid_offline_store.py	33.33%	2 Missing ⚠️
sdk/python/feast/infra/passthrough_provider.py	0.00%	1 Missing and 1 partial ⚠️
...ffline_stores/contrib/mssql_offline_store/mssql.py	0.00%	1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6617      +/-   ##
==========================================
+ Coverage   46.46%   46.74%   +0.28%     
==========================================
  Files         414      414              
  Lines       50138    50173      +35     
  Branches     7173     7179       +6     
==========================================
+ Hits        23295    23454     +159     
+ Misses      25204    25079     -125     
- Partials     1639     1640       +1

Flag	Coverage Δ
go-feature-server	`30.58% <ø> (ø)`
python-unit	`48.07% <70.27%> (+0.30%)`	⬆️

Files with missing lines	Coverage Δ
sdk/python/feast/infra/offline_stores/bigquery.py	`49.59% <100.00%> (+0.08%)`	⬆️
...line_stores/contrib/athena_offline_store/athena.py	`39.15% <100.00%> (+0.32%)`	⬆️
...res/contrib/clickhouse_offline_store/clickhouse.py	`32.55% <100.00%> (+0.39%)`	⬆️
...tores/contrib/couchbase_offline_store/couchbase.py	`30.76% <100.00%> (+30.76%)`	⬆️
...line_stores/contrib/oracle_offline_store/oracle.py	`39.75% <100.00%> (+0.18%)`	⬆️
..._stores/contrib/postgres_offline_store/postgres.py	`45.23% <100.00%> (+0.14%)`	⬆️
...ffline_stores/contrib/spark_offline_store/spark.py	`36.25% <100.00%> (+0.09%)`	⬆️
...ffline_stores/contrib/trino_offline_store/trino.py	`50.67% <100.00%> (+0.22%)`	⬆️
sdk/python/feast/infra/offline_stores/dask.py	`53.30% <100.00%> (+0.64%)`	⬆️
sdk/python/feast/infra/offline_stores/duckdb.py	`39.27% <100.00%> (+0.20%)`	⬆️
... and 9 more

... and 3 files with indirect coverage changes

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update faf85e0...3d2ea3e. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The Dask cutoff filtered rows after the left join, so an entity whose every candidate version was created after its timestamp lost all of its rows and disappeared from the result. get_historical_features must preserve entity-dataframe cardinality and return null features instead. Blank the feature-view columns rather than dropping the row, which leaves it indistinguishable from an unmatched left join. _drop_duplicates already sorts nulls first and keeps the last row, so a valid version still wins where one exists and a blanked row survives only when nothing else does. The unit tests now set fv.entity_columns. Without it the derived join keys are empty and _merge silently cross joins, which hid the per-entity behaviour because the existing cases all used a single entity row. Signed-off-by: David <david-adeniji@hotmail.co.uk>

The cutoff predicate cast created_timestamp to UTC while the event-timestamp comparison beside it did not. read_fv normalizes the event timestamp when it reads the source and left the created timestamp alone, so the predicate was compensating for a missing normalization at the one comparison site. Normalize both on read instead. The predicate then needs no cast and matches its neighbour. deduplicate() orders by created_timestamp_column regardless of the cutoff flag, so the normalization is unconditional rather than gated on it; casting a column that is already tz-aware compiles away, so this leaves the emitted query unchanged for tz-aware sources and retires the "mutate only if tz-naive" TODO. Signed-off-by: David <david-adeniji@hotmail.co.uk>

Signed-off-by: David <david-adeniji@hotmail.co.uk>

addenergyx · 2026-07-31T10:40:27Z

Thanks @franciscojavierarceo should be fixed now, it now nulls out the feature columns instead of dropping the row, so the entity row survives with null features

ntkathole

Looks good

…_features (feast-dev#6617) * feat: Add opt-in at_event_time created_timestamp cutoff to get_historical_features When a feature view has a created_timestamp_column, it is currently used only as a dedup tiebreaker in point-in-time joins, so retrieval can serve feature values whose created_timestamp is after the entity row's event timestamp (backfills, late corrections). This leaks future information into training data and makes training sets non-reproducible. Add an opt-in at_event_time flag (default False) to get_historical_features that adds a created_timestamp <= entity_timestamp predicate to the point-in-time join, so retrieval reflects what was known as of each entity row's timestamp. Implemented for the SQL template stores (BigQuery, Redshift, Spark, Trino, Athena, Postgres, ClickHouse, Couchbase), the ibis-based stores (DuckDB, MSSQL, Oracle) and the dask store. Snowflake, Remote and Ray raise NotImplementedError when the flag is set. Fixes feast-dev#6615 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * test: Add universal integration test and docs for at_event_time Adds a universal offline store integration test covering at_event_time (default returns backfilled values, at_event_time=True only returns values created at or before the entity timestamp, stores without support skip via NotImplementedError) and documents the flag on the point-in-time joins concept page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * refactor: Rename at_event_time to filter_by_created_timestamp and address review feedback Rename the flag to filter_by_created_timestamp to match the repo's mechanical flag naming and sit alongside created_timestamp_column. Review fixes: - HybridOfflineStore now forwards optional kwargs to the delegated store instead of raising TypeError for supported underlying stores. - The dask filter now excludes feature rows with a null created timestamp (consistent with the SQL predicate) while still keeping unmatched entity rows from the left join. - The integration test only treats NotImplementedError from get_historical_features itself as an unsupported-store skip. - Add template-render tests for all SQL dialects and a dask null created-timestamp test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * refactor: Centralize filter_by_created_timestamp support behind a store capability flag Simplification pass over the feature, reviewed again by Codex: - Declare support via OfflineStore.supports_filter_by_created_timestamp (default False) and check it once in the passthrough provider via ensure_filter_by_created_timestamp_supported, replacing the three per-store NotImplementedError guards. Unsupported stores can no longer silently ignore the flag. - The hybrid store re-checks the resolved child store before delegating. - Standardize every supporting store on an explicit filter_by_created_timestamp parameter instead of kwargs.get lookups. - Document the flag in the OfflineStore.get_historical_features docstring alongside start_date/end_date. - Return the dask created-timestamp filter lazily so the mask fuses into the following _drop_duplicates persist instead of forcing an extra materialization. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * chore: Trim comments to the non-obvious constraints Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * docs: Tighten filter_by_created_timestamp docstring Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * docs: Condense filter_by_created_timestamp caveats into a hint Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk> * fix: Keep entity rows whose candidate versions are all future-created The Dask cutoff filtered rows after the left join, so an entity whose every candidate version was created after its timestamp lost all of its rows and disappeared from the result. get_historical_features must preserve entity-dataframe cardinality and return null features instead. Blank the feature-view columns rather than dropping the row, which leaves it indistinguishable from an unmatched left join. _drop_duplicates already sorts nulls first and keeps the last row, so a valid version still wins where one exists and a blanked row survives only when nothing else does. The unit tests now set fv.entity_columns. Without it the derived join keys are empty and _merge silently cross joins, which hid the per-entity behaviour because the existing cases all used a single entity row. Signed-off-by: David <david-adeniji@hotmail.co.uk> * refactor: Normalize created timestamp on read, not in the join predicate The cutoff predicate cast created_timestamp to UTC while the event-timestamp comparison beside it did not. read_fv normalizes the event timestamp when it reads the source and left the created timestamp alone, so the predicate was compensating for a missing normalization at the one comparison site. Normalize both on read instead. The predicate then needs no cast and matches its neighbour. deduplicate() orders by created_timestamp_column regardless of the cutoff flag, so the normalization is unconditional rather than gated on it; casting a column that is already tz-aware compiles away, so this leaves the emitted query unchanged for tz-aware sources and retires the "mutate only if tz-naive" TODO. Signed-off-by: David <david-adeniji@hotmail.co.uk> * Shorten comments in the created timestamp cutoff Signed-off-by: David <david-adeniji@hotmail.co.uk> --------- Signed-off-by: David <david-adeniji@hotmail.co.uk> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

addenergyx and others added 3 commits July 19, 2026 18:38

addenergyx changed the title ~~feat: Add opt-in at_event_time created_timestamp cutoff to get_historical_features~~ feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features Jul 19, 2026

addenergyx and others added 2 commits July 19, 2026 19:30

Merge branch 'master' into at-event-time

9a2f805

addenergyx marked this pull request as ready for review July 21, 2026 17:07

addenergyx requested review from a team and sudohainguyen as code owners July 21, 2026 17:07

addenergyx requested review from ejscribner, robhowley and tokoko and removed request for a team July 21, 2026 17:07

addenergyx and others added 3 commits July 21, 2026 18:10

chore: Trim comments to the non-obvious constraints

6d1cf1d

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

docs: Tighten filter_by_created_timestamp docstring

6d237e5

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

docs: Condense filter_by_created_timestamp caveats into a hint

90c0bb4

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: David <david-adeniji@hotmail.co.uk>

franciscojavierarceo requested changes Jul 29, 2026

View reviewed changes

Merge branch 'master' into at-event-time

6abf239

ntkathole reviewed Jul 30, 2026

View reviewed changes

addenergyx added 5 commits July 31, 2026 09:41

Merge branch 'master' into at-event-time

521eaab

Merge branch 'master' into at-event-time

3fccd8e

Shorten comments in the created timestamp cutoff

3d2ea3e

Signed-off-by: David <david-adeniji@hotmail.co.uk>

ntkathole added the ok-to-test label Jul 31, 2026

ntkathole approved these changes Jul 31, 2026

View reviewed changes

ntkathole merged commit 79b33ce into feast-dev:master Jul 31, 2026
33 of 34 checks passed

Navigation Menu

Uh oh!

feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features - #6617

feat: Add opt-in filter_by_created_timestamp cutoff to get_historical_features#6617
ntkathole merged 14 commits into
feast-dev:masterfrom
addenergyx:at-event-time

addenergyx commented Jul 19, 2026 •

edited

Loading

Uh oh!

franciscojavierarceo left a comment

Uh oh!

ntkathole Jul 30, 2026

Uh oh!

addenergyx Jul 31, 2026

Uh oh!

codecov-commenter commented Jul 30, 2026 •

edited

Loading

Uh oh!

addenergyx commented Jul 31, 2026

Uh oh!

ntkathole left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

addenergyx commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it

Design

Store coverage

Notes

Which issue(s) this PR fixes

Misc

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

ntkathole Jul 30, 2026

Choose a reason for hiding this comment

Uh oh!

addenergyx Jul 31, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

addenergyx commented Jul 31, 2026

Uh oh!

ntkathole left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

addenergyx commented Jul 19, 2026 •

edited

Loading

codecov-commenter commented Jul 30, 2026 •

edited

Loading