Expected Behavior
When running feast materialize-incremental it should not be possible for a materialize to run starting from a date that hasn't occurred yet. Would expect materialize-incremental to choose min(now, most_recent_materialization_date)
Current Behavior
When running feast materialize-incremental for a future date, say 2025-04-08T00:00:00, it will set that date as the next start date. This breaks all future materialize-incremental commands up until that date, if you run daily materializations they will no longer function until the actual date goes beyond this erroneously entered start date. Because the data is stored as a serialized protobuf in the SQL-based registry this is also non-trivial to change.
You can see code here showing that there is no validation checks for the start date, merely the latest date that has ever run.
Steps to reproduce
Run a materialize-incremental command for a date far in the future.
Run a second materialize-incremental command for a date less than the last start date. Watch Feast always find no date to materialize.
Specifications
- Version: 0.34.1
- Platform: Mac OSX
- Subsystem: Sonoma 14.5
Possible Solution
Refactor the above function to choose min(now, most_recent_materialization_date). Happy to contribute to this fix.