Is your feature request related to a problem? Please describe.
Storing historical value of feature in serving store consume a lot of storage space. It becomes a real problem for limited storage such as Redis.
Describe the solution you'd like
Remove the storing feature as time series data functionality and only store the latest value of feature (which currently we already have)
Impact
- Ingestion
Removal of code which writes feature as time series data in Redis and Bigtable.
- Serving
Serving currently has 1 API which can request LAST feature or LIST of feature.
If the time series data is removed then LIST request will always return empty feature.
Currently, LAST request ignore time range filter, it has to be changed to respect it to allow filtering stale feature.
- Client
Client will have to always use LAST request. (currently it uses LIST when time range filtering is not requested)
To be discussed further
- Should we add new API to serving service which simplify request and response since we know it will always return LAST? Currently the request and response is complex to cater the needs LAST and LIST.
Thoughts?
@tims @zhilingc @woop @baskaranz @mansi @budi
Is your feature request related to a problem? Please describe.
Storing historical value of feature in serving store consume a lot of storage space. It becomes a real problem for limited storage such as Redis.
Describe the solution you'd like
Remove the storing feature as time series data functionality and only store the latest value of feature (which currently we already have)
Impact
Removal of code which writes feature as time series data in Redis and Bigtable.
Serving currently has 1 API which can request LAST feature or LIST of feature.
If the time series data is removed then LIST request will always return empty feature.
Currently, LAST request ignore time range filter, it has to be changed to respect it to allow filtering stale feature.
Client will have to always use LAST request. (currently it uses LIST when time range filtering is not requested)
To be discussed further
Thoughts?
@tims @zhilingc @woop @baskaranz @mansi @budi