RequestSource can not be used with write_to_online_store #4493
Description
Expected Behavior
Use RequestSource with write_to_online_store
Current Behavior
Steps to reproduce
feature view has request source defined such as
{
"spec": {
"name": "embedding_catalog",
"features": [
{
"name": "embeddings",
"valueType": "FLOAT_LIST"
},
{
"name": "event_timestamp",
"valueType": "UNIX_TIMESTAMP"
}
],
"ttl": "0s",
"batchSource": {
"type": "REQUEST_SOURCE",
"dataSourceClassType": "feast.data_source.RequestSource",
"requestDataOptions": {
"schema": [
{
"name": "embeddings",
"valueType": "FLOAT_LIST"
},
{
"name": "event_timestamp",
"valueType": "UNIX_TIMESTAMP"
}
]
},
"name": "embedding_catalog"
},
"online": true
},
"meta": {
"createdTimestamp": "2024-09-06T07:58:13.929715Z",
"lastUpdatedTimestamp": "2024-09-06T07:58:13.929715Z"
}
}
in the _convert_arrow_to_proto function it will fail at extracting the timestamp field:
event_timestamps = [
_coerce_datetime(val)
for val in pd.to_datetime(
table.column(feature_view.batch_source.timestamp_field).to_numpy(
zero_copy_only=False
)
)
]
since feature_view.batch_source (request source) doesn't have timestamp_field.
Specifications
- Version:
- Platform:
- Subsystem:
Possible Solution
add an attributee for RequestSource for the user to specify the timestamp_field.