Cannot push the data to Offline store server, due to incorrect timestamp fields #5549
Description
Expected Behavior
Push to Offline store works for following
df = pd.read_parquet('data/train.parquet')
fs.push(push_source_name='train_push_source', df=df, to=PushMode.OFFLINE)
Current Behavior
The push to offline store server from a clinet throws error:
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 684, in _ConvertFieldValuePair
self.ConvertMessage(value, sub_message, '{0}.{1}'.format(path, name))
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 540, in ConvertMessage
self._ConvertFieldValuePair(value, message, path)
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 696, in _ConvertFieldValuePair
raise ParseError(str(e)) from e
google.protobuf.json_format.ParseError: Message type "feast.core.DataSource.SourceMeta" has no field named "createdTimestamp" at "DataSource.meta".
Available Fields(except extensions): "['earliestEventTimestamp', 'latestEventTimestamp']"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.11/site-packages/feast/offline_server.py", line 256, in do_get
table = self.get_table_column_names_and_types_from_data_source(command)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app-root/lib64/python3.11/site-packages/feast/offline_server.py", line 513, in get_table_column_names_and_types_from_data_source
data_source = OfflineServer._extract_data_source_from_command(command)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/app-root/lib64/python3.11/site-packages/feast/offline_server.py", line 497, in _extract_data_source_from_command
Parse(data_source_proto_str, data_source_proto)
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 465, in Parse
raise e
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 461, in Parse
return ParseDict(
^^^^^^^^^^
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 495, in ParseDict
parser.ConvertMessage(js_dict, message, '')
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 540, in ConvertMessage
self._ConvertFieldValuePair(value, message, path)
File "/opt/app-root/lib64/python3.11/site-packages/google/protobuf/json_format.py", line 692, in _ConvertFieldValuePair
raise ParseError(
google.protobuf.json_format.ParseError: Failed to parse meta field: Message type "feast.core.DataSource.SourceMeta" has no field named "createdTimestamp" at "DataSource.meta".
Available Fields(except extensions): "['earliestEventTimestamp', 'latestEventTimestamp']".
Steps to reproduce:
- Create the datasource and point to the file having timestamp fields, also mention the custom timestamp fields in FileSource.
- Create PushSource from file source.
- Create a feature View with custom fields.
- Attempt to push the DataFrame to offline store with the push source and dataframe.
df = pd.read_parquet('data/train.parquet')
fs.push(push_source_name='train_push_source', df=df, to=PushMode.OFFLINE)
Specifications
- Version: 0.51.0
- Platform:
- Subsystem: