-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Closed
Labels
Description
Expected Behavior
Snowflake offline export to Parquet in S3 should successfully export the materialization data set, including timestamp fields.
Current Behavior
Export to S3 integration in Parquet format fails with the following message:
Error encountered when unloading to PARQUET: TIMESTAMP_TZ and LTZ types are not supported for unloading to Parquet. value get: TIMESTAMP_LTZ
This is a known issue with recommended guidance.
Steps to reproduce
- Configure offline store in Snowflake
- Attempt to run materialization to external destination (e.g. DynamoDB)
- Confirm "temporary_" table is created in the offline schema and an attempt was made to write to the storage integration
Specifications
- Version: 0.29.0+
- Platform: LInux + EKS
- Subsystem: Python, Snowflake, DynamoDB
Possible Solution
Two things will assist in resolving this issue:
- update the data extraction to cast timestamp fields to string or convert to a UTC timestamp integer
- add an optional configuration option to enable timestamp field treatment, default to None/False