Skip to content

to_remote_storage() resulting in undeleted temporary tables in Snowflake #3545

@connorwaldman

Description

@connorwaldman

Expected Behavior

When calling get_historical_features.to_remote_storage(), any temporary tables created in Snowflake are deleted after the Snowflake session ends.

Current Behavior

When calling get_historical_features.to_remote_storage(), the temporary tables created during the join process are not deleted after the Snowflake session ends. These tables are set to a retention time of 1 day, but they are not deleted and still exist after 24 hours.

I tested this with to_df() and the above described issue does not occur. I also tried explicitly ending the session to make sure that wasn't the issue, but even after confirming the session was ended, the issue still persists.

Steps to reproduce

  1. For the FeatureStore object, set the RepoConfig offline store config to specify the following:
{
     "blob_export_location": <s3_staging_url>,
     "storage_integration_name": <storage_integration>,
     "role": <stage_role>,
     "schema_": <stage_schema>,
}
  1. Call get_historical_features(entity_df=entity, features=features, full_feature_names=True).to_remote_storage()
  2. Check snowflake stage tables and look for tables created at the time of running that start with temporary_

Specifications

  • Version: 0.30.0
  • Platform: Ubuntu
  • Subsystem:

Possible Solution

No possible solution known at the time of reporting

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions