Skip to content

PostgreSQL Scalable Registry: Crash on multiple apply calls #3292

@alexbanta4

Description

@alexbanta4

Expected Behavior

Feast apply can run on an already-initialized repository in the event that changes need to be applied; the registry can thus be maintained over time.

Current Behavior

Feast, when using the PostgreSQL registry, will crash due to an inability to read the metadata tables whenever apply is run on an already-initialized feature repository. The error is as follows:
Traceback (most recent call last): File "/usr/local/anaconda3/envs/generic-api/bin/feast", line 8, in <module> sys.exit(cli()) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/core.py", line 1130, in __call__ return self.main(*args, **kwargs) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/cli.py", line 519, in apply_total_command apply_total(repo_config, repo, skip_source_validation) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/usage.py", line 283, in wrapper return func(*args, **kwargs) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/repo_operations.py", line 335, in apply_total apply_total_with_repo_instance( File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/repo_operations.py", line 296, in apply_total_with_repo_instance registry_diff, infra_diff, new_infra = store.plan(repo) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/usage.py", line 294, in wrapper raise exc.with_traceback(traceback) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/usage.py", line 283, in wrapper return func(*args, **kwargs) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/feature_store.py", line 742, in plan current_infra_proto = self._registry.proto().infra.__deepcopy__() File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/infra/registry/sql.py", line 675, in proto r.infra.CopyFrom(self.get_infra(project).to_proto()) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/infra/registry/sql.py", line 574, in get_infra return Infra.from_proto(infra_object) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/infra/infra_object.py", line 150, in from_proto infra.infra_objects += [ File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/infra/infra_object.py", line 151, in <listcomp> InfraObject.from_infra_object_proto(infra_object_proto) File "/usr/local/anaconda3/envs/generic-api/lib/python3.9/site-packages/feast/infra/infra_object.py", line 73, in from_infra_object_proto if infra_object_proto.infra_object_class_type: AttributeError: 'SqliteTable' object has no attribute 'infra_object_class_type'

Steps to reproduce

Step 1: Create a basic local repo with feast init -t local (my created directory was named native_duckling in testing)
Step 2: Reconfigure the feature_repo.yaml to point to any postgres registry, e.g.
registry: registry_type: sql path: <insert_connection_string_here>
Step 3: Run feast apply. It should return:
`Created entity driver
Created feature view driver_hourly_stats_fresh
Created feature view driver_hourly_stats
Created on demand feature view transformed_conv_rate
Created on demand feature view transformed_conv_rate_fresh
Created feature service driver_activity_v2
Created feature service driver_activity_v1
Created feature service driver_activity_v3

Created sqlite table native_duckling_driver_hourly_stats_fresh
Created sqlite table native_duckling_driver_hourly_stats
`
Step 4: Make any small change to the registry.
Step 5: Run feast apply again; the error above will be generated and the apply will fail.

Specifications

  • Version: feast 0.26.0, Python 3.9
  • Platform: macOS Big Sur 11.4
  • Subsystem:

Possible Solution

The issue does not appear to come up when using a file registry, but this is not a preferable solution as my team would like to use more scalable options in production settings.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions