Skip to content

Slow materialization #5707

@miroslawas

Description

@miroslawas

Expected Behavior

Feature materialization used to call "get latest feature values" function and return only the latest values - this was much faster

Current Behavior

Materializatio now pulls all features now and deduplicates in pandas - this makes materialization much slower (almost unusable for some cases)

Steps to reproduce

Try materialize

Specifications

  • Version: 0.56
  • Platform:
  • Subsystem:

offline_store: Clickhouse, online_store: Redis

Possible Solution

Give ability to pull only latest values by introducing flag/env variable

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions