Skip to content

Calls to the get_online_features API are slower in v0.42.0 than in v0.40.1 #4887

@jiuuon

Description

@jiuuon

Expected Behavior

The call to the get_online_features API must be faster or equal to v0.42.0 than v0.40.1.

Current Behavior

v0.42.0 is slower than v0.40.1

Steps to reproduce

Specifications

  • Version: v0.42.0, v0.41.*
  • Platform: Kubernetes + Redis (Online) + Bigquery (Offline) + PostgreSQL(SQLRegistry)
  • Subsystem:

Possible Solution

Need to change the default behavior to sync (based fastapi - anyio threadpool), or change the part where blocking (or delay) occurs in async to async.

I was running 0.40.1, and upgraded to 0.41.2 / 0.42.0, but I actually found that it got slower.

Looking at the structure of async def get_online_features
(1) Get the information of the requested feature_view.
(2) Check the permission (when using RBAC).
(3) Get the value from the online store using asynchronous.
(4) Deserialize the protocol buffer message from the online store into a dict.

It flows in this order, but only (3) is actually protected by the threadpool or eventloop.
However, (1), (2), and (4) also take a lot of time for protobuf serialization-deserialization, and as a result, (1), (2), and (4) lose time when performed with a single thread (async) compared to multi-threading.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions