Paritioned entity source #149
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
khorshuheng
force-pushed
the
partitioned-entity-source
branch
2 times, most recently
from
8d82a81 to
98411f4
Compare
python/feast_spark/client.py
Outdated
| Table will expire in 1 day. | ||
| Returns BigQuerySource with reference to created table. | ||
| """ | ||
| from google.cloud import bigquery |
There was a problem hiding this comment.
what is the rationale for importing inside a func ?
I guess this is not expected to run in user's env and thus this dependency should be present in env ?
Is this method expected to be called rarely ?
| ) | ||
|
|
||
|
|
||
| def stage_entities_to_bq_with_partition( |
There was a problem hiding this comment.
can we avoid duplication by adding partition as an optional argument in existing feast.staging.entities file ?
There was a problem hiding this comment.
The issue is that, we would have to make this change in the feast repository, and we are having issue publishing Python SDK for branch 0.9 at the moment.
khorshuheng
force-pushed
the
partitioned-entity-source
branch
from
98411f4 to
488e734
Compare
Signed-off-by: shuheng <khor.heng@gojek.com> Co-authored-by: shuheng <khor.heng@gojek.com>
khorshuheng
force-pushed
the
partitioned-entity-source
branch
2 times, most recently
from
950e689 to
ae5715a
Compare
3 participants
What this PR does / why we need it:
The bigquery table corresponds to the entity source should be partitioned in order to achieve better performance for historical retrieval job.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: