Skip to content

Remove feature set update from job update cycle#711

Closed
zhilingc wants to merge 6 commits intofeast-dev:masterfrom
zhilingc:jobs-no-update-fs
Closed

Remove feature set update from job update cycle#711
zhilingc wants to merge 6 commits intofeast-dev:masterfrom
zhilingc:jobs-no-update-fs

Conversation

@zhilingc
Copy link
Copy Markdown
Collaborator

@zhilingc zhilingc commented May 15, 2020

What this PR does / why we need it:
Removes the feature set update from the job update cycle, because job updates are long running and are write-transactional, and so can block feast core for long periods of time. Hopefully this improves the performance of the system.

The responsibility of making feature set status updates is pushed to when the feature set is retrieved (either by ListFeatureSets or GetFeatureSet).

Would like the dataflow tests to be fixed and run on this PR before this is merged in, ideally.

Which issue(s) this PR fixes:

Fixes #664

Does this PR introduce a user-facing change?:


@zhilingc
Copy link
Copy Markdown
Collaborator Author

/hold

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zhilingc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@zhilingc zhilingc force-pushed the jobs-no-update-fs branch from 35297db to e05f014 Compare May 18, 2020 06:51
@zhilingc zhilingc force-pushed the jobs-no-update-fs branch from 5beabcc to c58a617 Compare May 18, 2020 07:30
@zhilingc
Copy link
Copy Markdown
Collaborator Author

/test test-end-to-end

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

@zhilingc: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
test-end-to-end-redis-cluster c58a617 link /test test-end-to-end-redis-cluster
test-end-to-end c58a617 link /test test-end-to-end

Full PR test history

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@zhilingc
Copy link
Copy Markdown
Collaborator Author

PR is broken after rebase 😿

@woop
Copy link
Copy Markdown
Member

woop commented Jun 19, 2020

Let's pick this up in the future if its necessary. Made somewhat redundant by #792.

@woop woop closed this Jun 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Job Management Optimizations

3 participants