Skip to content

Improve GCP exception handling#1561

Merged
woop merged 5 commits into
feast-dev:masterfrom
woop:remove-anon-client
May 15, 2021
Merged

Improve GCP exception handling#1561
woop merged 5 commits into
feast-dev:masterfrom
woop:remove-anon-client

Conversation

@woop

@woop woop commented May 14, 2021

Copy link
Copy Markdown
Member

Signed-off-by: Willem Pienaar git@willem.co

What this PR does / why we need it:

If no gcloud account is specified, then users today are not provided with a useful exception. Instead, the exception is hidden behind a failure due to an anonymous client not having access to their resources.

Does this PR introduce a user-facing change?:

NONE

@woop woop requested review from a team, jklegar and tsotnet as code owners May 14, 2021 23:06
@feast-ci-bot

Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: woop

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jklegar

jklegar commented May 14, 2021

Copy link
Copy Markdown
Collaborator

/lgtm

@feast-ci-bot

Copy link
Copy Markdown
Collaborator

New changes are detected. LGTM label has been removed.

@woop woop changed the title Remove silent failure with GCS registry Improve GCP exception handling May 14, 2021
Comment thread sdk/python/feast/infra/gcp.py Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc we didn't have google cloud dependencies imported at the top because Feast will try to import them even when the user is not using gcp (e.g. they're just trying things locally) due to this file being imported somewhere else. If you uninstall the google cloud pip packages in your venv and try the quickstart you can confirm whether this happens

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, but I think its cleaner to conditionally import our providers and stores than their subdependencies. Let me test to validate it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think only this change was necessary. Our tests are failing for other reasons

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here as well, I think (though it may have been just one of gcp.py and bigquery.py where this happened)

woop added 5 commits May 14, 2021 18:28
Signed-off-by: Willem Pienaar <git@willem.co>
Signed-off-by: Willem Pienaar <git@willem.co>
Signed-off-by: Willem Pienaar <git@willem.co>
Signed-off-by: Willem Pienaar <git@willem.co>
Signed-off-by: Willem Pienaar <git@willem.co>
@woop woop force-pushed the remove-anon-client branch from 438c013 to 1169b30 Compare May 15, 2021 01:28
@codecov-commenter

codecov-commenter commented May 15, 2021

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 52.17391% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.72%. Comparing base (1436a02) to head (1169b30).
⚠️ Report is 2773 commits behind head on master.

Files with missing lines Patch % Lines
sdk/python/feast/infra/gcp.py 50.00% 3 Missing ⚠️
sdk/python/feast/infra/offline_stores/bigquery.py 50.00% 3 Missing ⚠️
sdk/python/feast/registry.py 25.00% 3 Missing ⚠️
sdk/python/feast/errors.py 60.00% 2 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1561      +/-   ##
==========================================
- Coverage   83.81%   83.72%   -0.09%     
==========================================
  Files          65       65              
  Lines        5628     5634       +6     
==========================================
  Hits         4717     4717              
- Misses        911      917       +6     
Flag Coverage Δ
integrationtests 83.63% <52.17%> (-0.09%) ⬇️
unittests 78.66% <36.36%> (+1.16%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@woop woop merged commit 73ac3de into feast-dev:master May 15, 2021
@tedhtchang

Copy link
Copy Markdown
Contributor

Fixes #1560 #1485

woop added a commit that referenced this pull request May 19, 2021
* Remove try/catch that fails silently

Signed-off-by: Willem Pienaar <git@willem.co>

* Remove unused import

Signed-off-by: Willem Pienaar <git@willem.co>

* Provided clearer exception handling for GCP dependencies

Signed-off-by: Willem Pienaar <git@willem.co>

* Ensure that GCP dependencies aren't loaded unnecessarily

Signed-off-by: Willem Pienaar <git@willem.co>

* Fix lint

Signed-off-by: Willem Pienaar <git@willem.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants