Skip to content

Compute and write metrics for rows prior to store writes#763

Merged
feast-ci-bot merged 2 commits intofeast-dev:masterfrom
zhilingc:preingest-metrics
Jun 19, 2020
Merged

Compute and write metrics for rows prior to store writes#763
feast-ci-bot merged 2 commits intofeast-dev:masterfrom
zhilingc:preingest-metrics

Conversation

@zhilingc
Copy link
Copy Markdown
Collaborator

@zhilingc zhilingc commented Jun 2, 2020

What this PR does / why we need it:

In PR #568, ingestion metrics were shifted to be computed and written post-store-writes, in order to accurately reflect the values that are actually available to users.

However, for certain stores (like BQ), successfully written rows cannot be retrieved post-write, and as a result feature row metrics are not produced for ingestion jobs writing to such stores. This PR re-introduces inflight feature row metrics as an alternative source of monitoring.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

Adds the tag "metrics_namespace" to the following metrics:
- feature_row_lag_ms_*
- feature_value_lag_ms_*
- feature_row_ingested_count
- feature_value_missing_count
- feature_value_*

The tag can contain either of:
- `Inflight` 
- `WriteToStoreSuccess`

@zhilingc
Copy link
Copy Markdown
Collaborator Author

zhilingc commented Jun 2, 2020

/test test-end-to-end-batch

@Yanson
Copy link
Copy Markdown
Contributor

Yanson commented Jun 2, 2020

Does this fix #747?

@mrzzy mrzzy added the compat/breaking Breaking user-facing change label Jun 3, 2020
@zhilingc
Copy link
Copy Markdown
Collaborator Author

zhilingc commented Jun 3, 2020

Does this fix #747?

Sort of. It provides an alternative means to monitor elements being ingested when the sink is BigQuery.

@woop woop mentioned this pull request Jun 4, 2020
@woop woop added this to the v0.6.0 milestone Jun 4, 2020
@woop
Copy link
Copy Markdown
Member

woop commented Jun 9, 2020

@zhilingc can we add this change in metrics to our documentation please?

@feast-ci-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: woop, zhilingc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@woop
Copy link
Copy Markdown
Member

woop commented Jun 19, 2020

/lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants