Skip to content

Remove Duplicated Strip Projects Code from SDKs #820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
feast-ci-bot merged 23 commits into from
Jun 25, 2020
Merged

Remove Duplicated Strip Projects Code from SDKs #820

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
492e682
Added project field to ServingService proto's GetOnlineFeaturesRespon…
mrzzy Jun 23, 2020
6e2c296
Update Serving's OnlineServingService to apply project override when …
mrzzy Jun 23, 2020
894ce17
Updated Python SDK use project field and remove strip project code
mrzzy Jun 23, 2020
41779f3
Recompile & update go protos based on current protobuf spec
mrzzy Jun 23, 2020
9d0cf1d
Update Go SDK to use project field and remove strip project code
mrzzy Jun 23, 2020
2892788
Update Java SDK to use project and remove strip project code.
mrzzy Jun 23, 2020
6f402bd
Fix python lint
mrzzy Jun 23, 2020
271d82e
Revert ignore project flag in python SDK's FeatureRef.from_str()
mrzzy Jun 23, 2020
3aecc8e
Reorder basic e2e tests to ensure that list features and entities tes…
mrzzy Jun 24, 2020
7aa22be
Remove redudant nested if condition in Serving's RedisOnlineRetriever
mrzzy Jun 24, 2020
f260f00
Added support for project field in getFeatureSets() method in Serving…
mrzzy Jun 24, 2020
8aca999
Fixed issue where Serving's OnlineServingService was only handling on…
mrzzy Jun 24, 2020
4151fe1
Revert removal of getFeatureRefsByName() in FeatureSetRequest
mrzzy Jun 24, 2020
a5370b9
Fix java unit tests
mrzzy Jun 24, 2020
76f9a0d
Fix unresolved conflicts in e2e batch tests
mrzzy Jun 24, 2020
8943342
Revert format on batch e2e test
mrzzy Jun 24, 2020
b6429fa
Revert row count used for testing
mrzzy Jun 24, 2020
3bd1cd9
Refactor Serving's OnlineServingServiceTest to reduce code deduplication
mrzzy Jun 24, 2020
b969cae
Make exception on parse feature reference more explict
mrzzy Jun 25, 2020
1ccaf74
Clarify the functionality of CachedSpecService getFeatureSets()'s pro…
mrzzy Jun 25, 2020
80c1138
Add javadoc to CachedSpecService's buildFeatureSetRequest() method
mrzzy Jun 25, 2020
ef430a9
Fix java lint
mrzzy Jun 25, 2020
a0a9a26
Further reduce duplication in OnlineServingServiceTest.
mrzzy Jun 25, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ require (
golang.org/x/lint v0.0.0-20200302205851-738671d3881b // indirect
golang.org/x/net v0.0.0-20200513185701-a91f0712d120
golang.org/x/sys v0.0.0-20200515095857-1151b9dac4a9 // indirect
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 // indirect
golang.org/x/tools v0.0.0-20200622203043-20e05c1c8ffa // indirect
google.golang.org/grpc v1.29.1
google.golang.org/protobuf v1.24.0 // indirect
gopkg.in/russross/blackfriday.v2 v2.0.0 // indirect
Expand Down
3 changes: 3 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ github.com/evanphx/json-patch v0.0.0-20190203023257-5858425f7550/go.mod h1:50XU6
github.com/evanphx/json-patch v4.2.0+incompatible/go.mod h1:50XU6AFN0ol/bzJsmQLiYLvXMP4fmwYFNcr97nuDLSk=
github.com/feast-dev/feast/sdk/go v0.0.0-20200516052424-09ff3dda724c h1:jltCNN1tpaFxCslQtHUfU5u5qodH/D8rqPmxvRWGNWM=
github.com/feast-dev/feast/sdk/go v0.0.0-20200516052424-09ff3dda724c/go.mod h1:F3sMDmJ9hxjlh0Z7fM6/atvMJd8moahKxUF1LCzVthQ=
github.com/feast-dev/feast/sdk/go v0.5.0 h1:h4UiFgqeWrw9voPqigwiSYWl/vjUn2mgoUAFIa29s3A=
github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/getkin/kin-openapi v0.1.1-0.20190507152207-d3180292eead/go.mod h1:V1z9xl9oF5Wt7v32ne4FmiF1alpS4dM6mNzoywPOXlk=
Expand Down Expand Up @@ -471,6 +472,8 @@ golang.org/x/tools v0.0.0-20200521211927-2b542361a4fc/go.mod h1:EkVYQZoAsY45+roY
golang.org/x/tools v0.0.0-20200601175630-2caf76543d99/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8 h1:8Xr1qwxn90MXYKftwNxIO2g4J+26naghxFS5rYiTZww=
golang.org/x/tools v0.0.0-20200604042327-9b20fe4cabe8/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/tools v0.0.0-20200622203043-20e05c1c8ffa h1:mMXQKlWCw9mIWgVLLfiycDZjMHMMYqiuakI4E/l2xcA=
golang.org/x/tools v0.0.0-20200622203043-20e05c1c8ffa/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
Expand Down
5 changes: 5 additions & 0 deletions protos/feast/serving/ServingService.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,11 @@ message GetOnlineFeaturesRequest {
// values will be returned.
bool omit_entities_in_response = 3;

// Optional field to specify project name override. If specified, uses the
// given project for retrieval. Overrides the projects specified in
// Feature References if both are specified.
string project = 5;

message EntityRow {
// Request timestamp of this row. This value will be used,
// together with maxAge, to determine feature staleness.
Expand Down
23 changes: 0 additions & 23 deletions sdk/go/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@ import (
"github.com/opentracing/opentracing-go"

"github.com/feast-dev/feast/sdk/go/protos/feast/serving"
"github.com/feast-dev/feast/sdk/go/protos/feast/types"
"google.golang.org/grpc"

"go.opencensus.io/plugin/ocgrpc"
Expand Down Expand Up @@ -58,28 +57,6 @@ func (fc *GrpcClient) GetOnlineFeatures(ctx context.Context, req *OnlineFeatures
entityRefs[ref] = struct{}{}
}
}

// strip projects from feature refs recieved
for _, fieldValues := range resp.GetFieldValues() {
stripFields := make(map[string]*types.Value, len(fieldValues.Fields))
stripStatuses := make(map[string]serving.GetOnlineFeaturesResponse_FieldStatus, len(fieldValues.Statuses))
for refStr, _ := range fieldValues.Fields {
_, isEntity := entityRefs[refStr]
stripRefStr := refStr
if !isEntity { // is feature ref
featureRef, err := parseFeatureRef(refStr, true)
if err != nil {
return nil, err
}
stripRefStr = toFeatureRefStr(featureRef)
}

stripFields[stripRefStr] = fieldValues.Fields[refStr]
stripStatuses[stripRefStr] = fieldValues.Statuses[refStr]
}
fieldValues.Fields, fieldValues.Statuses = stripFields, stripStatuses
}

return &OnlineFeaturesResponse{RawResponse: resp}, err
}

Expand Down
21 changes: 1 addition & 20 deletions sdk/go/client_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,25 +34,6 @@ func TestGetOnlineFeatures(t *testing.T) {
},
Project: "driver_project",
},
// check GetOnlineFeatures() should strip projects returned from serving
recieve: OnlineFeaturesResponse{
RawResponse: &serving.GetOnlineFeaturesResponse{
FieldValues: []*serving.GetOnlineFeaturesResponse_FieldValues{
{
Fields: map[string]*types.Value{
"driver_project/driver:rating": Int64Val(1),
"driver_project/rating": Int64Val(1),
"driver_project/null_value": {},
},
Statuses: map[string]serving.GetOnlineFeaturesResponse_FieldStatus{
"driver_project/driver:rating": serving.GetOnlineFeaturesResponse_PRESENT,
"driver_project/rating": serving.GetOnlineFeaturesResponse_PRESENT,
"driver_project/null_value": serving.GetOnlineFeaturesResponse_NULL_VALUE,
},
},
},
},
},
want: OnlineFeaturesResponse{
RawResponse: &serving.GetOnlineFeaturesResponse{
FieldValues: []*serving.GetOnlineFeaturesResponse_FieldValues{
Expand Down Expand Up @@ -83,7 +64,7 @@ func TestGetOnlineFeatures(t *testing.T) {
ctx := context.Background()
_, traceCtx := opentracing.StartSpanFromContext(ctx, "get_online_features")
rawRequest, _ := tc.req.buildRequest()
resp := tc.recieve.RawResponse
resp := tc.want.RawResponse
cli.EXPECT().GetOnlineFeatures(traceCtx, rawRequest).Return(resp, nil).Times(1)

client := &GrpcClient{
Expand Down
Loading