Skip to content

Update Feast Core API to support Project namespacing #359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
woop wants to merge 4 commits into from
Closed

Update Feast Core API to support Project namespacing #359

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6739053
Update Feast Core API to support Project namespacing
woop Dec 11, 2019
c46b892
Updated Feast Serving API to have project namespacing
woop Dec 12, 2019
c5e1bd9
Update Feast Java SDK client to support project namespacing
woop Dec 13, 2019
44a4820
Update Feast Java SDK documentation to support project namespacing
woop Dec 16, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions core/src/main/java/feast/core/service/SpecService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,8 @@ public GetFeatureSetResponse getFeatureSet(GetFeatureSetRequest request)
* Get featureSets matching the feature name and version provided in the filter. If the feature
* name is not provided, the method will return all featureSets currently registered to Feast.
*
* <p>The feature set name in the filter accepts any valid regex string. All matching featureSets
* will be returned.
* <p>The feature set name in the filter accepts an asterisk as a wildcard. All matching
* featureSets will be returned.
*
* <p>The version filter is optional; If not provided, this method will return all featureSet
* versions of the featureSet name provided. Valid version filters should optionally contain a
Expand Down
69 changes: 62 additions & 7 deletions protos/feast/core/CoreService.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,11 +58,28 @@ service CoreService {
//
// If the changes are valid, core will return the given store configuration in response, and
// start or update the necessary feature population jobs for the updated store.
rpc UpdateStore(UpdateStoreRequest) returns (UpdateStoreResponse);
rpc UpdateStore (UpdateStoreRequest) returns (UpdateStoreResponse);

// Creates a project. Projects serve as namespaces within which resources like features will be
// created. Both feature set names as well as field names must be unique within a project. Project
// names themselves must be globally unique.
rpc CreateProject (CreateProjectRequest) returns (CreateProjectResponse);

// Archives a project. Archived projects will continue to exist and function, but won't be visible
// through the Core API. Any existing ingestion or serving requests will continue to function,
// but will result in warning messages being logged. It is not possible to unarchive a project
// through the Core API
rpc ArchiveProject (ArchiveProjectRequest) returns (ArchiveProjectResponse);

// Lists all projects active projects.
rpc ListProjects (ListProjectsRequest) returns (ListProjectsResponse);
}

// Request for a single feature set
message GetFeatureSetRequest {
// Name of project the feature set belongs to (required)
string project = 3;

// Name of feature set (required).
string name = 1;

Expand All @@ -77,21 +94,25 @@ message GetFeatureSetResponse {

// Retrieves details for all versions of a specific feature set
message ListFeatureSetsRequest {
// Name of project that the feature sets belongs to.
string project = 2;

Filter filter = 1;

message Filter {
// Name of the desired feature set. Valid regex strings are allowed.
// Name of the desired feature set. Asterisks can be used as wildcards in the name.
// e.g.
// - .* can be used to match all feature sets
// - my-project-.* can be used to match all features prefixed by "my-project"
// - * can be used to match all feature sets
// - my-feature-set* can be used to match all features prefixed by "my-feature-set"
string feature_set_name = 1;

// Version of the desired feature set. Either a number or valid expression can be provided.
// e.g.
// - 1 will match version 1 exactly
// - >=1 will match all versions greater or equal to 1
// - <10 will match all versions less than 10
string feature_set_version = 2;
}

Filter filter = 1;
}

message ListFeatureSetsResponse {
Expand All @@ -112,6 +133,9 @@ message ListStoresResponse {
}

message ApplyFeatureSetRequest {
// Name of project that the feature set belongs to.
string project = 2;
Copy link
Copy Markdown
Collaborator

@khorshuheng khorshuheng Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this becomes a mandatory parameter?

Copy link
Copy Markdown
Member Author

@woop woop Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes


// Feature set version and source will be ignored
feast.core.FeatureSetSpec feature_set = 1;
}
Expand All @@ -133,7 +157,8 @@ message ApplyFeatureSetResponse {
Status status = 2;
}

message GetFeastCoreVersionRequest {}
message GetFeastCoreVersionRequest {
}

message GetFeastCoreVersionResponse {
string version = 1;
Expand All @@ -153,4 +178,34 @@ message UpdateStoreResponse {
}
feast.core.Store store = 1;
Status status = 2;
}

// Request to create a project
message CreateProjectRequest {
// Name of project (required)
string name = 1;
}

// Response for creation of a project
message CreateProjectResponse {
}

// Request for the archival of a project
message ArchiveProjectRequest {
Copy link
Copy Markdown
Collaborator

@khorshuheng khorshuheng Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens to the features ingested when a project is archived?

Copy link
Copy Markdown
Member Author

@woop woop Dec 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It continues to work. Ingestion is unaffected.

// Name of project to be archived
string name = 1;
}

// Response for archival of a project
message ArchiveProjectResponse {
}

// Request for listing of projects
message ListProjectsRequest {
}

// Response for listing of projects
message ListProjectsResponse {
// List of project names (archived projects are filtered out)
repeated string projects = 1;
}
5 changes: 4 additions & 1 deletion protos/feast/core/FeatureSet.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,13 @@ import "feast/core/Source.proto";
import "google/protobuf/duration.proto";

message FeatureSetSpec {
// Name of project that this feature set belongs to (read-only).
string project = 7;

// Name of the featureSet. Must be unique.
string name = 1;

// FeatureSet version.
// FeatureSet version (read-only).
int32 version = 2;

// List of entities contained within this featureSet.
Expand Down
4 changes: 4 additions & 0 deletions protos/feast/core/Store.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,10 @@ message Store {
}

message Subscription {
// Name of project where subscribed feature sets can be found. All feature sets must be located
// within this project. Wildcards are not supported.
string project = 3;

// Name of featureSet to subscribe to. This field supports any valid basic POSIX regex,
// e.g. customer_.* or .*
// https://www.regular-expressions.info/posix.html
Expand Down
22 changes: 11 additions & 11 deletions protos/feast/serving/ServingService.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,15 +62,15 @@ message GetFeastServingInfoResponse {
string job_staging_location = 10;
}

message FeatureSetRequest {
// Feature set name
string name = 1;
message FeatureReference {
// Project name
string project = 1;

// Feature set version
int32 version = 2;
// Feature name
string name = 2;

// Features that should be retrieved from this feature set
repeated string feature_names = 3;
// Feature version
int32 version = 3;

// The features will be retrieved if:
// entity_timestamp - max_age <= event_timestamp <= entity_timestamp
Expand All @@ -81,8 +81,8 @@ message FeatureSetRequest {
}

message GetOnlineFeaturesRequest {
// List of feature sets and their features that are being retrieved
repeated FeatureSetRequest feature_sets = 1;
// List of features that are being retrieved
repeated FeatureReference features = 4;

// List of entity rows, containing entity id and timestamp data.
// Used during retrieval of feature rows and for joining feature
Expand All @@ -104,8 +104,8 @@ message GetOnlineFeaturesRequest {
}

message GetBatchFeaturesRequest {
// List of feature sets and their features that are being retrieved.
repeated FeatureSetRequest feature_sets = 1;
// List of features that are being retrieved
repeated FeatureReference features = 3;

// Source of the entity dataset containing the timestamps and entity keys to retrieve
// features for.
Expand Down
2 changes: 1 addition & 1 deletion sdk/go/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.13

require (
github.com/golang/protobuf v1.3.2
github.com/google/go-cmp v0.3.0
github.com/google/go-cmp v0.3.1
github.com/opentracing/opentracing-go v1.1.0
github.com/stretchr/testify v1.4.0 // indirect
go.opencensus.io v0.22.1
Expand Down
2 changes: 2 additions & 0 deletions sdk/go/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/opentracing/opentracing-go v1.1.0 h1:pWlfV3Bxv7k65HYwkikxat0+s3pV4bsqf19k25Ur8rU=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
Expand Down
Loading