Background

Currently Feast uses string references in 3 areas:

• Feature Set References
• Feature References
• Subscriptions

As a result many implementations of string reference parsing and formatting has cropped up all over Feast's code:

• Feast Serving's RefUtil
• Feast Core's Store
• Feast Ingestion's SpecUtil
• Feast Storage Writers: Redis RedisCluster.
• Feast Storage Retrievers Redis RedisCluster
• etc...

Problem

Changes to the feature/feature set references/subscriptions are difficult as we have to track down all the different string parsing/formatting implementations. Changes to these areas should not require a comb over the entire codebase to rectify all these parsing and formatting implementations.

Examples of currently WIP and potential changes that is currently affected by this and why the problem is relevant:

• Feature Set version removal Feature set versioning #386
• Changes to the Feast concept & hierarchy and data model Feast API: Feature references, concept hierarchy, and data model #479
• Removal of Projects from Feature References when in the string form Validate feature reference and project functionality #631

Scope

String references that should still be kept around:

• those exposed by the user facing API (ie the SDKs, config files) as they need to be humanly readable.
• those used by interface with components outside of feast (ie PostgreSQL, BigQuery, etc.), although non string references should be preferred if possible.

Possible Solution

Replace string references in non user facing components with their Protobuf equivalents.
Examples of this include:

• Using message types directly in protobuf definitions:

message FeatureRow {
    repeated Field fields = 2;
    google.protobuf.Timestamp event_timestamp = 3;
    FeatureSetReference feature_set_ref = 8;
    string ingestion_id = 7;
}

• Using references as keys in maps directly.

HashMap<FeatureSetReference, FeatureSetSpec> featureSetMap = new HashMap<>();

⚠️ There might be possible caveats to this

• Serialize references without having to extract its internals:

String refStr = Base64.getEncoder().encodeToString(featureSetReference.toByteArray());