Skip to content

feat: Serialization version 2 and below removed #5435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 2 commits into from
Jun 22, 2025
Merged

feat: Serialization version 2 and below removed #5435

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
146185e
Serialization version 2 and below removed
jyejare Jun 9, 2025
88b1715
Redis Test Fixed for entity key prefix
jyejare Jun 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/getting-started/quickstart.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
{% endtab %}

Expand Down
8 changes: 4 additions & 4 deletions docs/how-to-guides/entity-reserialization-of-from-v2-to-v3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Entity Key Re-Serialization from Version 2 to 3
# Entity Key Serialization from Version 2 to Version 3

Entity Key Serialization version 2 will soon be deprecated, hence we need to shift the serialization and deserilization to version 3.
Entity Key Serialization version 2 is now deprecated. All new and existing Feast deployments should shift to using serialization version 3.

But here comes the challegnge where existing FeatuteViews on stores has written features with version 2. A version 2 serialized entity key cant be retrived using version 3 deserilization algorithm.
However, a challenge arises when existing FeatureViews in online or offline stores have written features with version 2. A version 2 serialized entity key cannot be retrieved using the version 3 deserialization algorithm.

## Reserialize the Feature Views entity Keys to version 3

The solution is to reserialize the entity keys from version 2 to version 3.

Follow the following procedures to reserialize the entity key to version 3 in feature View in an offline / online store.

In hosrt, you need to iterate through all the feature views in your Feast repository, retrieve their serialized entity keys (if stored in version 2), reserialize them to version 3, and then update the online/offline store or wherever the serialized keys are stored.
In short, you need to iterate through all the feature views in your Feast repository, retrieve their serialized entity keys (if stored in version 2), reserialize them to version 3, and then update the online/offline store or wherever the serialized keys are stored.

### 1. Initialize the Feature Store

Expand Down
2 changes: 1 addition & 1 deletion docs/how-to-guides/running-feast-in-production.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def materialize(data_interval_start=None, data_interval_end=None):
provider="aws",
offline_store="file",
online_store=DynamoDBOnlineStoreConfig(region="us-west-2"),
entity_key_serialization_version=2
entity_key_serialization_version=3
)
store = FeatureStore(config=repo_config)
# Option 1: materialize just one feature view
Expand Down
6 changes: 3 additions & 3 deletions docs/how-to-guides/starting-feast-servers-tls-mode.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down Expand Up @@ -121,7 +121,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down Expand Up @@ -161,7 +161,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
offline_store:
type: remote
host: localhost
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/batch-materialization/spark.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ repo_config = RepoConfig(
"partitions": 10
},
online_store=DynamoDBOnlineStoreConfig(region="us-west-1"),
entity_key_serialization_version=2
entity_key_serialization_version=3
)

store = FeatureStore(config=repo_config)
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/feast-cli-commands.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ online_store:
path: data/online_store.db
offline_store:
type: dask
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/online-stores/couchbase.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ online_store:
password: password # Couchbase password from access credentials
bucket_name: feast # Couchbase bucket name, defaults to feast
kv_port: 11210 # Couchbase key-value port, defaults to 11210. Required if custom ports are used.
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
{% endcode %}

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/online-stores/remote.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ online_store:
path: http://localhost:6566
type: remote
cert: /path/to/cert.pem
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
```
Expand Down
22 changes: 15 additions & 7 deletions examples/credit-risk-end-to-end/02_Deploying_the_Feature_Store.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 2,
"id": "8bd21689-4a8e-4b0c-937d-0911df9db1d3",
"metadata": {},
"outputs": [],
Expand Down Expand Up @@ -102,15 +102,15 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 6,
"id": "b3757221-2037-49eb-867f-b9529fec06e2",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing Feature_Store/feature_store.yaml\n"
"Overwriting Feature_Store/feature_store.yaml\n"
]
}
],
Expand All @@ -125,7 +125,7 @@
"online_store:\n",
" type: sqlite\n",
" path: data/online_store.db\n",
"entity_key_serialization_version: 2"
"entity_key_serialization_version: 3"
]
},
{
Expand Down Expand Up @@ -383,10 +383,18 @@
"id": "c764a60a-b911-41a8-ba8f-7ef0a0bc7257",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/jyejare/gitrepos/feast/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\n",
" warnings.warn(\n"
]
},
{
"data": {
"text/plain": [
"RepoConfig(project='loan_applications', provider='local', registry_config='data/registry.db', online_config={'type': 'sqlite', 'path': 'data/online_store.db'}, offline_config={'type': 'dask'}, batch_engine_config='local', feature_server=None, flags=None, repo_path=PosixPath('Feature_Store'), entity_key_serialization_version=2, coerce_tz_aware=True)"
"RepoConfig(project='loan_applications', provider='local', registry_config='data/registry.db', online_config={'type': 'sqlite', 'path': 'data/online_store.db'}, auth={'type': 'no_auth'}, offline_config={'type': 'dask'}, batch_engine_config='local', feature_server=None, flags=None, repo_path=PosixPath('Feature_Store'), entity_key_serialization_version=3, coerce_tz_aware=True)"
]
},
"execution_count": 7,
Expand Down Expand Up @@ -779,7 +787,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"display_name": ".venv",
"language": "python",
"name": "python3"
},
Expand All @@ -793,7 +801,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
"version": "3.11.13"
}
},
"nbformat": 4,
Expand Down
2 changes: 1 addition & 1 deletion examples/credit-risk-end-to-end/03_Credit_Risk_Model_Training.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@
" \"host\": \"localhost\",\n",
" \"port\": 8815\n",
" },\n",
" entity_key_serialization_version=2\n",
" entity_key_serialization_version=3\n",
"))"
]
},
Expand Down
5 changes: 2 additions & 3 deletions examples/java-demo/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Running Feast Java Server with Redis & calling with python (with registry in GCP)

For this tutorial, we setup Feast with Redis, using the Feast CLI to register and materialize features, and then retrieving via a Feast Java server deployed in Kubernetes via a gRPC call.
Expand Down Expand Up @@ -40,7 +39,7 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an
connection_string: localhost:6379,password=[YOUR PASSWORD]
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
4. Run `feast apply` to apply your local features to the remote registry
- Note: you may need to authenticate to gcloud first with `gcloud auth login`
Expand Down Expand Up @@ -86,7 +85,7 @@ For this tutorial, we setup Feast with Redis, using the Feast CLI to register an
>2. `make build-java-docker-dev`
>3. In the `application-override.yaml`, uncomment the two `image: tag: dev` blocks
>4. `helm install feast-release ../../../infra/charts/feast --values application-override.yaml`
5. (Optional): check logs of the server to make sure it’s working
5. (Optional): check logs of the server to make sure it's working
```bash
kubectl logs svc/feast-release-feature-server
```
Expand Down
2 changes: 1 addition & 1 deletion examples/java-demo/feature_repo/application-override.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ feature-server:
host: my-redis-master
port: 6379
password: [YOUR PASSWORD]
entityKeySerializationVersion: 2
entityKeySerializationVersion: 3
# Uncomment below for dev
# image:
# tag: dev
Expand Down
2 changes: 1 addition & 1 deletion examples/java-demo/feature_repo/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ online_store:
connection_string: localhost:6379,password=[YOUR PASSWORD]
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/kind-quickstart/client/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ offline_store:
online_store:
path: http://localhost:8003
type: remote
entity_key_serialization_version: 2
entity_key_serialization_version: 3
auth:
type: no_auth
2 changes: 1 addition & 1 deletion examples/podman_local/feature_repo/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
5 changes: 2 additions & 3 deletions examples/python-helm-demo/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

# Running Feast Python / Go Feature Server with Redis on Kubernetes

For this tutorial, we set up Feast with Redis.
Expand Down Expand Up @@ -64,7 +63,7 @@ Manifests have been taken from [Deploy Minio in your project](https://ai-on-open
connection_string: localhost:6379,password=****
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
```
1. To run `feast apply` from the current machine we need to define the AWS credentials to connect the MinIO S3 store, which
are defined in [minio.env](./minio.env):
Expand Down Expand Up @@ -142,7 +141,7 @@ are defined in [minio.env](./minio.env):
]'
kubectl wait --for=condition=available deployment/online-server --timeout=2m
```
1. (Optional): check logs of the server to make sure it’s working
1. (Optional): check logs of the server to make sure it's working
```bash
kubectl logs svc/online-server
```
Expand Down
2 changes: 1 addition & 1 deletion examples/python-helm-demo/feature_repo/feature_store.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ online_store:
connection_string: localhost:6379,password=_REDIS_PASSWORD_
offline_store:
type: file
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/python-helm-demo/test/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ provider: local
online_store:
path: http://localhost:6566
type: remote
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-local/client/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
entity_key_serialization_version: 2
offline_store:
host: localhost
port: 8815
Expand All @@ -10,3 +9,4 @@ project: rbac
registry:
path: localhost:6570
registry_type: remote
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/client/oidc/feature_repo/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ auth:
client_secret: update-this-value
username: ${FEAST_USERNAME}
password: ${FEAST_PASSWORD}
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/feature_repo/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@ offline_store:
db_schema: public
user: feast
password: feast
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/k8s/feature_store_offline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ offline_store:
password: feast
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/k8s/feature_store_online.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ offline_store:
port: 80
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/k8s/feature_store_registry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ registry:
pool_pre_ping: true
auth:
type: kubernetes
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/oidc/feature_store_offline.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/oidc/feature_store_online.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/rbac-remote/server/oidc/feature_store_registry.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ auth:
type: oidc
auth_discovery_url: https://keycloak-feast-dev.apps.com/realms/feast-rbac/.well-known/openid-configuration
client_id: feast-client
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/remote-offline-store/offline_client/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ offline_store:
type: remote
host: localhost
port: 8815
entity_key_serialization_version: 2
entity_key_serialization_version: 3
2 changes: 1 addition & 1 deletion examples/remote-offline-store/offline_server/feature_repo/feature_store.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ provider: local
online_store:
type: sqlite
path: data/online_store.db
entity_key_serialization_version: 2
entity_key_serialization_version: 3
Loading
Loading