Skip to content

Latest commit

 

History

History
132 lines (116 loc) · 4.7 KB

File metadata and controls

132 lines (116 loc) · 4.7 KB

feature_store.yaml

Overview

feature_store.yaml is a file that is placed at the root of the Feature Repository. This file contains configuration about how the feature store runs. An example feature_store.yaml is shown below:

{% code title="feature_store.yaml" %}

project: loyal_spider
registry: data/registry.db
provider: local
online_store:
    type: sqlite
    path: data/online_store.db

{% endcode %}

Fields in feature_store.yaml

  • provider ("local" or "gcp") — Defines the environment in which Feast will execute data flows.
  • registry (a local or GCS filepath) — Defines the location of the feature registry.
  • online_store — Configures the online store. This field will have various subfields depending on the type of online store:
    • type ("sqlite" or "datastore") — Defines the type of online store.
    • path (a local filepath) — Parameter for the sqlite online store. Defines the path to the SQLite database file.
    • project_id — Optional parameter for the datastore online store. Sets the GCP project id used by Feast, if not set Feast will use the default GCP project id in the local environment.
  • project — Defines a namespace for the entire feature store. Can be used to isolate multiple deployments in a single installation of Feast.

Providers

The provider field defines the environment in which Feast will execute data flows. As a result, it also determines the default values for other fields.

Local

When using the local provider:

  • Feast can read from local Parquet data sources.
  • Feast performs historical feature retrieval (point-in-time joins) using pandas.
  • Feast performs online feature serving from a SQLite database.

GCP

When using the GCP provider:

  • Feast can read data from BigQuery data sources.
  • Feast performs historical feature retrieval (point-in-time joins) in BigQuery.
  • Feast performs online feature serving from Google Cloud Datastore.

Permissions

Command Component Permissions Recommended Role
Apply BigQuery (source)

bigquery.jobs.create

bigquery.readsessions.create

bigquery.readsessions.getData

roles/bigquery.user
Apply Datastore (destination)

datastore.entities.allocateIds

datastore.entities.create

datastore.entities.delete

datastore.entities.get

datastore.entities.list

datastore.entities.update

roles/datastore.owner
Materialize BigQuery (source) bigquery.jobs.create roles/bigquery.user
Materialize Datastore (destination)

datastore.entities.allocateIds

datastore.entities.create

datastore.entities.delete

datastore.entities.get

datastore.entities.list

datastore.entities.update

datastore.databases.get

roles/datastore.owner
Get Online Features Datastore datastore.entities.get roles/datastore.user
Get Historical Features BigQuery (source)

bigquery.datasets.get

bigquery.tables.get

bigquery.tables.create

bigquery.tables.updateData

bigquery.tables.update

bigquery.tables.delete

bigquery.tables.getData

roles/bigquery.dataEditor