Skip to content

Cannot use SQLite as the SQL registry #5565

@oonisim

Description

@oonisim

Expected Behavior

feature_store.yaml with SQLite as sql registry works.

project: my_project
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
# registry: data/registry.db     # <--- works with file registry but not with sqlite as sql registry.
registry:
  registry_type: sql
  path: sqlite:///data/registry.db
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
offline_store:
  type: file
online_store:
    type: sqlite
    path: data/online_store.db
entity_key_serialization_version: 3
# By default, no_auth for authentication and authorization, other possible values kubernetes and oidc. Refer the documentation for more details.
auth:
    type: no_auth

Current Behavior

feature_store = FeatureStore(repo_path=".")
...
OperationalError: (sqlite3.OperationalError) database is locked
[SQL: INSERT INTO feast_metadata (project_id, metadata_key, metadata_value, last_updated_timestamp) VALUES (?, ?, ?, ?)]
[parameters: ('my_project', 'last_updated_timestamp', '1754965241', 1754965241)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Steps to reproduce

Create a project/repository structure.

my_project/feature_repo
├── data
│   └── driver_stats.parquet
├── example_repo.py
└── feature_store.yaml

Run.

feature_store = FeatureStore(repo_path=".")

example_repo.py:

# This is an example feature definition file

from datetime import timedelta

from feast import (
    Entity,
    FeatureService,
    FeatureView,
    Field,
    FileSource,
    Project,
)
from feast.feature_logging import LoggingConfig
from feast.infra.offline_stores.file_source import FileLoggingDestination
from feast.types import Float32, Int64

project = Project(name="my_project", description="A project for driver statistics")

driver_hourly_stats = FileSource(
    name="driver_hourly_stats_source",
    path="data/driver_stats.parquet",
    timestamp_field="event_timestamp",
    created_timestamp_column="created",
)

# You can think of an entity as a primary key used to fetch features.
driver = Entity(name="driver", join_keys=["driver_id"])

driver_hourly_stats_view = FeatureView(
    name="driver_hourly_stats",
    entities=[driver],
    ttl=timedelta(days=1),
    # The list of features defined below act as a schema to both define features
    # for both materialization of features into a store, and are used as references
    # during retrieval for building a training dataset or serving features
    schema=[
        Field(name="conv_rate", dtype=Float32),
        Field(name="acc_rate", dtype=Float32),
        Field(name="avg_daily_trips", dtype=Int64, description="Average daily trips"),
    ],
    # Tell FEAST to materialise into online store.
    online=True,
    source=driver_hourly_stats,           # <--- Link to the raw data storage technology
    tags={"team": "driver_performance"},
)

# This groups features into a model version
driver_activity_v1 = FeatureService(
    name="driver_activity_v1",
    features=[
        # driver_hourly_stats_view[["conv_rate", "acc_rate"]],  # Sub-selects a feature from a feature view
        driver_hourly_stats_view,
    ],
    logging_config=LoggingConfig(
        destination=FileLoggingDestination(path="data")
    ),
)

Specifications

  • Version:
import feast
feast.__version__

'0.51.0'
  • Platform: MacOS M2 (15.5 (24F74))
  • Subsystem: Python 3.11

Possible Solution

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions