{% hint style="warning" %} Alpha Feature: The dbt integration is currently in early development and subject to change.
Current Limitations:
- Supported data sources: BigQuery, Snowflake, and File-based sources only
- Single entity per model
- Manual entity column specification required
Breaking changes may occur in future releases. {% endhint %}
This guide explains how to use Feast's dbt integration to automatically import dbt models as Feast FeatureViews. This enables you to leverage your existing dbt transformations as feature definitions without manual duplication.
dbt (data build tool) is a popular tool for transforming data in your warehouse. Many teams already use dbt to create feature tables. Feast's dbt integration allows you to:
- Discover dbt models tagged for feature engineering
- Import model metadata (columns, types, descriptions) as Feast objects
- Generate Python code for Entity, DataSource, and FeatureView definitions
This eliminates the need to manually define Feast objects that mirror your dbt models.
- A dbt project with compiled artifacts (
target/manifest.json) - Feast installed with dbt support:
pip install 'feast[dbt]'Or install the parser directly:
pip install dbt-artifacts-parserIn your dbt project, add a feast tag to models you want to import:
{% code title="models/driver_features.sql" %}
{{ config(
materialized='table',
tags=['feast']
) }}
SELECT
driver_id,
event_timestamp,
avg_rating,
total_trips,
is_active
FROM {{ ref('stg_drivers') }}{% endcode %}
Feast uses column metadata from your schema.yml to determine feature types:
{% code title="models/schema.yml" %}
version: 2
models:
- name: driver_features
description: "Driver aggregated features for ML models"
columns:
- name: driver_id
description: "Unique driver identifier"
data_type: STRING
- name: event_timestamp
description: "Feature timestamp"
data_type: TIMESTAMP
- name: avg_rating
description: "Average driver rating"
data_type: FLOAT64
- name: total_trips
description: "Total completed trips"
data_type: INT64
- name: is_active
description: "Whether driver is currently active"
data_type: BOOLEAN{% endcode %}
cd your_dbt_project
dbt compileThis generates target/manifest.json which Feast will read.
Use the Feast CLI to discover tagged models:
feast dbt list target/manifest.json --tag-filter feastOutput:
Found 1 model(s) with tag 'feast':
driver_features
Description: Driver aggregated features for ML models
Columns: driver_id, event_timestamp, avg_rating, total_trips, is_active
Tags: feast
Generate a Python file with Feast object definitions:
feast dbt import target/manifest.json \
--entity-column driver_id \
--data-source-type bigquery \
--tag-filter feast \
--output features/driver_features.pyThis generates:
{% code title="features/driver_features.py" %}
"""
Feast feature definitions generated from dbt models.
Source: target/manifest.json
Project: my_dbt_project
Generated by: feast dbt import
"""
from datetime import timedelta
from feast import Entity, FeatureView, Field
from feast.types import Bool, Float64, Int64
from feast.infra.offline_stores.bigquery_source import BigQuerySource
# Entities
driver_id = Entity(
name="driver_id",
join_keys=["driver_id"],
description="Entity key for dbt models",
tags={'source': 'dbt'},
)
# Data Sources
driver_features_source = BigQuerySource(
name="driver_features_source",
table="my_project.my_dataset.driver_features",
timestamp_field="event_timestamp",
description="Driver aggregated features for ML models",
tags={'dbt.model': 'driver_features', 'dbt.tag.feast': 'true'},
)
# Feature Views
driver_features_fv = FeatureView(
name="driver_features",
entities=[driver_id],
ttl=timedelta(days=1),
schema=[
Field(name="avg_rating", dtype=Float64, description="Average driver rating"),
Field(name="total_trips", dtype=Int64, description="Total completed trips"),
Field(name="is_active", dtype=Bool, description="Whether driver is currently active"),
],
online=True,
source=driver_features_source,
description="Driver aggregated features for ML models",
tags={'dbt.model': 'driver_features', 'dbt.tag.feast': 'true'},
){% endcode %}
Discover dbt models available for import.
feast dbt list <manifest_path> [OPTIONS]Arguments:
manifest_path: Path to dbt'smanifest.jsonfile
Options:
--tag-filter,-t: Filter models by dbt tag (e.g.,feast)--model,-m: Filter to specific model name(s)
Import dbt models as Feast object definitions.
feast dbt import <manifest_path> [OPTIONS]Arguments:
manifest_path: Path to dbt'smanifest.jsonfile
Options:
| Option | Description | Default |
|---|---|---|
--entity-column, -e |
Column to use as entity key | (required) |
--data-source-type, -d |
Data source type: bigquery, snowflake, file |
bigquery |
--tag-filter, -t |
Filter models by dbt tag | None |
--model, -m |
Import specific model(s) only | None |
--timestamp-field |
Timestamp column name | event_timestamp |
--ttl-days |
Feature TTL in days | 1 |
--exclude-columns |
Columns to exclude from features | None |
--no-online |
Disable online serving | False |
--output, -o |
Output Python file path | None (stdout) |
--dry-run |
Preview without generating code | False |
Feast automatically maps dbt/warehouse column types to Feast types:
| dbt/SQL Type | Feast Type |
|---|---|
STRING, VARCHAR, TEXT |
String |
INT, INTEGER, BIGINT |
Int64 |
SMALLINT, TINYINT |
Int32 |
FLOAT, REAL |
Float32 |
DOUBLE, FLOAT64 |
Float64 |
BOOLEAN, BOOL |
Bool |
TIMESTAMP, DATETIME |
UnixTimestamp |
BYTES, BINARY |
Bytes |
ARRAY<type> |
Array(type) |
Snowflake NUMBER(precision, scale) types are handled specially:
- Scale > 0:
Float64 - Precision <= 9:
Int32 - Precision <= 18:
Int64 - Precision > 18:
Float64
feast dbt import manifest.json -e user_id -d bigquery -o features.pyGenerates BigQuerySource with the full table path from dbt metadata:
BigQuerySource(
table="project.dataset.table_name",
...
)feast dbt import manifest.json -e user_id -d snowflake -o features.pyGenerates SnowflakeSource with database, schema, and table:
SnowflakeSource(
database="MY_DB",
schema="MY_SCHEMA",
table="TABLE_NAME",
...
)feast dbt import manifest.json -e user_id -d file -o features.pyGenerates FileSource with a placeholder path:
FileSource(
path="/data/table_name.parquet",
...
){% hint style="info" %} For file sources, update the generated path to point to your actual data files. {% endhint %}
Create a standard tagging convention in your dbt project:
# dbt_project.yml
models:
my_project:
features:
+tags: ['feast'] # All models in features/ get the feast tagColumn descriptions from schema.yml are preserved in the generated Feast definitions, making your feature catalog self-documenting.
Use --dry-run to preview what will be generated:
feast dbt import manifest.json -e user_id -d bigquery --dry-runCommit the generated Python files to your repository. This allows you to:
- Track changes to feature definitions over time
- Review dbt-to-Feast mapping in pull requests
- Customize generated code if needed
Add dbt import to your CI pipeline:
# .github/workflows/features.yml
- name: Compile dbt
run: dbt compile
- name: Generate Feast definitions
run: |
feast dbt import target/manifest.json \
-e user_id -d bigquery -t feast \
-o feature_repo/features.py
- name: Apply Feast changes
run: feast apply- Single entity support: Currently supports one entity column per import. For multi-entity models, run multiple imports or manually adjust the generated code.
- No incremental updates: Each import generates a complete file. Use version control to track changes.
- Column types required: Models without
data_typein schema.yml default toStringtype.
Run dbt compile or dbt run first to generate the manifest file.
Check that your models have the correct tag in their config:
{{ config(tags=['feast']) }}Ensure your dbt model includes the entity column specified with --entity-column. Models missing this column are skipped with a warning.
By default, Feast looks for event_timestamp. Use --timestamp-field to specify a different column name.