Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
cc37a3d
Add ScyllaDB docs
zseta Jun 20, 2023
c25c62b
Formatting
zseta Jun 20, 2023
a2723fd
ci: Add bigtable cleanup script
adchia Jul 3, 2023
da96fd1
fix: Missing Catalog argument in athena connector (#3661)
GyuminJack Jul 3, 2023
d3d2d3f
ci: Disable flaky lambda materialization test
adchia Jul 3, 2023
3790e18
fix: Broken non-root path with projects-list.json (#3665)
bjfletcher Jul 3, 2023
15d425f
fix: Manage redis pipe's context (#3655)
jiuuon Jul 4, 2023
b6b74ba
chore: Bump tough-cookie from 4.0.0 to 4.1.3 in /sdk/python/feast/ui …
dependabot[bot] Jul 11, 2023
f671c0a
chore: Bump tough-cookie from 4.0.0 to 4.1.3 in /ui (#3676)
dependabot[bot] Jul 11, 2023
6bfbc87
fix: For SQL registry, increase max data_source_name length to 255 (#…
radonnachie Jul 13, 2023
f776400
fix: Optimize bytes processed when retrieving entity df schema to 0 (…
sudohainguyen Jul 13, 2023
f604e40
fix: Entityless fv breaks with `KeyError: __dummy` applying feature_s…
wfoschiera Jul 13, 2023
79e8fc8
chore: Bump protobufjs from 7.1.1 to 7.2.4 in /ui (#3674)
dependabot[bot] Jul 17, 2023
854ea8b
chore: Bump protobufjs from 7.1.2 to 7.2.4 in /sdk/python/feast/ui (#…
dependabot[bot] Jul 17, 2023
aa235f8
chore: Bump semver from 6.3.0 to 6.3.1 in /ui (#3678)
dependabot[bot] Jul 17, 2023
2474246
chore: Bump semver from 6.3.0 to 6.3.1 in /sdk/python/feast/ui (#3679)
dependabot[bot] Jul 17, 2023
b879056
chore: Bump google.golang.org/grpc from 1.47.0 to 1.53.0 (#3670)
dependabot[bot] Jul 17, 2023
82e4c7f
chore(release): release 0.32.0
feast-ci-bot Jul 17, 2023
fb627d3
fix: Redshift push ignores schema (#3671)
metavee Jul 24, 2023
d6c6e73
fix: Add aws-sts dependency in java sdk so that S3 client acquires IR…
harmeet-singh-discovery Aug 1, 2023
c37377e
Switch from `macos-10.15` to `macos-latest` (#3722)
felixwang9817 Aug 14, 2023
ece2754
ci: Upgrade `checkout` Github Action (#3723)
felixwang9817 Aug 14, 2023
111c7ae
ci: Upgrade `setup-python` Github Action (#3724)
felixwang9817 Aug 14, 2023
14b7eb7
ci: Upgrade `setup-node` Github Action (#3725)
felixwang9817 Aug 14, 2023
4eb080b
ci: Upgrade `upload-artifact` Github Action (#3726)
felixwang9817 Aug 14, 2023
22b0254
feat: Add possibility to save dataset as table, when spark config has…
nadejdaSuraeva Aug 14, 2023
731bea8
chore(release): release 0.33.0
feast-ci-bot Aug 14, 2023
366fd33
ci: Temporarily disable smoke tests (#3727)
felixwang9817 Aug 14, 2023
db80a2b
fix: Run store.plan() only when need it. (#3708)
shuchu Aug 14, 2023
f5d7f04
fix: Fix python unit tests (#3734)
crispin-ki Aug 16, 2023
20995a9
fix: Update py3.8 ci requirements for cython 3.0 release (#3735)
crispin-ki Aug 28, 2023
23c6683
fix: Pin protobuf version to avoid seg fault on some machines
adchia Aug 28, 2023
2cc2e5a
ci: Pin pip version to avoid test breakage
adchia Aug 28, 2023
a7e5b33
ci: Pin pip version to slightly higher to avoid test breakage
adchia Aug 28, 2023
67a3f77
ci: Fix ci breakage from old pip version being restored from pip cache
adchia Sep 5, 2023
8e2a3b4
fix: Handle unknown postgres source types gracefully (#3634)
msnidal Sep 5, 2023
1966cde
chore: Fix linting issue
adchia Sep 5, 2023
c671cbd
docs: Add Dragonfly as an online-store option to Feast Readme and doc…
yirenlu92 Sep 5, 2023
96f4851
fix: Remove unwanted excessive splitting of gcs path, so expected gcs…
crispin-ki Sep 5, 2023
30e96ab
feat: Implement gRPC server to ingest streaming features (#3687)
mehmettokgoz Sep 7, 2023
a6446d7
fix: Add NUMERIC to bq_to_feast type map (#3719)
sudohainguyen Sep 7, 2023
4951289
chore: Widen redis requirement to make it easier to install alongside…
breno-costa Sep 7, 2023
46f54b5
fix: Saved datasets no longer break CLI registry-dump command (#3717)
malcolmk181 Sep 7, 2023
ae67885
feat: Enhance customization of Trino connections when using Trino-bas…
boliri Sep 7, 2023
08e11b9
chore(release): release 0.34.0
feast-ci-bot Sep 7, 2023
52eec86
fix: Fix warnings from deprecated paths and update default log level …
adchia Sep 7, 2023
66bbfd2
fix: Set keepalives_idle None by default (#3756)
ndemashov Sep 7, 2023
787dbd8
fix: Pin numpy version to avoid spammy deprecation messages
adchia Sep 7, 2023
2c1f4b3
feat: Apply cache to load proto registry for performance (#3702)
jiuuon Sep 12, 2023
c1873c7
feat: Add materialize and materialize-incremental rest endpoints (#3761)
snowron Sep 12, 2023
696b085
feat: Add support for `table_create_disposition` in bigquery job for …
nickozilla Sep 13, 2023
eb0fc38
chore: Updating docs to fix some outdated assets (e.g. release process)
adchia Sep 21, 2023
22a5b41
feat: Add support for in_cluster config and additional labels for byt…
james-crabtree-sp Sep 21, 2023
1ab82c9
chore: Bump org.apache.avro:avro from 1.10.2 to 1.11.3 in /java/servi…
dependabot[bot] Oct 1, 2023
d346a93
chore: Bump cryptography from 41.0.3 to 41.0.4 in /sdk/python/require…
dependabot[bot] Oct 1, 2023
8ce3ed6
fix: UI project cannot handle fallback routes (#3766)
snowron Oct 11, 2023
d1eb37e
fix: Update jackson and remove unnecessary logging (#3809)
achals Oct 20, 2023
1b24b5d
chore: Add harry to CODEOWNERS (#3808)
achals Oct 20, 2023
dc1d3a1
chore: Bump golang.org/x/net from 0.7.0 to 0.17.0 (#3792)
dependabot[bot] Oct 20, 2023
9651d5f
chore: Bump @babel/traverse from 7.17.10 to 7.23.2 in /sdk/python/fea…
dependabot[bot] Oct 20, 2023
874cc5b
chore: Bump avro from 1.10.0 to 1.11.3 in /sdk/python/requirements (#…
dependabot[bot] Oct 20, 2023
c1513c1
feat: Support GCS filesystem for bytewax engine (#3774)
sudohainguyen Oct 20, 2023
4c2d7f5
feat: Make bytewax job write as mini-batches (#3777)
sudohainguyen Oct 20, 2023
8cd9045
fix: Resolve hbase hotspot issue when materializing (#3790)
sudohainguyen Oct 20, 2023
012e899
fix: Make generated temp table name escaped (#3797)
BearDimonR Oct 20, 2023
7285d54
fix: Adopt connection pooling for HBase (#3793)
sudohainguyen Oct 21, 2023
75dbaef
fix: Redundant feature materialization and premature incremental mate…
james-crabtree-sp Oct 25, 2023
ee0c7cb
chore: Bump urllib3 from 1.26.16 to 1.26.18 in /sdk/python/requiremen…
dependabot[bot] Oct 25, 2023
bc7b584
feat: Add get online feature rpc to gprc server (#3815)
sudohainguyen Oct 31, 2023
57d2c63
fix: Add async refresh to prevent synchronous refresh in main thread …
sudohainguyen Oct 31, 2023
7611aa8
fix: Bytewax engine create configmap from object (#3821)
sudohainguyen Nov 8, 2023
84fdacf
fix: Set upper bound for bigquery client due to its breaking changes
sudohainguyen Nov 4, 2023
5fbc6ad
fix: update dependencies versions due to conflicts
sudohainguyen Nov 8, 2023
a241834
chore: generate reqs file from up-to-date deps
sudohainguyen Nov 8, 2023
1278b35
feat: Optimize bytewax pod resource with zero-copy
sudohainguyen Nov 4, 2023
d900c55
fix: improve parsing bytewax job status
sudohainguyen Nov 6, 2023
2a16bd7
fix: make bytewax settings unexposed
sudohainguyen Nov 8, 2023
69bcd58
feat: add redis sentinel support
snowron Nov 2, 2023
02b9242
feat: add redis sentinel support
snowron Nov 2, 2023
cdab45d
feat: add redis sentinel support format lint
snowron Nov 2, 2023
b8001f0
chore: Bump pyarrow
dependabot[bot] Nov 9, 2023
68851f5
fix: upgrade the pyarrow to latest v14.0.1 for CVE-2023-47248.
shuchu Nov 17, 2023
99ded6c
chore: Bump follow-redirects from 1.15.0 to 1.15.4 in /sdk/python/fea…
dependabot[bot] Jan 10, 2024
288a267
chore: Bump follow-redirects from 1.14.7 to 1.15.4 in /ui (#3878)
dependabot[bot] Jan 10, 2024
71cf78b
Update release.yml to latest nodejs
woop Jan 13, 2024
ef5ff9e
chore(release): release 0.35.0
feast-ci-bot Jan 13, 2024
6515c98
feat: Add python bytes to array type conversion support proto (#3874)
robhowley Jan 15, 2024
97425b8
chore: Bump jupyter-server from 2.10.1 to 2.11.2 in /sdk/python/requi…
dependabot[bot] Jan 15, 2024
c30f190
chore: Bump jinja2 from 3.1.2 to 3.1.3 in /sdk/python/requirements (#…
dependabot[bot] Jan 16, 2024
7ea9cb7
Move maintainers to emeritus (#3888)
woop Jan 16, 2024
86de725
chore: Bump jupyter-lsp from 2.2.0 to 2.2.2 in /sdk/python/requiremen…
dependabot[bot] Jan 19, 2024
7b21c84
chore: Bump cryptography from 41.0.5 to 41.0.6 in /sdk/python/require…
dependabot[bot] Jan 19, 2024
342464f
fix: Allow trancated timestamps when converting (#3861)
jiuuon Jan 19, 2024
59e2126
chore: Bump zod from 3.15.1 to 3.22.3 in /sdk/python/feast/ui (#3816)
dependabot[bot] Jan 22, 2024
efbb1e3
chore: Bump google.golang.org/grpc from 1.53.0 to 1.56.3 (#3820)
dependabot[bot] Jan 22, 2024
261c2e3
chore: Bump @babel/traverse from 7.16.10 to 7.23.2 in /ui (#3801)
dependabot[bot] Jan 22, 2024
4e6e23f
chore: Bump zod from 3.19.1 to 3.22.3 in /ui (#3817)
dependabot[bot] Jan 22, 2024
a5a8788
fix: Correct the returning class proto type of StreamFeatureView to S…
shuchu Jan 24, 2024
cc313a8
fix: Verify the existence of Registry tables in snowflake before call…
shuchu Jan 24, 2024
56b8768
revert: Verify the existence of Registry tables in snowflake… (#3907)
etirelli Jan 25, 2024
41b0a35
feat: Add support for arrays in snowflake (#3769)
JohnLemmonMedely Jan 25, 2024
38157f6
feat: Support s3gov schema by snowflake offline store during material…
alex-vinnik-sp Jan 27, 2024
8e851d9
fix: Create index only if not exists during MySQL online store update…
gee-senbong Jan 30, 2024
7c3bd2e
fix: Bytewax materialization engine fails when loading feature_store.…
marti-jorda-roca Jan 30, 2024
61a3dbe
Update maintainers.md (#3918)
woop Jan 30, 2024
a1cbe38
docs: Updating maintainers.md (#3919)
franciscojavierarceo Jan 30, 2024
125cb1c
docs: Updated development guide to include compiling the protos (#3896)
franciscojavierarceo Jan 31, 2024
2b72364
fix: Prevent spamming pull busybox from dockerhub (#3923)
sudohainguyen Feb 2, 2024
6393ab0
chore: Bump jupyterlab from 4.0.8 to 4.0.11 in /sdk/python/requiremen…
dependabot[bot] Feb 4, 2024
d9b2cd2
fix: Remove unnecessary dependency on mysqlclient (#3925)
tokoko Feb 4, 2024
3f22fa8
ci: Extend python base version for test cases (#3929)
sudohainguyen Feb 6, 2024
ce94503
feat: Add gRPC Registry Server (#3924)
tokoko Feb 6, 2024
c646d03
chore: Set upper bound for moto package (#3937)
sudohainguyen Feb 7, 2024
1f28c6d
Merge branch 'feast-dev:master' into docs-scylladb
zseta Feb 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@
* [MySQL (contrib)](reference/online-stores/mysql.md)
* [Rockset (contrib)](reference/online-stores/rockset.md)
* [Hazelcast (contrib)](reference/online-stores/hazelcast.md)
* [ScyllaDB (contrib)](reference/online-stores/scylladb.md)
* [Providers](reference/providers/README.md)
* [Local](reference/providers/local.md)
* [Google Cloud Platform](reference/providers/google-cloud-platform.md)
Expand Down
3 changes: 3 additions & 0 deletions docs/reference/online-stores/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,7 @@ Please see [Online Store](../../getting-started/architecture-and-components/onli
[hazelcast.md](hazelcast.md)
{% endcontent-ref %}

{% content-ref url="scylladb.md" %}
[scylladb.md](scylladb.md)
{% endcontent-ref %}

94 changes: 94 additions & 0 deletions docs/reference/online-stores/scylladb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# ScyllaDB Cloud online store

## Description

ScyllaDB is a low-latency and high-performance Cassandra-compatible (uses CQL) database. You can use the existing Cassandra connector to use ScyllaDB as an online store in Feast.

The [ScyllaDB](https://www.scylladb.com/) online store provides support for materializing feature values into a ScyllaDB or [ScyllaDB Cloud](https://www.scylladb.com/product/scylla-cloud/) cluster for serving online features real-time.

## Getting started

Install Feast with Cassandra support:
```bash
pip install "feast[cassandra]"
```

Create a new Feast project:
```bash
feast init REPO_NAME -t cassandra
```

### Example (ScyllaDB)

{% code title="feature_store.yaml" %}
```yaml
project: scylla_feature_repo
registry: data/registry.db
provider: local
online_store:
type: cassandra
hosts:
- 172.17.0.2
keyspace: feast
username: scylla
password: password
```
{% endcode %}

### Example (ScyllaDB Cloud)

{% code title="feature_store.yaml" %}
```yaml
project: scylla_feature_repo
registry: data/registry.db
provider: local
online_store:
type: cassandra
hosts:
- node-0.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
- node-1.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
- node-2.aws_us_east_1.xxxxxxxx.clusters.scylla.cloud
keyspace: feast
username: scylla
password: password
```
{% endcode %}


The full set of configuration options is available in [CassandraOnlineStoreConfig](https://rtd.feast.dev/en/master/#feast.infra.online_stores.contrib.cassandra_online_store.cassandra_online_store.CassandraOnlineStoreConfig).
For a full explanation of configuration options please look at file
`sdk/python/feast/infra/online_stores/contrib/cassandra_online_store/README.md`.

Storage specifications can be found at `docs/specs/online_store_format.md`.

## Functionality Matrix

The set of functionality supported by online stores is described in detail [here](overview.md#functionality).
Below is a matrix indicating which functionality is supported by the Cassandra plugin.

| | Cassandra |
| :-------------------------------------------------------- | :-------- |
| write feature values to the online store | yes |
| read feature values from the online store | yes |
| update infrastructure (e.g. tables) in the online store | yes |
| teardown infrastructure (e.g. tables) in the online store | yes |
| generate a plan of infrastructure changes | yes |
| support for on-demand transforms | yes |
| readable by Python SDK | yes |
| readable by Java | no |
| readable by Go | no |
| support for entityless feature views | yes |
| support for concurrent writing to the same key | no |
| support for ttl (time to live) at retrieval | no |
| support for deleting expired data | no |
| collocated by feature view | yes |
| collocated by feature service | no |
| collocated by entity key | no |

To compare this set of functionality against other online stores, please see the full [functionality matrix](overview.md#functionality-matrix).

## Resources

* [Sample application with ScyllaDB](https://feature-store.scylladb.com/stable/)
* [ScyllaDB website](https://www.scylladb.com/)
* [ScyllaDB Cloud documentation](https://cloud.docs.scylladb.com/stable/)