Skip to content

chore: Upgrade dask dependency #4050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 6 commits into from
Apr 16, 2024
Merged

chore: Upgrade dask dependency #4050

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d1b7ca0
add dataframe extra to dask
tokoko Mar 28, 2024
abde690
Merge branch 'master' into dask-expr
tokoko Apr 3, 2024
0569fb1
update pinned versions
tokoko Apr 9, 2024
ab23495
Merge branch 'feast-dev:master' into dask-expr
tokoko Apr 9, 2024
9de4f18
add ignore rules to mssql, it's too broken
tokoko Apr 9, 2024
6a9811b
ruff reformat
tokoko Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/python/feast/infra/offline_stores/contrib/mssql_offline_store/mssql.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -423,7 +423,7 @@ def _upload_entity_df_into_sqlserver_and_get_entity_schema(

elif isinstance(entity_df, pandas.DataFrame):
# Drop the index so that we don't have unnecessary columns
engine.execute(_df_to_create_table_sql(entity_df, table_id))
engine.execute(_df_to_create_table_sql(entity_df, table_id)) # type: ignore

@HaoXuAI HaoXuAI Apr 15, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why suddenly adding the type ignore?

@tokoko tokoko Apr 15, 2024

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not exactly sure tbh, but probably something related to this #4066. We broke mssql with sqlalchemy upgrade, it passed the ci, because we are not testing contribs in the ci. in other words, it's already broken with or without the type check.

entity_df.to_sql(name=table_id, con=engine, index=False, if_exists="append")
entity_schema = dict(zip(entity_df.columns, entity_df.dtypes)), table_id

Expand Down
2 changes: 1 addition & 1 deletion sdk/python/feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ def create_data_source(
engine = create_engine(connection_string)
destination_name = self.get_prefixed_table_name(destination_name)
# Create table
engine.execute(_df_to_create_table_sql(df, destination_name))
engine.execute(_df_to_create_table_sql(df, destination_name)) # type: ignore

# Upload dataframe to azure table
df.to_sql(destination_name, engine, index=False, if_exists="append")
Expand Down
88 changes: 48 additions & 40 deletions sdk/python/requirements/py3.10-ci-requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,6 @@ anyio==4.3.0
# watchfiles
appdirs==1.4.4
# via fissix
appnope==0.1.4
# via ipykernel
argon2-cffi==23.1.0
# via jupyter-server
argon2-cffi-bindings==21.2.0
Expand All @@ -36,7 +34,7 @@ async-lru==2.0.4
# via jupyterlab
async-timeout==4.0.3
# via redis
atpublic==4.0
atpublic==4.1.0
# via ibis-framework
attrs==23.2.0
# via
Expand All @@ -61,18 +59,18 @@ bidict==0.23.1
# via ibis-framework
bleach==6.1.0
# via nbconvert
boto3==1.34.69
boto3==1.34.80
# via
# feast (setup.py)
# moto
botocore==1.34.69
botocore==1.34.80
# via
# boto3
# moto
# s3transfer
bowler==0.9.0
# via feast (setup.py)
build==1.1.1
build==1.2.1
# via
# feast (setup.py)
# pip-tools
Expand Down Expand Up @@ -138,8 +136,12 @@ cryptography==42.0.5
# snowflake-connector-python
# types-pyopenssl
# types-redis
dask==2024.2.1
# via feast (setup.py)
dask[array,dataframe]==2024.4.1
# via
# dask-expr
# feast (setup.py)
dask-expr==1.0.10
# via dask
db-dtypes==1.2.0
# via google-cloud-bigquery
debugpy==1.8.1
Expand Down Expand Up @@ -167,7 +169,7 @@ duckdb==0.10.1
# via
# duckdb-engine
# ibis-framework
duckdb-engine==0.11.2
duckdb-engine==0.11.4
# via ibis-framework
entrypoints==0.4
# via altair
Expand All @@ -176,15 +178,15 @@ exceptiongroup==1.2.0
# anyio
# ipython
# pytest
execnet==2.0.2
execnet==2.1.1
# via pytest-xdist
executing==2.0.1
# via stack-data
fastapi==0.110.0
fastapi==0.110.1
# via feast (setup.py)
fastjsonschema==2.19.1
# via nbformat
filelock==3.13.1
filelock==3.13.3
# via
# snowflake-connector-python
# virtualenv
Expand Down Expand Up @@ -214,14 +216,15 @@ google-api-core[grpc]==2.18.0
# google-cloud-datastore
# google-cloud-firestore
# google-cloud-storage
google-api-python-client==2.123.0
google-api-python-client==2.125.0
# via firebase-admin
google-auth==2.29.0
# via
# google-api-core
# google-api-python-client
# google-auth-httplib2
# google-cloud-core
# google-cloud-firestore
# google-cloud-storage
# kubernetes
google-auth-httplib2==0.2.0
Expand All @@ -241,7 +244,7 @@ google-cloud-core==2.4.1
# google-cloud-storage
google-cloud-datastore==2.19.0
# via feast (setup.py)
google-cloud-firestore==2.15.0
google-cloud-firestore==2.16.0
# via firebase-admin
google-cloud-storage==2.16.0
# via
Expand Down Expand Up @@ -289,7 +292,7 @@ grpcio-testing==1.62.1
# via feast (setup.py)
grpcio-tools==1.62.1
# via feast (setup.py)
gunicorn==21.2.0
gunicorn==21.2.0 ; platform_system != "Windows"
# via feast (setup.py)
h11==0.14.0
# via
Expand All @@ -301,7 +304,7 @@ hazelcast-python-client==5.3.0
# via feast (setup.py)
hiredis==2.3.2
# via feast (setup.py)
httpcore==1.0.4
httpcore==1.0.5
# via httpx
httplib2==0.22.0
# via
Expand Down Expand Up @@ -338,9 +341,9 @@ importlib-resources==6.4.0
# via feast (setup.py)
iniconfig==2.0.0
# via pytest
ipykernel==6.29.3
ipykernel==6.29.4
# via jupyterlab
ipython==8.22.2
ipython==8.23.0
# via
# great-expectations
# ipykernel
Expand Down Expand Up @@ -413,11 +416,11 @@ jupyter-server==2.13.0
# notebook-shim
jupyter-server-terminals==0.5.3
# via jupyter-server
jupyterlab==4.1.5
jupyterlab==4.1.6
# via notebook
jupyterlab-pygments==0.3.0
# via nbconvert
jupyterlab-server==2.25.4
jupyterlab-server==2.26.0
# via
# jupyterlab
# notebook
Expand Down Expand Up @@ -482,7 +485,7 @@ nbclient==0.10.0
# via nbconvert
nbconvert==7.16.3
# via jupyter-server
nbformat==5.10.3
nbformat==5.10.4
# via
# great-expectations
# jupyter-server
Expand All @@ -501,6 +504,7 @@ notebook-shim==0.2.4
numpy==1.24.4
# via
# altair
# dask
# db-dtypes
# feast (setup.py)
# great-expectations
Expand All @@ -512,13 +516,14 @@ oauthlib==3.2.2
# via requests-oauthlib
overrides==7.7.0
# via jupyter-server
packaging==24.0
packaging==21.3
# via
# build
# dask
# db-dtypes
# deprecation
# docker
# duckdb-engine
# google-cloud-bigquery
# great-expectations
# gunicorn
Expand All @@ -536,6 +541,8 @@ packaging==24.0
pandas==2.2.1
# via
# altair
# dask
# dask-expr
# db-dtypes
# feast (setup.py)
# google-cloud-bigquery
Expand All @@ -544,7 +551,7 @@ pandas==2.2.1
# snowflake-connector-python
pandocfilters==1.5.1
# via nbconvert
parso==0.8.3
parso==0.8.4
# via jedi
parsy==2.1
# via ibis-framework
Expand Down Expand Up @@ -620,22 +627,23 @@ py4j==0.10.9.7
# via pyspark
pyarrow==15.0.2
# via
# dask-expr
# db-dtypes
# feast (setup.py)
# google-cloud-bigquery
# ibis-framework
# snowflake-connector-python
pyarrow-hotfix==0.6
# via ibis-framework
pyasn1==0.5.1
pyasn1==0.6.0
# via
# pyasn1-modules
# rsa
pyasn1-modules==0.3.0
pyasn1-modules==0.4.0
# via google-auth
pybindgen==0.22.1
# via feast (setup.py)
pycparser==2.21
pycparser==2.22
# via cffi
pydantic==2.6.4
# via
Expand All @@ -655,7 +663,7 @@ pyjwt[crypto]==2.8.0
# via
# msal
# snowflake-connector-python
pymssql==2.2.11
pymssql==2.3.0
# via feast (setup.py)
pymysql==1.1.0
# via feast (setup.py)
Expand All @@ -667,6 +675,7 @@ pyparsing==3.1.2
# via
# great-expectations
# httplib2
# packaging
pyproject-hooks==1.0.0
# via
# build
Expand All @@ -686,7 +695,7 @@ pytest==7.4.4
# pytest-xdist
pytest-benchmark==3.4.1
# via feast (setup.py)
pytest-cov==4.1.0
pytest-cov==5.0.0
# via feast (setup.py)
pytest-env==1.1.3
# via feast (setup.py)
Expand Down Expand Up @@ -791,13 +800,13 @@ rsa==4.9
# via google-auth
ruamel-yaml==0.17.17
# via great-expectations
ruff==0.3.4
ruff==0.3.5
# via feast (setup.py)
s3transfer==0.10.1
# via boto3
scipy==1.12.0
scipy==1.13.0
# via great-expectations
send2trash==1.8.2
send2trash==1.8.3
# via jupyter-server
six==1.16.0
# via
Expand Down Expand Up @@ -838,7 +847,7 @@ sphinxcontrib-qthelp==1.0.7
# via sphinx
sphinxcontrib-serializinghtml==1.1.10
# via sphinx
sqlalchemy[mypy]==1.4.52
sqlalchemy[mypy]==2.0.29
# via
# duckdb-engine
# feast (setup.py)
Expand All @@ -847,15 +856,13 @@ sqlalchemy[mypy]==1.4.52
# sqlalchemy-views
sqlalchemy-views==0.3.2
# via ibis-framework
sqlalchemy2-stubs==0.0.2a38
# via sqlalchemy
sqlglot==20.11.0
# via ibis-framework
stack-data==0.6.3
# via ipython
starlette==0.36.3
starlette==0.37.2
# via fastapi
substrait==0.14.1
substrait==0.15.0
# via ibis-substrait
tabulate==0.9.0
# via feast (setup.py)
Expand Down Expand Up @@ -938,7 +945,7 @@ types-pytz==2024.1.0.20240203
# via feast (setup.py)
types-pyyaml==6.0.12.20240311
# via feast (setup.py)
types-redis==4.6.0.20240311
types-redis==4.6.0.20240409
# via feast (setup.py)
types-requests==2.30.0.0
# via feast (setup.py)
Expand All @@ -948,7 +955,7 @@ types-tabulate==0.9.0.20240106
# via feast (setup.py)
types-urllib3==1.26.25.14
# via types-requests
typing-extensions==4.10.0
typing-extensions==4.11.0
# via
# anyio
# async-lru
Expand All @@ -957,11 +964,12 @@ typing-extensions==4.10.0
# fastapi
# great-expectations
# ibis-framework
# ipython
# mypy
# pydantic
# pydantic-core
# snowflake-connector-python
# sqlalchemy2-stubs
# sqlalchemy
# typeguard
# uvicorn
tzdata==2024.1
Expand Down Expand Up @@ -1011,7 +1019,7 @@ websocket-client==1.7.0
# kubernetes
websockets==12.0
# via uvicorn
werkzeug==3.0.1
werkzeug==3.0.2
# via moto
wheel==0.43.0
# via pip-tools
Expand Down
Loading