Skip to content

feat: support milvus #1681

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoanFM merged 16 commits into from
Jul 19, 2023
Merged

feat: support milvus #1681

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
b0c675b
chore: first commit
jupyterjazz Jul 2, 2023
0e24ce9
feat: support milvus (#1666)
maxwelljin Jul 2, 2023
d1edd96
refactor: some changes
jupyterjazz Jul 13, 2023
2113cdd
fix: merge conflicts
jupyterjazz Jul 13, 2023
d8fdb49
chore: extra name
jupyterjazz Jul 13, 2023
93f2dc0
chore: refactor extra
jupyterjazz Jul 13, 2023
06d0602
chore: install milvus
jupyterjazz Jul 13, 2023
30d133b
fix: installing pymilvus
jupyterjazz Jul 13, 2023
9ddb0ff
chore: bump tensorflow
jupyterjazz Jul 13, 2023
eff5ab1
refactor: collection loading
jupyterjazz Jul 14, 2023
da8187e
feat: subindex
jupyterjazz Jul 14, 2023
cf8aa92
feat: configurations
jupyterjazz Jul 14, 2023
87d487d
feat: query builder
jupyterjazz Jul 14, 2023
6dce316
refactor: minor adjustments
jupyterjazz Jul 14, 2023
92d05c6
refactor: adjust to recent changes
jupyterjazz Jul 17, 2023
059a8f4
chore: merge recent changes
jupyterjazz Jul 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ jobs:
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --without dev
poetry run pip install tensorflow==2.11.0
poetry run pip install tensorflow==2.12.0
poetry run pip install jax
- name: Test basic import
run: poetry run python -c 'from docarray import DocList, BaseDoc'
Expand Down Expand Up @@ -198,7 +198,7 @@ jobs:
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --all-extras
poetry run pip install protobuf==3.19.0 # we check that we support 3.19
poetry run pip install protobuf==3.20.0 # we check that we support 3.19
Copy link
Copy Markdown
Member

@samsja samsja Jul 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
poetry run pip install protobuf==3.20.0 # we check that we support 3.19
poetry run pip install protobuf==3.20.0 # we check that we support 3.20

sudo apt-get update
sudo apt-get install --no-install-recommends ffmpeg
- name: Test
Expand Down Expand Up @@ -230,7 +230,7 @@ jobs:
fail-fast: false
matrix:
python-version: [3.8]
db_test_folder: [base_classes, elastic, hnswlib, qdrant, weaviate, redis]
db_test_folder: [base_classes, elastic, hnswlib, qdrant, weaviate, redis, milvus]
steps:
- uses: actions/checkout@v2.5.0
- name: Set up Python ${{ matrix.python-version }}
Expand All @@ -242,8 +242,8 @@ jobs:
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --all-extras
poetry run pip install protobuf==3.19.0
poetry run pip install tensorflow==2.11.0
poetry run pip install protobuf==3.20.0
poetry run pip install tensorflow==2.12.0
sudo apt-get update
sudo apt-get install --no-install-recommends ffmpeg

Expand Down Expand Up @@ -287,8 +287,8 @@ jobs:
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --all-extras
poetry run pip install protobuf==3.19.0
poetry run pip install tensorflow==2.11.0
poetry run pip install protobuf==3.20.0
poetry run pip install tensorflow==2.12.0
poetry run pip install elasticsearch==8.6.2
sudo apt-get update
sudo apt-get install --no-install-recommends ffmpeg
Expand Down Expand Up @@ -332,8 +332,8 @@ jobs:
python -m pip install --upgrade pip
python -m pip install poetry
poetry install --all-extras
poetry run pip install protobuf==3.19.0
poetry run pip install tensorflow==2.11.0
poetry run pip install protobuf==3.20.0
poetry run pip install tensorflow==2.12.0
sudo apt-get update
sudo apt-get install --no-install-recommends ffmpeg

Expand Down
3 changes: 2 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -624,7 +624,7 @@ Like the [PyTorch approach](#coming-from-pytorch), you can also use DocArray wit
First off, to use DocArray with TensorFlow we first need to install it as follows:

```
pip install tensorflow==2.11.0
pip install tensorflow==2.12.0
pip install protobuf==3.19.0
```

Expand Down Expand Up @@ -879,3 +879,4 @@ Both are user-friendly and are best suited to small to medium-sized datasets.


> DocArray is a trademark of LF AI Projects, LLC
>
14 changes: 13 additions & 1 deletion docarray/index/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,17 @@
from docarray.index.backends.qdrant import QdrantDocumentIndex # noqa: F401
from docarray.index.backends.weaviate import WeaviateDocumentIndex # noqa: F401
from docarray.index.backends.redis import RedisDocumentIndex # noqa: F401
from docarray.index.backends.milvus import MilvusDocumentIndex # noqa: F401

__all__ = ['InMemoryExactNNIndex']
__all__ = [
'InMemoryExactNNIndex',
'ElasticDocIndex',
'ElasticV7DocIndex',
'QdrantDocumentIndex',
'WeaviateDocumentIndex',
'RedisDocumentIndex',
'MilvusDocumentIndex',
]


def __getattr__(name: str):
Expand All @@ -35,6 +44,9 @@ def __getattr__(name: str):
elif name == 'WeaviateDocumentIndex':
import_library('weaviate', raise_error=True)
import docarray.index.backends.weaviate as lib
elif name == 'MilvusDocumentIndex':
import_library('pymilvus', raise_error=True)
import docarray.index.backends.milvus as lib
elif name == 'RedisDocumentIndex':
import_library('redis', raise_error=True)
import docarray.index.backends.redis as lib
Expand Down
Loading