You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/migration_guide.md
+18-18Lines changed: 18 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,61 +2,61 @@
2
2
3
3
If you are using DocArray v<0.30.0, you will be familiar with its [dataclass API](https://docarray.jina.ai/fundamentals/dataclass/).
4
4
5
-
_DocArray v2 is that idea, taken seriously._ Every document is created through dataclass-like interface,
5
+
_DocArray v2 is that idea, taken seriously._ Every document is created through a dataclass-like interface,
6
6
courtesy of [Pydantic](https://pydantic-docs.helpmanual.io/usage/models/).
7
7
8
8
This gives the following advantages:
9
9
10
-
-**Flexibility:** No need to conform to a fixed set of fields -- your data defines the schema
11
-
-**Multi-modality:** Easily store multiple modalities and multiple embeddings in the same Document
12
-
-**Language agnostic:** At its core, Documents are just dictionaries. This makes it easy to create and send them from any language, not just Python.
10
+
-**Flexibility:** No need to conform to a fixed set of fields -- your data defines the schema.
11
+
-**Multi-modality:** Easily store multiple modalities and multiple embeddings in the same document.
12
+
-**Language agnostic:** At their core, documents are just dictionaries. This makes it easy to create and send them from any language, not just Python.
13
13
14
14
You may also be familiar with our old Document Stores for vector DB integration.
15
15
They are now called **Document Indexes** and offer the following improvements:
16
16
17
-
-**Hybrid search:** You can now combine vector search with text search, and even filter by arbitrary fields
18
-
-**Production-ready:** The new Document Indexes are a much thinner wrapper around the various vector DB libraries, making them more robust and easier to maintain
19
-
-**Increased flexibility:** We strive to support any configuration or setting that you could perform through the DB's first-party client
17
+
-**Hybrid search:** You can now combine vector search with text search, and even filter by arbitrary fields.
18
+
-**Production-ready:** The new Document Indexes are a much thinner wrapper around the various vector DB libraries, making them more robust and easier to maintain.
19
+
-**Increased flexibility:** We strive to support any configuration or setting that you could perform through the DB's first-party client.
20
20
21
21
For now, Document Indexes support **[Weaviate](https://weaviate.io/)**, **[Qdrant](https://qdrant.tech/)**, **[ElasticSearch](https://www.elastic.co/)**, and **[HNSWLib](https://github.com/nmslib/hnswlib)**, with more to come.
22
22
23
23
## Changes to `Document`
24
24
25
25
-`Document` has been renamed to [`BaseDoc`][docarray.BaseDoc].
26
-
-`BaseDoc`can not be used directly, but instead has to be extended. Therefore, each document class is created through a dataclass-like interface.
26
+
-`BaseDoc`cannot be used directly, but instead has to be extended. Therefore, each document class is created through a dataclass-like interface.
27
27
- Following from the previous point, the extending of `BaseDoc` allows for a flexible schema while the
28
28
`Document` class in v1 only allowed for a fixed schema, with one of `tensor`, `text` and `blob`,
29
29
and additional `chunks` and `matches`.
30
30
- Due to the added flexibility, one can not know what fields your document class will provide.
31
31
Therefore, various methods from v1 (such as `.load_uri_to_image_tensor()`) are not supported in v2.
32
-
Instead, we provide some of those methods on [typing-level](data_types/first_steps.md).
32
+
Instead, we provide some of those methods on the [typing-level](data_types/first_steps.md).
33
33
- In v2 we have the [`LegacyDocument`][docarray.documents.legacy.LegacyDocument] class,
34
34
which extends `BaseDoc` while following the same schema as v1's `Document`.
35
35
The `LegacyDocument` can be useful to start migrating your codebase from v1 to v2.
36
36
Nevertheless, the API is not fully compatible with DocArray v1 `Document`.
37
-
Indeed, none of the method associated with `Document` are present.
37
+
Indeed, none of the methods associated with `Document` are present.
38
38
Only the schema of the data is similar.
39
39
40
40
## Changes to `DocumentArray`
41
41
42
42
### DocList
43
43
44
44
- The `DocumentArray` class from v1 has been renamed to [`DocList`][docarray.array.DocList],
45
-
to be more descriptive of its actual functionality, since it is a list of `BaseDoc`s
45
+
to be more descriptive of its actual functionality, since it is a list of `BaseDoc`s.
46
46
47
47
### DocVec
48
48
49
-
- Additionally, we introduced the class [`DocVec`][docarray.array.DocVec], which is a columnbased representation of `BaseDoc`s.
49
+
- Additionally, we have introduced the class [`DocVec`][docarray.array.DocVec], which is a column-based representation of `BaseDoc`s.
50
50
Both `DocVec` and `DocList` extend `AnyDocArray`.
51
-
-`DocVec` is a container of Documents appropriates to perform computation that require batches of data
51
+
-`DocVec` is a container of Documents appropriate for performing computation that requires batches of data
52
52
(ex: matrix multiplication, distance calculation, deep learning forward pass).
53
53
- A `DocVec` has a similar interface as `DocList`
54
-
but with an underlying implementation that is columnbased instead of rowbased.
54
+
but with an underlying implementation that is column-based instead of row-based.
55
55
Each field of the schema of the `DocVec` (the `.doc_type` which is a
56
56
`BaseDoc`) will be stored in a column.
57
57
If the field is a tensor, the data from all Documents will be stored as a single
58
-
doc_vec (torch/np/tf) tensor. If the tensor field is `AnyTensor` or a Union of tensor types, the
59
-
`.tensor_type` will be used to determine the type of the doc_vec column.
58
+
`doc_vec` (torch/np/tf) tensor. If the tensor field is `AnyTensor` or a Union of tensor types, the
59
+
`.tensor_type` will be used to determine the type of the `doc_vec` column.
60
60
61
61
### Parameterized DocList
62
62
- With the added flexibility of your document schema, and therefore endless options to design your document schema,
@@ -78,7 +78,7 @@ of the attribute's name on your DocArray instance.
78
78
- In v2 you don't have to use the plural, but instead just use the document's attribute name,
79
79
since `AnyDocArray` will expose the same attributes as the `BaseDoc`s it contains.
80
80
This will return a list of `type(attribute)`.
81
-
However, this only works if (and only if) all the `BaseDoc`s in the `AnyDocArray` have the same schema. Therfore this only works
81
+
However, this works if (and only if) all the `BaseDoc`s in the `AnyDocArray` have the same schema. Therfore only this works:
82
82
83
83
```python
84
84
from docarray import BaseDoc, DocList
@@ -114,4 +114,4 @@ in v2 you can initialize a `DocIndex` object of your choice, such as:
114
114
db = HnswDocumentIndex[MyDoc](work_dir='/my/work/dir')
115
115
```
116
116
117
-
In contrast, [`DocStore`](user_guide/storing/first_step.md#document-store) in v2 can be used for simple long-term storage, such as with AWS S3 buckets or JINA AI Cloud.
117
+
In contrast, [`DocStore`](user_guide/storing/first_step.md#document-store) in v2 can be used for simple long-term storage, such as with AWS S3 buckets or Jina AI Cloud.
0 commit comments