feat: make compatible based type with fastapi by samsja · Pull Request #809 · docarray/docarray

samsja · 2022-11-17T16:10:49Z

Context

The goal of this PR is make our Document model compatible with FastAPI.

What this PR do:

Mainly add json dump to types

allow to do

from fastapi import FastAPI

from docarray import Document, Image, Text
from docarray.typing import Tensor


class InputDoc(Document):
    text: str

class OutputDoc(Document):
    embedding_clip: Tensor
    embedding_bert: Tensor

input_doc = InputDoc(text='hello')

app = FastAPI()

@app.post("/doc/", response_model=OutputDoc)
async def create_item(doc: InputDoc) -> OutputDoc:
    ## call my fancy model to generate the embeddings
    return OutputDoc(
        embedding_clip=np.zeros((100, 1)), embedding_bert=np.zeros((100, 1))
    )

and it is working as well with our custom types

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: add tensor type for ndarray * fix: fix mypy typing * feat: torch tensor type Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: protobuf for pytorch type Signed-off-by: Johannes Messner <messnerjo@gmail.com> * ci: install all extras in the ci Signed-off-by: Johannes Messner <messnerjo@gmail.com> * refactor: make nice looking * docs: update docarray/typing/tensor/torch_tensor.py Co-authored-by: samsja <55492238+samsja@users.noreply.github.com> Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> * refactor: code style Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: black and mypy Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: suppress mypy import error * ci: fix ci install Signed-off-by: Johannes Messner <messnerjo@gmail.com> Signed-off-by: Johannes Messner <messnerjo@gmail.com> Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Co-authored-by: samsja <55492238+samsja@users.noreply.github.com>

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

* feat: allow da bulk access to return da for document Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: fix mypy type pb Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: add link to the mypy issue Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: remove useless list type hint Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat: torch tensor type (#800) * feat: add tensor type for ndarray * fix: fix mypy typing * feat: torch tensor type Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: protobuf for pytorch type Signed-off-by: Johannes Messner <messnerjo@gmail.com> * ci: install all extras in the ci Signed-off-by: Johannes Messner <messnerjo@gmail.com> * refactor: make nice looking * docs: update docarray/typing/tensor/torch_tensor.py Co-authored-by: samsja <55492238+samsja@users.noreply.github.com> Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> * refactor: code style Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: black and mypy Signed-off-by: Johannes Messner <messnerjo@gmail.com> * fix: suppress mypy import error * ci: fix ci install Signed-off-by: Johannes Messner <messnerjo@gmail.com> Signed-off-by: Johannes Messner <messnerjo@gmail.com> Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Co-authored-by: samsja <55492238+samsja@users.noreply.github.com> * feat: add fastapi to dependency Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat(wip): add fake method to dump tensor to json Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat(wip): add fastapi test Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat: add json dump for type and document Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat: add json compatible with orjson Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * refactor: clean tests Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: remove duplicate Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: better json schema for tensor Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: fix fast api test Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * refactor: move to json test to integration Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: json laod from tensor type now working Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: add tensor to fastapi test Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: add new fastapi test Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: fix mypy Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * feat: add more testing for text uri about json Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: fix default orson not returning Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: apply johannes suggestion on docstring Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: does not perform copy anymore on torch tensor unwrap Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: add johannes suggestion Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Signed-off-by: samsja <55492238+samsja@users.noreply.github.com> * fix: fix ruff line lenght Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> * fix: fix mypy pb Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr> Signed-off-by: Johannes Messner <messnerjo@gmail.com> Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Signed-off-by: samsja <55492238+samsja@users.noreply.github.com> Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>

samsja marked this pull request as draft November 17, 2022 16:10

samsja changed the base branch from main to feat-rewrite-v2 November 17, 2022 16:11

samsja and others added 8 commits November 17, 2022 18:04

feat: allow da bulk access to return da for document

77a2055

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: fix mypy type pb

bd6d128

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: add link to the mypy issue

d778cec

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: remove useless list type hint

717e7ee

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

feat: add fastapi to dependency

e5d678c

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

feat(wip): add fake method to dump tensor to json

414a04c

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

feat(wip): add fastapi test

42c6fe4

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

samsja force-pushed the feat-v2-fastapi-test branch from a652aaa to 42c6fe4 Compare November 17, 2022 17:04

feat: add json dump for type and document

56497b9

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

samsja force-pushed the feat-v2-fastapi-test branch from 3a894fa to 56497b9 Compare November 18, 2022 14:23

samsja added 8 commits November 21, 2022 20:09

feat: add json compatible with orjson

945a72a

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

refactor: clean tests

83236cd

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: remove duplicate

f1a4d7b

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: better json schema for tensor

c983246

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: fix fast api test

3873a9a

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

refactor: move to json test to integration

51a402d

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: json laod from tensor type now working

2e92866

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: add tensor to fastapi test

606134b

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

samsja force-pushed the feat-v2-fastapi-test branch from 100916c to 87bab05 Compare November 22, 2022 13:43

fix: add new fastapi test

731feff

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

samsja force-pushed the feat-v2-fastapi-test branch from 87bab05 to 731feff Compare November 22, 2022 13:50

samsja added 3 commits November 22, 2022 15:28

Merge branch 'feat-rewrite-v2' into feat-v2-fastapi-test

e114b91

fix: fix mypy

061b05e

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

feat: add more testing for text uri about json

94c0069

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

samsja marked this pull request as ready for review November 22, 2022 14:47

dongxiang123 approved these changes Nov 22, 2022

View reviewed changes

JohannesMessner requested changes Nov 22, 2022

View reviewed changes

Comment thread docarray/document/io/json.py

Comment thread docarray/typing/tensor/tensor.py

Comment thread docarray/typing/tensor/tensor.py

Comment thread docarray/typing/tensor/tensor.py

Comment thread docarray/typing/tensor/torch_tensor.py Outdated

samsja added 3 commits November 22, 2022 16:56

fix: fix default orson not returning

e04285a

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: apply johannes suggestion on docstring

b8de8bc

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: does not perform copy anymore on torch tensor unwrap

19ffb0b

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

JohannesMessner reviewed Nov 22, 2022

View reviewed changes

Comment thread docarray/typing/tensor/torch_tensor.py Outdated

samsja and others added 3 commits November 22, 2022 17:52

fix: add johannes suggestion

0b7dac7

Co-authored-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com> Signed-off-by: samsja <55492238+samsja@users.noreply.github.com>

fix: fix ruff line lenght

a334c65

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

fix: fix mypy pb

909ee74

Signed-off-by: Sami Jaghouar <sami.jaghouar@hotmail.fr>

JohannesMessner approved these changes Nov 23, 2022

View reviewed changes

JohannesMessner merged commit 42c7cdf into feat-rewrite-v2 Nov 23, 2022

JohannesMessner deleted the feat-v2-fastapi-test branch November 23, 2022 08:07

samsja mentioned this pull request Nov 23, 2022

Make compatible with fastapi #838

Closed

Navigation Menu

Uh oh!

feat: make compatible based type with fastapi - #809

feat: make compatible based type with fastapi#809
JohannesMessner merged 27 commits into
feat-rewrite-v2from
feat-v2-fastapi-test

samsja commented Nov 17, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

samsja commented Nov 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

What this PR do:

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

samsja commented Nov 17, 2022 •

edited

Loading