Skip to content

docs: document inlining of small data in read_* APIs #670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shobsi merged 4 commits into from
May 8, 2024
Merged

docs: document inlining of small data in read_* APIs #670

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bb9fb94
docs: document inlining of small data in `read_*` APIs
shobsi May 8, 2024
c54607b
mention that threshold is in memory size
shobsi May 8, 2024
e29d84a
Merge remote-tracking branch 'refs/remotes/github/main' into shobs-do…
shobsi May 8, 2024
08b582d
non-bigquery instead of non-"bigquery"
shobsi May 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions bigframes/session/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -934,6 +934,11 @@ def read_pandas(
The pandas DataFrame will be persisted as a temporary BigQuery table, which can be
automatically recycled after the Session is closed.

.. note::
Data is inlined in the query SQL if it is small enough (roughly 5MB
or less in memory). Larger size data is loaded to a BigQuery table
instead.

**Examples:**

>>> import bigframes.pandas as bpd
Expand Down
5 changes: 5 additions & 0 deletions third_party/bigframes_vendored/pandas/io/parquet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ def read_parquet(
Instead, set a serialized index column as the index and sort by
that in the resulting DataFrame.

.. note::
For non-"bigquery" engine, data is inlined in the query SQL if it is
small enough (roughly 5MB or less in memory). Larger size data is
loaded to a BigQuery table instead.

**Examples:**

>>> import bigframes.pandas as bpd
Expand Down
10 changes: 10 additions & 0 deletions third_party/bigframes_vendored/pandas/io/parsers/readers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ def read_csv(
file. Instead, set a serialized index column as the index and sort by
that in the resulting DataFrame.

.. note::
For non-bigquery engine, data is inlined in the query SQL if it is
small enough (roughly 5MB or less in memory). Larger size data is
loaded to a BigQuery table instead.

**Examples:**

>>> import bigframes.pandas as bpd
Expand Down Expand Up @@ -167,6 +172,11 @@ def read_json(
file. Instead, set a serialized index column as the index and sort by
that in the resulting DataFrame.

.. note::
For non-bigquery engine, data is inlined in the query SQL if it is
small enough (roughly 5MB or less in memory). Larger size data is
loaded to a BigQuery table instead.

**Examples:**

>>> import bigframes.pandas as bpd
Expand Down
5 changes: 5 additions & 0 deletions third_party/bigframes_vendored/pandas/io/pickle.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ def read_pickle(
If the content of the pickle file is a Series and its name attribute is None,
the name will be set to '0' by default.

.. note::
Data is inlined in the query SQL if it is small enough (roughly 5MB
or less in memory). Larger size data is loaded to a BigQuery table
instead.

**Examples:**

>>> import bigframes.pandas as bpd
Expand Down