Skip to content

Commit ce1a402

Browse files
committed
add new pandas resource
1 parent 0d43142 commit ce1a402

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

content/pages/03-data/16-pandas.markdown

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,3 +129,15 @@ is a data structures and analysis library.
129129

130130
* [How to convert JSON to Excel with Python and pandas](https://www.marsja.se/how-to-convert-json-to-excel-python-pandas/)
131131
provides instructions for creating a spreadsheet out of JSON file.
132+
133+
* [Loading large datasets in Pandas](https://towardsdatascience.com/loading-large-datasets-in-pandas-11bdddd36f7b)
134+
explains how to get around the `MemoryError` issue that occurs
135+
when using `read_csv` because the data set is larger than the
136+
available memory on a machine. You can use chunking with
137+
the `read_csv` function to divide the data set into smaller parts that
138+
each can be loaded into memory. Alternatively, you can use a
139+
[SQLite database](/sqlite.html) to create a [relational database](/databases.html)
140+
with the data then use SQL queries or an
141+
[object-relational mapper (ORM)](/object-relational-mappers-orms.html)
142+
to load the data and perform analysis in pandas.
143+

0 commit comments

Comments
 (0)