perf: batch spaCy processing for PDF text - #4430
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 5 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="unstructured/nlp/tokenize.py">
<violation number="1" location="unstructured/nlp/tokenize.py:196">
P2: A text-dense page can now hold every parsed spaCy Doc for all unique text blocks in memory simultaneously for the whole page. Before this change each `_process` call built one Doc that was consumed and discarded, so peak memory was roughly a single Doc; now the `docs` dict accumulates all of them until the context exits. The `batch_size` argument only chunks `nlp.pipe` internally and does not bound retained memory, so the PR's 'page-bounded / safe for large PDFs' memory goal isn't actually enforced by this implementation. Consider retaining and releasing Docs in chunks (still reusing them across the three tokenizer steps per chunk) so peak memory remains bounded for dense pages, and document that `batch_size` controls pipeline chunking, not per-page retention.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Re-trigger cubic
|
|
||
| nlp = _get_nlp() | ||
| prepared_inputs = tuple(_prepare_text(text, nlp) for text in pipeline_inputs) | ||
| docs = dict( |
There was a problem hiding this comment.
P2: A text-dense page can now hold every parsed spaCy Doc for all unique text blocks in memory simultaneously for the whole page. Before this change each _process call built one Doc that was consumed and discarded, so peak memory was roughly a single Doc; now the docs dict accumulates all of them until the context exits. The batch_size argument only chunks nlp.pipe internally and does not bound retained memory, so the PR's 'page-bounded / safe for large PDFs' memory goal isn't actually enforced by this implementation. Consider retaining and releasing Docs in chunks (still reusing them across the three tokenizer steps per chunk) so peak memory remains bounded for dense pages, and document that batch_size controls pipeline chunking, not per-page retention.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At unstructured/nlp/tokenize.py, line 196:
<comment>A text-dense page can now hold every parsed spaCy Doc for all unique text blocks in memory simultaneously for the whole page. Before this change each `_process` call built one Doc that was consumed and discarded, so peak memory was roughly a single Doc; now the `docs` dict accumulates all of them until the context exits. The `batch_size` argument only chunks `nlp.pipe` internally and does not bound retained memory, so the PR's 'page-bounded / safe for large PDFs' memory goal isn't actually enforced by this implementation. Consider retaining and releasing Docs in chunks (still reusing them across the three tokenizer steps per chunk) so peak memory remains bounded for dense pages, and document that `batch_size` controls pipeline chunking, not per-page retention.</comment>
<file context>
@@ -162,9 +168,55 @@ def _process(text: str) -> spacy.tokens.Doc:
+
+ nlp = _get_nlp()
+ prepared_inputs = tuple(_prepare_text(text, nlp) for text in pipeline_inputs)
+ docs = dict(
+ zip(
+ pipeline_inputs,
</file context>
There was a problem hiding this comment.
Thanks, good catch. PDFMiner records are now processed in chunks of BATCH_SIZE, with a separate batch_process_texts() context per chunk, so the documents are released between chunks.
e862c41 to
83541c7
Compare
8290373 to
12aa311
Compare
1 participant
Summary
Batch spaCy processing during PDFMiner-based PDF text classification while avoiding NLP work for elements that can be classified using inexpensive rules.
FAST PDF partitioning previously processed each extracted text block independently. A block could invoke spaCy multiple times through sentence tokenization and POS tagging.
This change:
nlp.pipe().Docobjects across tokenizer helpers.Classification results, element order, coordinates, metadata, and links remain unchanged.
Key Changes
batch_process_texts()for reusable spaCyDocobjects.nlp.pipe().Performance Benchmark
Model-warm, cache-cleared classification benchmark using five iterations, batch size 256, and page-like contexts of 64 elements:
1,000 Elements
Sequential and selectively batched modes produced identical category/text fingerprints for every workload and corpus size.
Reproduce the complete matrix with:
uv run --no-sync python scripts/performance/benchmark_text_classification.py \ --workload all \ --counts 1,4,16,40,100,1000 \ --context-size 64 \ --batch-size 256 \ --iterations 5 ### Local Real PDF ValidationA model-warm, cache-cleared FAST benchmark on a 24-page PDF produced:
origin/mainThe page/type/text/order fingerprint was identical.
Compatibility
element_from_text()behavior remains unchanged.Verification & Testing