fix: lower DOCUMENT_PDF_GRAPHICS_LIMIT default 5000 -> 1000
The OOM hotfix (#852) set graphics_limit=5000, which caught the 955k-drawing bomb but let a second pathology through: form/table PDFs (e.g. student records) have ~1.5k grid-line vector drawings per page -- under 5000, so uncapped. With those pages uncapped, pymupdf4llm's O(n^2) find_tables grinds ~17s/page, so a 7-page form hits the 120s timeout. All 6 current backfill parse failures in tenant-blackbox-demo are this exact timeout (zero OOM, zero error). Measured on a 7-page sample: graphics_limit=2000 -> 119s (timeout), 1000 -> 2.9s -- with identical extracted text and ZERO recovered tables either way (the expensive analysis produces nothing useful on these dense forms). Lowering the default to 1000 makes them index in ~3s; the bomb file (955k >> 1000) stays capped, and pages with genuine simple tables (<1000 drawings) still get table detection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
eb5f59f085
commit
77a36fa821
@@ -182,7 +182,7 @@ async def test_processor_success_builds_page_boundaries(monkeypatch):
|
||||
assert "Hello world" in result.text
|
||||
assert result.metadata["page_boundaries"][0]["page"] == 1
|
||||
# settings forwarded to the isolated parse
|
||||
assert seen["graphics_limit"] == 5000
|
||||
assert seen["graphics_limit"] == 1000
|
||||
assert seen["timeout_seconds"] == 120
|
||||
assert seen["mem_limit_mb"] == 1536
|
||||
|
||||
|
||||
Reference in New Issue
Block a user