fix: lower DOCUMENT_PDF_GRAPHICS_LIMIT default 5000 -> 1000

The OOM hotfix (#852) set graphics_limit=5000, which caught the 955k-drawing
bomb but let a second pathology through: form/table PDFs (e.g. student records)
have ~1.5k grid-line vector drawings per page -- under 5000, so uncapped. With
those pages uncapped, pymupdf4llm's O(n^2) find_tables grinds ~17s/page, so a
7-page form hits the 120s timeout. All 6 current backfill parse failures in
tenant-blackbox-demo are this exact timeout (zero OOM, zero error).

Measured on a 7-page sample: graphics_limit=2000 -> 119s (timeout), 1000 -> 2.9s
-- with identical extracted text and ZERO recovered tables either way (the
expensive analysis produces nothing useful on these dense forms). Lowering the
default to 1000 makes them index in ~3s; the bomb file (955k >> 1000) stays
capped, and pages with genuine simple tables (<1000 drawings) still get table
detection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-05 00:06:40 +02:00
co-authored by Claude Opus 4.8
parent eb5f59f085
commit 77a36fa821
2 changed files with 10 additions and 5 deletions
+1 -1
View File
@@ -182,7 +182,7 @@ async def test_processor_success_builds_page_boundaries(monkeypatch):
assert "Hello world" in result.text
assert result.metadata["page_boundaries"][0]["page"] == 1
# settings forwarded to the isolated parse
assert seen["graphics_limit"] == 5000
assert seen["graphics_limit"] == 1000
assert seen["timeout_seconds"] == 120
assert seen["mem_limit_mb"] == 1536