fix(review): close doc via try/finally; don't count parse failures as indexed

Address PR #852 review:

- pymupdf.py: the metadata `doc` was only closed on the PdfParseFailed and
  success paths, so a failure in `_extract_metadata`/`mkdir`/`get_settings`
  leaked it. `doc` is only needed for metadata + page_count (the heavy parse
  works from `content` bytes in the worker), so open it, read metadata, and
  close it immediately under try/finally; drop the two later doc.close() calls.

- processor.py: a permanent parse failure early-returned from `_index_document`,
  after which `process_document` still recorded record_qdrant_operation("upsert",
  "success") + record_vector_sync_processing(success) -- counting an OOM/timeout
  bomb as astrolabe_documents_indexed_total{status="success"}. `_index_document`
  now returns False on that path and the caller skips the success metrics (the
  failure is already recorded via document_parse_failed_total + the registry's
  document_parse_total{error}).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-04 22:32:34 +02:00
co-authored by Claude Opus 4.8
parent 7db8d3e301
commit 7ec116a3c7
2 changed files with 22 additions and 13 deletions
@@ -107,14 +107,19 @@ class PyMuPDFProcessor(DocumentProcessor):
if progress_callback:
await progress_callback(0, 100, "Opening PDF document")
# Open document and extract metadata in thread
# Open document only to read metadata + page count, then close it
# immediately (try/finally so a failure in _extract_metadata can't
# leak it). The heavy extraction below works from ``content`` bytes
# in the isolated worker, so ``doc`` is not needed past this point.
doc = await anyio.to_thread.run_sync( # type: ignore[attr-defined]
lambda: pymupdf.open("pdf", content)
)
metadata = self._extract_metadata(doc, filename)
metadata["file_size"] = len(content)
page_count = doc.page_count
try:
metadata = self._extract_metadata(doc, filename)
metadata["file_size"] = len(content)
page_count = doc.page_count
finally:
doc.close()
if progress_callback:
await progress_callback(10, 100, f"Extracting {page_count} pages")
@@ -143,7 +148,6 @@ class PyMuPDFProcessor(DocumentProcessor):
mem_limit_mb=settings.document_parse_mem_limit_mb,
)
except PdfParseFailed as exc:
doc.close()
logger.warning(
"Isolated PDF parse failed for %s (reason=%s): %s",
filename or "<bytes>",
@@ -198,9 +202,6 @@ class PyMuPDFProcessor(DocumentProcessor):
metadata["image_paths"] = image_paths
metadata["page_boundaries"] = page_boundaries
# Close document
doc.close()
if progress_callback:
await progress_callback(100, 100, "Processing complete")