fix(review): type timeout as float; document worker reuse + identity check

Address PR #852 round 3 (all 🟡, no blockers):

- config: DOCUMENT_PARSE_TIMEOUT_SECONDS is now float (default 120.0) so a
  fractional value is honoured rather than silently stored in an int field;
  matches anyio.move_on_after's float seconds.
- _isolation: comment that a clean rlimit MemoryError leaves the worker alive
  in anyio's pool (vs the SIGKILL/BrokenWorkerProcess path that respawns) --
  acceptable since RLIMIT_AS caps virtual address space, not RSS.
- processor: note the `if indexed is False` is a deliberate identity check --
  a successful index (incl. dedup hit) returns None and must not be mistaken
  for a parse failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-04 22:32:35 +02:00
co-authored by Claude Opus 4.8
parent 6589e8e7fc
commit 93f0f4f881
3 changed files with 12 additions and 2 deletions
+4 -2
View File
@@ -134,7 +134,7 @@ _DEFAULTS: dict[str, Any] = {
"document_chunk_overlap": 200,
# PDF parse isolation (OOM guard)
"document_pdf_graphics_limit": 5000,
"document_parse_timeout_seconds": 120,
"document_parse_timeout_seconds": 120.0,
"document_parse_mem_limit_mb": 1536,
# Observability
"metrics_enabled": True,
@@ -716,7 +716,9 @@ class Settings:
# >=1 -- pymupdf4llm treats 0 as "no cap", which re-exposes the OOM.
document_pdf_graphics_limit: int = 5000
# wall-clock cap per parse; the worker subprocess is killed on timeout.
document_parse_timeout_seconds: int = 120
# float so a fractional DOCUMENT_PARSE_TIMEOUT_SECONDS is honoured, matching
# anyio.move_on_after's float seconds.
document_parse_timeout_seconds: float = 120.0
# RLIMIT_AS in the parse subprocess (below the pod limit). Applied once per
# worker for its lifetime, so changing it needs a pod restart.
document_parse_mem_limit_mb: int = 1536