fix(review): type timeout as float; document worker reuse + identity check
Address PR #852 round 3 (all 🟡, no blockers): - config: DOCUMENT_PARSE_TIMEOUT_SECONDS is now float (default 120.0) so a fractional value is honoured rather than silently stored in an int field; matches anyio.move_on_after's float seconds. - _isolation: comment that a clean rlimit MemoryError leaves the worker alive in anyio's pool (vs the SIGKILL/BrokenWorkerProcess path that respawns) -- acceptable since RLIMIT_AS caps virtual address space, not RSS. - processor: note the `if indexed is False` is a deliberate identity check -- a successful index (incl. dedup hit) returns None and must not be mistaken for a parse failure. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
6589e8e7fc
commit
93f0f4f881
@@ -115,6 +115,11 @@ async def run_isolated_pdf_parse(
|
||||
cancellable=True,
|
||||
)
|
||||
except MemoryError as e:
|
||||
# A clean rlimit breach: the worker raised MemoryError and stays
|
||||
# ALIVE in anyio's pool (unlike the BrokenWorkerProcess/SIGKILL path,
|
||||
# which spawns a fresh worker). Its heap may be slightly fragmented
|
||||
# for the next document. Acceptable: RLIMIT_AS caps virtual address
|
||||
# space (not RSS), so practical fragmentation risk is low.
|
||||
raise PdfParseFailed("oom", str(e)) from e
|
||||
except BrokenWorkerProcess as e:
|
||||
# Worker died without a clean exception (e.g. SIGKILL from the OS OOM
|
||||
|
||||
Reference in New Issue
Block a user