refactor(ingest): doc legacy ocr queue; fix docstrings + getattr guard
Address claude-review round 2 on #922: - Legacy `ingest-ocr` tier resolution (important 1): document why it deliberately resolves to `fast` rather than mapping to `ocr-upstream` — stranded pre-split jobs re-extract empty and re-escalate via the ladder to the cheap `ocr-incluster` rung, keeping them OFF the paid upstream rung. Added a tier_for_queue(LEGACY_INGEST_QUEUE_OCR) == "fast" assertion. - Double get_settings() in `_get_batch_client` (important 2): bind once to a local. - Stale docstrings (important 3): OcrProcessor (serves both rungs now), _tier_available (both OCR rungs gated), evaluate_escalation (targets ocr-incluster, falls through to ocr-upstream). - Misconfigured model_setting (nit 5): OcrProcessor.__init__ raises ValueError on an unknown settings attr (fail-fast at startup vs AttributeError mid-OCR); also removes the dynamic-getattr static-analysis smell SonarCloud flagged. - Redundant guard (nit 4): kept `and ocr_tier is not None` — it's required for ty to narrow ocr_tier to str for record_document_escalation; added a comment. - Test gap (nit 6): added a test pinning the CURRENT incluster-failure -> tier-1 fallback (does NOT cascade to upstream) so the future 503-escalation change is an explicit diff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
87b8edd139
commit
6e32bd9561
@@ -360,6 +360,10 @@ class ProcessorRegistry:
|
||||
from_tier, settings, minimum="ocr-incluster"
|
||||
)
|
||||
ocr = self._pdf_processor_for_tier(ocr_tier) if ocr_tier else None
|
||||
# `ocr is not None` already implies `ocr_tier is not None` at runtime,
|
||||
# but the type checker can't infer that across the conditional above,
|
||||
# so the explicit guard narrows `ocr_tier` to `str` for the
|
||||
# record_document_escalation(from_tier, ocr_tier, reason) call below.
|
||||
if ocr is not None and ocr_tier is not None:
|
||||
reason = (
|
||||
"corrupt_glyphs"
|
||||
@@ -514,8 +518,8 @@ class ProcessorRegistry:
|
||||
``ignore_ocr_enabled`` drops only the OCR-enabled gate (not the registered-
|
||||
processor requirement): it answers "would this tier run if OCR were turned
|
||||
on?" — used to compute the *ideal* escalation target for the what-if-OCR
|
||||
suppressed-escalation signal. (Today only ``ocr`` has an enabled gate; a
|
||||
future per-tier gate would extend the condition below.)
|
||||
suppressed-escalation signal. (Both OCR rungs — ``ocr-incluster`` and
|
||||
``ocr-upstream`` — have their own enabled gate; non-OCR tiers have none.)
|
||||
"""
|
||||
if self._pdf_processor_for_tier(tier) is None:
|
||||
return False
|
||||
@@ -624,9 +628,11 @@ class ProcessorRegistry:
|
||||
|
||||
Target-tier routing:
|
||||
|
||||
- ``total_chars == 0`` (scanned / no text layer) -> target the ``ocr``
|
||||
tier directly. Text-extractor tiers (``structured``) cannot conjure
|
||||
text from a pure raster scan, so a structured hop would just be wasted.
|
||||
- ``total_chars == 0`` (scanned / no text layer) -> target the cheapest
|
||||
OCR rung (``ocr-incluster``) directly; ``next_available_tier`` falls
|
||||
through to ``ocr-upstream`` if in-cluster is disabled/unregistered.
|
||||
Text-extractor tiers (``structured``) cannot conjure text from a pure
|
||||
raster scan, so a structured hop would just be wasted.
|
||||
- glyph-corrupt text layer (``recommended_tier == "structured"``) -> target
|
||||
the ``structured`` tier; pymupdf re-extracts a broken-/ToUnicode layer
|
||||
correctly, so OCR is never the target for this case.
|
||||
|
||||
Reference in New Issue
Block a user