refactor(ingest): doc legacy ocr queue; fix docstrings + getattr guard

Address claude-review round 2 on #922:

- Legacy `ingest-ocr` tier resolution (important 1): document why it deliberately
  resolves to `fast` rather than mapping to `ocr-upstream` — stranded pre-split
  jobs re-extract empty and re-escalate via the ladder to the cheap
  `ocr-incluster` rung, keeping them OFF the paid upstream rung. Added a
  tier_for_queue(LEGACY_INGEST_QUEUE_OCR) == "fast" assertion.
- Double get_settings() in `_get_batch_client` (important 2): bind once to a local.
- Stale docstrings (important 3): OcrProcessor (serves both rungs now),
  _tier_available (both OCR rungs gated), evaluate_escalation (targets
  ocr-incluster, falls through to ocr-upstream).
- Misconfigured model_setting (nit 5): OcrProcessor.__init__ raises ValueError on
  an unknown settings attr (fail-fast at startup vs AttributeError mid-OCR); also
  removes the dynamic-getattr static-analysis smell SonarCloud flagged.
- Redundant guard (nit 4): kept `and ocr_tier is not None` — it's required for ty
  to narrow ocr_tier to str for record_document_escalation; added a comment.
- Test gap (nit 6): added a test pinning the CURRENT incluster-failure ->
  tier-1 fallback (does NOT cascade to upstream) so the future 503-escalation
  change is an explicit diff.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-18 00:16:20 +02:00
co-authored by Claude Opus 4.8
parent 87b8edd139
commit 6e32bd9561
5 changed files with 55 additions and 8 deletions
@@ -287,7 +287,9 @@ def build_ocr_backend(
class OcrProcessor(DocumentProcessor):
"""Tier-3 OCR processor (gateway or direct Mistral backend)."""
"""OCR processor for both OCR rungs — tier2 in-cluster (gateway-only GPU) and
tier3 upstream (gateway or direct Mistral backend). One class, two registered
instances bound to different ``(tier, model_setting, gateway_only)``."""
def __init__(
self,
@@ -303,6 +305,11 @@ class OcrProcessor(DocumentProcessor):
# gateway-only (its model, e.g. surya, is reachable solely via the
# gateway); the upstream rung keeps the configurable gateway/mistral
# selection. surya is NEVER hard-coded here — only a config default.
# Fail fast on a misconfigured model_setting (a typo in a constructor call)
# so it surfaces at startup, not as an AttributeError mid-OCR. The string
# only ever comes from hardcoded defaults in __init__.py, never user input.
if not hasattr(Settings, model_setting):
raise ValueError(f"Unknown model_setting: {model_setting!r}")
self._name = name
self._tier = tier
self._model_setting = model_setting
@@ -450,9 +457,10 @@ class OcrProcessor(DocumentProcessor):
self._batch_client_lock = anyio.Lock()
async with self._batch_client_lock:
if not self._batch_client_resolved: # double-checked
settings = get_settings()
self._batch_client = build_gateway_batch_client(
get_settings(),
model=getattr(get_settings(), self._model_setting),
settings,
model=getattr(settings, self._model_setting),
)
self._batch_client_resolved = True
return self._batch_client
@@ -360,6 +360,10 @@ class ProcessorRegistry:
from_tier, settings, minimum="ocr-incluster"
)
ocr = self._pdf_processor_for_tier(ocr_tier) if ocr_tier else None
# `ocr is not None` already implies `ocr_tier is not None` at runtime,
# but the type checker can't infer that across the conditional above,
# so the explicit guard narrows `ocr_tier` to `str` for the
# record_document_escalation(from_tier, ocr_tier, reason) call below.
if ocr is not None and ocr_tier is not None:
reason = (
"corrupt_glyphs"
@@ -514,8 +518,8 @@ class ProcessorRegistry:
``ignore_ocr_enabled`` drops only the OCR-enabled gate (not the registered-
processor requirement): it answers "would this tier run if OCR were turned
on?" — used to compute the *ideal* escalation target for the what-if-OCR
suppressed-escalation signal. (Today only ``ocr`` has an enabled gate; a
future per-tier gate would extend the condition below.)
suppressed-escalation signal. (Both OCR rungs — ``ocr-incluster`` and
``ocr-upstream`` — have their own enabled gate; non-OCR tiers have none.)
"""
if self._pdf_processor_for_tier(tier) is None:
return False
@@ -624,9 +628,11 @@ class ProcessorRegistry:
Target-tier routing:
- ``total_chars == 0`` (scanned / no text layer) -> target the ``ocr``
tier directly. Text-extractor tiers (``structured``) cannot conjure
text from a pure raster scan, so a structured hop would just be wasted.
- ``total_chars == 0`` (scanned / no text layer) -> target the cheapest
OCR rung (``ocr-incluster``) directly; ``next_available_tier`` falls
through to ``ocr-upstream`` if in-cluster is disabled/unregistered.
Text-extractor tiers (``structured``) cannot conjure text from a pure
raster scan, so a structured hop would just be wasted.
- glyph-corrupt text layer (``recommended_tier == "structured"``) -> target
the ``structured`` tier; pymupdf re-extracts a broken-/ToUnicode layer
correctly, so OCR is never the target for this case.