feat(ingest): record suppressed OCR escalations (what-if-OCR signal)

OCR is the paid, opt-in tier (DOCUMENT_OCR_ENABLED, default off). The per-tier
escalation gate already declines to hop to OCR when it's disabled (the pre-OCR
tier is terminal — no surprise cost), but that left operators blind to how much
OCR demand exists.

evaluate_escalation now returns a structured EscalationDecision:
- "hop"        — a higher tier can run; the caller raises EscalateError (queue-hop).
- "suppressed" — the ideal next tier (e.g. ocr) exists but is DISABLED; the caller
                 indexes the current tier's output as terminal and records the
                 would-be hop on the new astrolabe_document_escalation_suppressed_total
                 {from_tier,to_tier,reason} counter instead of hopping.
- None         — index as-is (good text, or no such tier at all).

So with OCR off, escalation_suppressed_total{to_tier="ocr"} is the latent OCR
demand an operator weighs before enabling OCR; enabling it converts these into
real document_escalation_total{to_tier="ocr"} hops. next_available_tier gains an
ignore_enabled flag to compute the *ideal* (enabled-gate-ignored) target.

Tests: registry suppressed vs hop vs terminal (incl. structured-hop-not-suppressed
when OCR off but structured available); _parse_pdf_tier records suppressed +
indexes without raising.

Deck #324 (parent #323).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-13 15:16:03 +02:00
co-authored by Claude Opus 4.8
parent 6db48830af
commit a27ddb2d5a
6 changed files with 227 additions and 33 deletions
@@ -21,11 +21,36 @@ mapping lives in the queue layer, which imports :class:`EscalateError` from here
from __future__ import annotations
from dataclasses import dataclass
# Cheapest-first. ``llm`` is reserved (see base.DocumentProcessor.tier) and not
# wired yet, so it is intentionally absent from the live ladder.
TIER_LADDER: tuple[str, ...] = ("fast", "structured", "ocr")
@dataclass(frozen=True)
class EscalationDecision:
"""Outcome of the post-parse quality gate (``ProcessorRegistry.evaluate_escalation``).
``kind``:
* ``"hop"`` — the parse is too poor and a higher tier *can run*; the caller
raises :class:`EscalateError` to requeue the document onto ``to_tier``.
* ``"suppressed"`` — the parse would escalate to ``to_tier`` (the *ideal*
next tier), but that tier is **disabled** (e.g. OCR off). The caller does
NOT hop — it indexes the current tier's output as terminal — and records
the would-be escalation so operators see the latent demand ("what-if OCR
were enabled"). Enabling the tier turns these into real ``"hop"`` events.
A ``None`` return from ``evaluate_escalation`` (not an instance of this class)
means "index as-is, nothing to escalate" — good text, or no higher tier
exists at all (no processor registered for it).
"""
kind: str # "hop" | "suppressed"
to_tier: str
reason: str # empty_text | low_confidence
def next_tier(current: str) -> str | None:
"""The next tier above ``current`` in the ladder, or ``None`` if terminal.