fix(ingest): address review round 1 (reclaim queue + tests)

- Register the periodic stalled-job reclaim on a dedicated ingest-maintenance
  queue that every worker drains (any --tier), so reclaim still fires when the
  fast fleet is scaled to zero and only ocr workers run. procrastinate's
  periodic-defer dedup keeps it single-run across drainers.
- escalation: mark `unsupported`/`forced` reason labels as reserved (not raised).
- processor: note that options/progress_callback are intentionally not threaded
  through _parse_pdf_tier yet (symmetric with the inline path).
- tests: assert TieredEscalationStrategy backoff progression (4/8/16/…/300s);
  cover get_ingest_pending per-queue aggregation + the legacy job_counts
  fallback; add an external-path zero-page no-escalation case; use the canonical
  INGEST_QUEUE_FAST instead of the back-compat alias.

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-13 13:34:03 +02:00
co-authored by Claude Opus 4.8
parent 9676bb3106
commit 35f8204a16
8 changed files with 86 additions and 11 deletions
@@ -53,8 +53,10 @@ class EscalateError(Exception):
the junk text is never indexed, and it must never be swallowed by a broad
``except Exception`` on the indexing path.
``reason`` uses the existing escalation label vocabulary:
``empty_text`` | ``low_confidence`` | ``unsupported`` | ``forced``.
``reason`` uses the existing escalation label vocabulary. This PR raises
``empty_text`` (scanned / no text layer) and ``low_confidence`` (junk text
layer); ``unsupported`` and ``forced`` are reserved for future callers and
not raised yet.
"""
def __init__(self, *, from_tier: str, to_tier: str, reason: str) -> None: