feat(ingest): split OCR into tier2 in-cluster (GPU, gateway-only) + tier3 upstream
Insert a configurable in-cluster OCR rung into the escalation ladder (Deck #353): a tier2-eligible doc is OCR'd on the on-demand burst GPU before falling through to paid upstream OCR. The in-cluster backend is reached ONLY via the embedding gateway (model prefix routes to the GPU over the tailnet) and is a config value (default surya/surya-ocr-2, swappable to e.g. lightonocr) — never hard-coded. Ladder: fast -> structured -> ocr-incluster -> ocr-upstream (queues ingest-ocr-incluster / ingest-ocr-upstream). - escalation.py: 4-tier ladder; in-cluster flag folded into the dead-letter signature. - ocr.py: OcrProcessor(name, tier, model_setting, gateway_only); build_ocr_backend( ..., model=, gateway_only=) — gateway_only forces the gateway backend (never the direct Mistral fallback), disabling the tier with a warning if no gateway URL. - registry.py: per-rung enable map; scanned docs target minimum="ocr-incluster"; inline path runs the cheapest available OCR rung. - procrastinate.py: two OCR queues; legacy ingest-ocr kept as a drain target. - config.py: DOCUMENT_OCR_INCLUSTER_ENABLED (off) + DOCUMENT_OCR_INCLUSTER_MODEL. - __init__.py: register the two OCR instances; vector/processor.py: pages_ocr metered for the upstream (paid) rung only; cli.py: new --tier choices + legacy drain. - metrics.py: zero the legacy ingest-ocr queue gauge during rollout. - tests: migrated to the split ladder + new tests (gateway-only forcing, per-tier model incl. lightonocr override, no-hard-coded-surya guard). 1792 pass; ruff + ty green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
060084029f
commit
c21804fbbc
@@ -335,7 +335,7 @@ def _init_worker_observability(settings: Settings) -> None:
|
||||
)
|
||||
@click.option(
|
||||
"--tier",
|
||||
type=click.Choice(["fast", "structured", "ocr"]),
|
||||
type=click.Choice(["fast", "structured", "ocr-incluster", "ocr-upstream"]),
|
||||
default=None,
|
||||
help=(
|
||||
"Run only this extraction tier's queue (Deck #323). Omit to drain ALL "
|
||||
@@ -386,21 +386,31 @@ def worker(concurrency: int | None, tier: str | None):
|
||||
ALL_INGEST_QUEUES,
|
||||
INGEST_QUEUE_MAINTENANCE,
|
||||
LEGACY_INGEST_QUEUE,
|
||||
LEGACY_INGEST_QUEUE_OCR,
|
||||
TIER_QUEUES,
|
||||
apply_ingest_queue_schema,
|
||||
get_procrastinate_app,
|
||||
)
|
||||
|
||||
# Which queues this process drains. A single tier -> just its queue; no tier
|
||||
# -> every tier queue PLUS the legacy single queue, so a rolling upgrade
|
||||
# never strands jobs deferred under the pre-#323 name. Every worker also
|
||||
# -> every tier queue PLUS the legacy queues, so a rolling upgrade never
|
||||
# strands jobs deferred under the pre-#323 single queue or the pre-#353 single
|
||||
# OCR queue. The upstream OCR worker also drains the pre-split ``ingest-ocr``
|
||||
# queue (the old single OCR tier defaulted to upstream/Mistral). Every worker
|
||||
# drains the maintenance queue so the periodic stalled-job reclaim fires
|
||||
# regardless of which tier(s) are scaled up (procrastinate dedups the
|
||||
# periodic, so multiple drainers don't multiply the reclaim).
|
||||
if tier is not None:
|
||||
queues = [TIER_QUEUES[tier], INGEST_QUEUE_MAINTENANCE]
|
||||
if tier == "ocr-upstream":
|
||||
queues.insert(1, LEGACY_INGEST_QUEUE_OCR)
|
||||
else:
|
||||
queues = [*ALL_INGEST_QUEUES, LEGACY_INGEST_QUEUE, INGEST_QUEUE_MAINTENANCE]
|
||||
queues = [
|
||||
*ALL_INGEST_QUEUES,
|
||||
LEGACY_INGEST_QUEUE,
|
||||
LEGACY_INGEST_QUEUE_OCR,
|
||||
INGEST_QUEUE_MAINTENANCE,
|
||||
]
|
||||
|
||||
# This is the consumer side of the distributed (postgres) ingest backend.
|
||||
# Unlike the in-process anyio pool, the worker talks to procrastinate's App
|
||||
|
||||
Reference in New Issue
Block a user