feat(ocr): opt-in batch OCR mode via the gateway's async batch routes

Add DOCUMENT_OCR_MODE=sync|batch (default sync). In batch mode the tier-3 OCR
processor submits documents to the embedding gateway's async Batch OCR routes
(POST /v1/ocr/batch + GET /v1/ocr/batch/{job_id}, astrolabe-cloud-website#372)
for ~50% cheaper large-corpus backfill. The direct Mistral OCR path is left
untouched. Tracked on Deck #332.

Batch jobs run minutes-hours, so the OCR tier cannot block (the procrastinate
worker reclaims jobs in `doing` after INGEST_STALLED_JOB_SECONDS). Instead it
submits, records the gateway job id in a new per-tenant `batch_ocr_jobs` table
(procrastinate args are immutable across retries), and raises a BatchPending
signal that TieredEscalationStrategy turns into a same-queue deferred re-poll —
releasing the worker slot between polls. On completion the per-page markdown is
indexed like the sync path; a failure or a job past
DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS marks the document parse-failed.

Batch is opt-in and gateway-only: with the direct mistral backend, no gateway
URL, or the inline/memory pipeline (which can't defer), it falls back to sync.
One batch job per document (coalescing N docs/job is a follow-up).

- embedding/gateway_batch_client.py: submit/poll client (reuses GatewayTokenProvider).
- vector/batch_ocr_store.py + migration 008: job tracking (portable SQLite+PG).
- document_processors/escalation.py: BatchPending control-flow signal.
- document_processors/ocr.py: batch state machine + sync fallback.
- vector/processor.py: thread doc identity to the OCR tier; raise BatchPending
  from the pending sentinel; propagate it as control flow (not a failure).
- vector/queue/procrastinate.py: BatchPending -> same-queue retry_in, exempt
  from the transient cap (bounded by the processor's deadline).
- config + docs; tests across client/store/processor/strategy/parse-tier.

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-15 09:41:19 +02:00
co-authored by Claude Opus 4.8
parent 07ee91399b
commit 3b7e8d779b
14 changed files with 1238 additions and 59 deletions
+36
View File
@@ -572,6 +572,42 @@ A PDF larger than `DOCUMENT_MAX_PDF_SIZE_MB` fails fast with reason `oversize`
of being handed to the tiers, where a 40+ MB scan would otherwise burn the full
OCR timeout for zero recovered text.
#### OCR execution mode: synchronous vs batch (Deck #332)
The tier-3 OCR processor has two execution modes, selected by `DOCUMENT_OCR_MODE`:
```dotenv
DOCUMENT_OCR_MODE=sync # "sync" (default) | "batch"
DOCUMENT_OCR_BATCH_POLL_SECONDS=120 # re-poll cadence for a batch job (default: 120)
DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS=86400 # give up + mark timeout after this (default: 24h)
```
- **`sync`** (default) — transcribe the document inline via the backend's
synchronous path (`POST /v1/ocr` for the gateway, or the direct Mistral OCR
API). The document is parsed in a single call.
- **`batch`** — submit the document to the **gateway's async Batch OCR** job
(`POST /v1/ocr/batch`) and re-poll `GET /v1/ocr/batch/{job_id}` until it
finishes. This trades latency (a batch job runs minuteshours) for roughly
**half the OCR cost**, so it suits large-corpus backfill rather than
interactive ingest.
Batch mode is **opt-in and gateway-only**: it routes Mistral's Batch API
*through* the gateway's batch routes (no provider keys in the pod). With the
direct `mistral` backend, no `EMBEDDING_GATEWAY_URL`, or on the in-process
(`INGEST_QUEUE=memory`) pipeline — which can't defer a poll — batch transparently
**falls back to synchronous OCR**. So enabling it requires the Postgres ingest
queue (the per-tier procrastinate workers) and the gateway embedding backend.
Mechanics: the OCR tier submits the job, records its id in the `batch_ocr_jobs`
app-DB table (keyed on the document + its etag), and raises a re-poll deferral so
procrastinate re-runs the tier after `DOCUMENT_OCR_BATCH_POLL_SECONDS` — releasing
the worker slot between polls (a long batch never pins a worker or is reclaimed as
stalled). On completion the per-page markdown is indexed exactly like the sync
path; a failure or a job exceeding `DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS` marks the
document parse-failed. Each poll re-fetches + re-classifies the PDF (a known v1
inefficiency, bounded by the poll cadence); one batch job is submitted per
document (coalescing many documents per job is a planned follow-up).
### Embedding Service Configuration
The server picks an embedding provider via auto-detection. Priority order