feat(ocr): opt-in batch OCR mode via the gateway's async batch routes

Add DOCUMENT_OCR_MODE=sync|batch (default sync). In batch mode the tier-3 OCR
processor submits documents to the embedding gateway's async Batch OCR routes
(POST /v1/ocr/batch + GET /v1/ocr/batch/{job_id}, astrolabe-cloud-website#372)
for ~50% cheaper large-corpus backfill. The direct Mistral OCR path is left
untouched. Tracked on Deck #332.

Batch jobs run minutes-hours, so the OCR tier cannot block (the procrastinate
worker reclaims jobs in `doing` after INGEST_STALLED_JOB_SECONDS). Instead it
submits, records the gateway job id in a new per-tenant `batch_ocr_jobs` table
(procrastinate args are immutable across retries), and raises a BatchPending
signal that TieredEscalationStrategy turns into a same-queue deferred re-poll —
releasing the worker slot between polls. On completion the per-page markdown is
indexed like the sync path; a failure or a job past
DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS marks the document parse-failed.

Batch is opt-in and gateway-only: with the direct mistral backend, no gateway
URL, or the inline/memory pipeline (which can't defer), it falls back to sync.
One batch job per document (coalescing N docs/job is a follow-up).

- embedding/gateway_batch_client.py: submit/poll client (reuses GatewayTokenProvider).
- vector/batch_ocr_store.py + migration 008: job tracking (portable SQLite+PG).
- document_processors/escalation.py: BatchPending control-flow signal.
- document_processors/ocr.py: batch state machine + sync fallback.
- vector/processor.py: thread doc identity to the OCR tier; raise BatchPending
  from the pending sentinel; propagate it as control flow (not a failure).
- vector/queue/procrastinate.py: BatchPending -> same-queue retry_in, exempt
  from the transient cap (bounded by the processor's deadline).
- config + docs; tests across client/store/processor/strategy/parse-tier.

1653 unit tests pass; ruff + ty green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/alembic/versions/20260615_1200_008_add_batch_ocr_jobs.py

@@ -0,0 +1,67 @@
+"""Add batch_ocr_jobs table for async batch OCR job tracking.
+
+Deck #332 / embedding-gateway batch OCR (astrolabe-cloud-website#372). When
+``DOCUMENT_OCR_MODE=batch`` the OCR tier submits a document to the gateway's
+async ``POST /v1/ocr/batch`` and must re-poll ``GET /v1/ocr/batch/{job_id}``
+across procrastinate retries. procrastinate job args are immutable, so the
+gateway ``job_id`` (and submit time, for the poll deadline) are persisted here,
+keyed on the document + its content version (``etag``).
+
+One row per in-flight job; the row is deleted once the job reaches a terminal
+state. Empty + unused unless batch mode is enabled (gateway-only), so OSS/SQLite
+self-hosters get an idle table and zero overhead.
+
+Portable types only (Text + unix-epoch BigInteger timestamps, like the rest of
+this schema except the CP-queried usage_events) so the same migration runs on
+both self-host SQLite and cloud Postgres.
+
+Revision ID: 008
+Revises: 007
+Create Date: 2026-06-15 12:00:00.000000
+"""
+
+import sqlalchemy as sa
+
+from alembic import op
+
+revision = "008"
+down_revision = "007"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "batch_ocr_jobs",
+        # Document identity (the same keys the OCR tier receives via the
+        # processor ``options``). ``etag`` is the content-version key: a changed
+        # document (new etag) is a new job, so a stale row never serves results
+        # for the wrong content.
+        sa.Column("user_id", sa.Text(), nullable=False),
+        sa.Column("doc_id", sa.Text(), nullable=False),
+        sa.Column("doc_type", sa.Text(), nullable=False),
+        sa.Column("etag", sa.Text(), nullable=False),
+        # The gateway's namespaced batch job id ("<provider>/<batch_job_id>") —
+        # the only handle for polling (the gateway is stateless).
+        sa.Column("job_id", sa.Text(), nullable=False),
+        # Gateway-normalised status mirror (pending|succeeded|failed). Kept for
+        # observability; the live decision always comes from a fresh poll.
+        sa.Column("status", sa.Text(), nullable=False),
+        # Unix-epoch seconds. ``submitted_at`` anchors the poll deadline
+        # (DOCUMENT_OCR_BATCH_MAX_WAIT_SECONDS).
+        sa.Column("submitted_at", sa.BigInteger(), nullable=False),
+        sa.Column("updated_at", sa.BigInteger(), nullable=False),
+        # One in-flight job per (document, content version). A resubmit for a new
+        # etag inserts a new row; the superseded row is swept on resubmit.
+        sa.UniqueConstraint(
+            "user_id",
+            "doc_id",
+            "doc_type",
+            "etag",
+            name="uq_batch_ocr_jobs_doc",
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("batch_ocr_jobs")