feat(ingest): per-tier escalation via procrastinate queue-hop

Split external (procrastinate) document processing into per-tier queues so a
document is attempted at most once per tier and requeued to the next tier's
queue on a low-quality parse, using procrastinate's native retry.

- escalation.py: TIER_LADDER (fast->structured->ocr) + EscalateError signal
- registry: process_tier (one tier) + evaluate_escalation post-parse gate
  (reuses classify_from_text) + next_available_tier; shared _classify_result
  and _oversize_result with the inline pipeline
- processor: process_document(tier=...) runs one tier and raises EscalateError
  before embed (junk text never indexed); inline memory path unchanged
- queue/procrastinate: ingest-fast|structured|ocr queues; TieredEscalationStrategy
  (queue-hop on EscalateError, bounded same-tier transient retry); queue-aware
  task; producer defers to ingest-fast; per-queue counts + all-queue reclaim
- cli: worker --tier {fast,structured,ocr}
- billing: pages_ocr usage event + pipeline_tier metadata (paid OCR billed apart)
- observability: astrolabe_ingest_queue_depth{queue,status} gauge + per-queue
  counts in nc_get_vector_sync_status / management status endpoint
- config: INGEST_ESCALATION_ENABLED (default true), INGEST_TRANSIENT_MAX_ATTEMPTS

INGEST_ESCALATION_ENABLED=false and INGEST_QUEUE=memory preserve prior behaviour.

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/vector/ingest_status.py

@@ -29,6 +29,10 @@ class IngestPending:
     # Per-status counts (todo/doing/failed/…) on the postgres backend; None on
     # the memory backend, which has no durable per-status breakdown.
     job_counts: dict[str, int] | None = None
+    # Per-tier-queue breakdown ``{queue: {status: count}}`` on the postgres
+    # backend (Deck #323); None on the memory backend. Feeds the per-tier status
+    # surface + the astrolabe_ingest_queue_depth gauge.
+    job_counts_by_queue: dict[str, dict[str, int]] | None = None
 
 
 async def get_ingest_pending(
@@ -47,13 +51,27 @@ async def get_ingest_pending(
     """
     if ingest_queue == "postgres":
         counts: dict[str, int] = {}
-        if task_producer is not None and hasattr(task_producer, "job_counts"):
+        by_queue: dict[str, dict[str, int]] | None = None
+        # Prefer the per-queue breakdown (Deck #323) and aggregate from it, so the
+        # fleet-wide totals and the per-tier view always agree. Fall back to the
+        # aggregated call for any producer that predates job_counts_by_queue.
+        if task_producer is not None and hasattr(task_producer, "job_counts_by_queue"):
+            try:
+                by_queue = await task_producer.job_counts_by_queue()
+                for per_status in by_queue.values():
+                    for status, value in per_status.items():
+                        counts[status] = counts.get(status, 0) + value
+            except Exception as e:
+                logger.warning("Failed to read ingest job counts by queue: %s", e)
+        elif task_producer is not None and hasattr(task_producer, "job_counts"):
             try:
                 counts = await task_producer.job_counts()
             except Exception as e:
                 logger.warning("Failed to read ingest job counts: %s", e)
         pending = counts.get("todo", 0) + counts.get("doing", 0)
-        return IngestPending(pending=pending, job_counts=counts)
+        return IngestPending(
+            pending=pending, job_counts=counts, job_counts_by_queue=by_queue
+        )
 
     if document_receive_stream is None:
         return IngestPending(pending=0)