feat(ingest): per-tier escalation via procrastinate queue-hop

Split external (procrastinate) document processing into per-tier queues so a
document is attempted at most once per tier and requeued to the next tier's
queue on a low-quality parse, using procrastinate's native retry.

- escalation.py: TIER_LADDER (fast->structured->ocr) + EscalateError signal
- registry: process_tier (one tier) + evaluate_escalation post-parse gate
  (reuses classify_from_text) + next_available_tier; shared _classify_result
  and _oversize_result with the inline pipeline
- processor: process_document(tier=...) runs one tier and raises EscalateError
  before embed (junk text never indexed); inline memory path unchanged
- queue/procrastinate: ingest-fast|structured|ocr queues; TieredEscalationStrategy
  (queue-hop on EscalateError, bounded same-tier transient retry); queue-aware
  task; producer defers to ingest-fast; per-queue counts + all-queue reclaim
- cli: worker --tier {fast,structured,ocr}
- billing: pages_ocr usage event + pipeline_tier metadata (paid OCR billed apart)
- observability: astrolabe_ingest_queue_depth{queue,status} gauge + per-queue
  counts in nc_get_vector_sync_status / management status endpoint
- config: INGEST_ESCALATION_ENABLED (default true), INGEST_TRANSIENT_MAX_ATTEMPTS

INGEST_ESCALATION_ENABLED=false and INGEST_QUEUE=memory preserve prior behaviour.

Deck #323.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/config.py

@@ -234,6 +234,19 @@ _DEFAULTS: dict[str, Any] = {
     # queue-depth metric clean). Set false to retain succeeded rows for audit
     # (note: indexing success is also recorded in logs/metrics regardless).
     "ingest_delete_succeeded_jobs": True,
+    # Per-tier escalation on the procrastinate (postgres) ingest path (Deck
+    # #323). When true, a document that a tier cannot parse well is requeued onto
+    # the next tier's queue (fast -> structured -> ocr) via a native procrastinate
+    # queue-hop. When false the ``fast`` tier is terminal -- reproduces the
+    # pre-#323 behaviour where the cheap tier's output is indexed as-is. No effect
+    # on the in-process ``memory`` backend, which keeps the inline escalation.
+    "ingest_escalation_enabled": True,
+    # Global cap on SAME-tier retries for transient infra errors (doc fetch /
+    # embed / Qdrant blips) on the procrastinate path. Parse-quality failures
+    # escalate (one parse attempt per tier) and do NOT consume this budget; only
+    # whitelisted transient exceptions retry in place. Shared across tiers because
+    # a queue-hop cannot reset a per-tier counter (see TieredEscalationStrategy).
+    "ingest_transient_max_attempts": 5,
     "collection_metadata_source": "qdrant",  # qdrant | api
     # CP base URL for COLLECTION_METADATA_SOURCE=api (e.g. http://control-plane).
     # Required only when the source is api.
@@ -317,6 +330,7 @@ _dynaconf = Dynaconf(
         Validator("METRICS_PORT", gte=1, lte=65535),
         # Positive integers
         Validator("INGEST_STALLED_JOB_SECONDS", gte=1),
+        Validator("INGEST_TRANSIENT_MAX_ATTEMPTS", gte=1),
         Validator("VECTOR_SYNC_SCAN_INTERVAL", gte=1),
         Validator("VECTOR_SYNC_PROCESSOR_WORKERS", gte=1),
         Validator("VECTOR_SYNC_QUEUE_MAX_SIZE", gte=1),
@@ -856,6 +870,8 @@ class Settings:
     mcp_role: str = "all"  # api | worker | all (Deck #183 two-pod model)
     ingest_stalled_job_seconds: int = 300  # crashed-worker reclaim threshold
     ingest_delete_succeeded_jobs: bool = True  # drop succeeded ingest jobs
+    ingest_escalation_enabled: bool = True  # per-tier queue-hop (Deck #323)
+    ingest_transient_max_attempts: int = 5  # same-tier transient-retry cap
     collection_metadata_source: str = "qdrant"  # qdrant | api
     collection_metadata_api_url: str | None = None  # CP URL when source=api
     embedding_gateway_url: str | None = None  # required when provider=gateway
@@ -1481,6 +1497,8 @@ def get_settings() -> Settings:
         "mcp_role": "MCP_ROLE",
         "ingest_stalled_job_seconds": "INGEST_STALLED_JOB_SECONDS",
         "ingest_delete_succeeded_jobs": "INGEST_DELETE_SUCCEEDED_JOBS",
+        "ingest_escalation_enabled": "INGEST_ESCALATION_ENABLED",
+        "ingest_transient_max_attempts": "INGEST_TRANSIENT_MAX_ATTEMPTS",
         "collection_metadata_source": "COLLECTION_METADATA_SOURCE",
         "collection_metadata_api_url": "COLLECTION_METADATA_API_URL",
         "embedding_gateway_url": "EMBEDDING_GATEWAY_URL",