feat: tiered PDF processor with pypdfium2 fast path (deprecate pymupdf4llm)

Replaces single-engine pymupdf4llm extraction with a tiered pipeline (Deck #205,
follows the tier-0 classifier #855). pypdfium2 becomes the default and only
hot-path PDF extractor; pymupdf4llm is deprecated to a rollback toggle.

Why: pymupdf4llm's O(n^2) find_tables drove the OOM (#852) and the form-PDF
parse timeouts (#856), carries AGPL/commercial licensing liability, and -- per
the benchmarks -- recovers near-zero usable tables on the real corpus. pypdfium2
(Apache/BSD) extracts the same text far faster (Student 1a.pdf: 120s timeout ->
0.2s) with no table-detection bomb.

- document_processors/pypdfium2_fast.py: tier-1 "fast" processor emitting text +
  exact page_boundaries (the pdf_highlighter contract). pymupdf processor is now
  tier "structured" (the rollback engine), registered but not default.
- registry: tiered routing in ProcessorRegistry. tier-1 fast extracts, then
  classification is DERIVED from that text (classifier.classify_from_text -- no
  PDF re-open), records the classification metrics, and escalates scanned /
  no-text-layer docs to the "ocr" tier when document_ocr_enabled (default off;
  no provider yet, so fast is terminal). Wires record_document_escalation + the
  real "escalated" span attribute (was hardcoded False).
- Removes the separate _shadow_classify pass from vector/processor.py -- it
  re-opened every PDF and re-extracted text (~0.5-1.3s/doc of pure duplicated
  CPU that lowered throughput); classification now rides the tier-1 extraction.
- Settings: document_tier1_engine ("pypdfium2" default | "pymupdf" rollback,
  enum-validated), document_ocr_enabled (default false).

Tests: pypdfium2 extractor, registry tiering (fast routing, rollback, classify
recording, OCR escalation on/off), classify_from_text. Full unit suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/config.py

@@ -136,8 +136,13 @@ _DEFAULTS: dict[str, Any] = {
     "document_pdf_graphics_limit": 1000,
     "document_parse_timeout_seconds": 120.0,
     "document_parse_mem_limit_mb": 1536,
-    # Tier-0 classifier (shadow mode: emits metrics, no routing change)
+    # Tier-0 classifier (records classification metrics on the tiered path)
     "document_classify_enabled": True,
+    # Tiered PDF pipeline: pypdfium2 is the default/only hot-path extractor;
+    # "pymupdf" is a deprecated rollback escape hatch. OCR (tier-3) is the only
+    # escalation target and is off by default (no provider wired yet).
+    "document_tier1_engine": "pypdfium2",
+    "document_ocr_enabled": False,
     # Observability
     "metrics_enabled": True,
     "metrics_port": 9090,
@@ -292,6 +297,7 @@ _dynaconf = Dynaconf(
         Validator("VECTOR_SYNC_PDF_TAG", len_min=1),
         # Enum constraints
         Validator("LOG_FORMAT", is_in=["text", "json"]),
+        Validator("DOCUMENT_TIER1_ENGINE", is_in=["pypdfium2", "pymupdf"]),
         Validator(
             "LOG_LEVEL",
             is_in=["DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL"],
@@ -729,9 +735,16 @@ class Settings:
     # RLIMIT_AS in the parse subprocess (below the pod limit). Applied once per
     # worker for its lifetime, so changing it needs a pod restart.
     document_parse_mem_limit_mb: int = 1536
-    # Tier-0 classifier. Shadow mode for now: runs a cheap pre-pass over each PDF
-    # and emits classification metrics, but does NOT change routing yet.
+    # Tier-0 classifier. Records classification metrics (recommended_tier,
+    # text-quality) on the tiered path, derived from the tier-1 extraction.
     document_classify_enabled: bool = True
+    # PDF extraction engine for the ``fast`` tier. "pypdfium2" (default,
+    # permissive license, no find_tables) is the hot path; "pymupdf" is a
+    # deprecated rollback to pymupdf4llm (AGPL, graphics-limited) for one corpus.
+    document_tier1_engine: str = "pypdfium2"
+    # Route scanned/no-text-layer PDFs to the tier-3 OCR provider. Off until an
+    # OCR backend is wired; when off, the fast tier is terminal.
+    document_ocr_enabled: bool = False
 
     # Observability settings
     metrics_enabled: bool = True
@@ -1345,6 +1358,8 @@ def get_settings() -> Settings:
         "document_parse_timeout_seconds": "DOCUMENT_PARSE_TIMEOUT_SECONDS",
         "document_parse_mem_limit_mb": "DOCUMENT_PARSE_MEM_LIMIT_MB",
         "document_classify_enabled": "DOCUMENT_CLASSIFY_ENABLED",
+        "document_tier1_engine": "DOCUMENT_TIER1_ENGINE",
+        "document_ocr_enabled": "DOCUMENT_OCR_ENABLED",
         # Observability settings
         "metrics_enabled": "METRICS_ENABLED",
         "metrics_port": "METRICS_PORT",