Chris CoutinhoandClaude Opus 4.8
c21804fbbc
Insert a configurable in-cluster OCR rung into the escalation ladder (Deck #353): a tier2-eligible doc is OCR'd on the on-demand burst GPU before falling through to paid upstream OCR. The in-cluster backend is reached ONLY via the embedding gateway (model prefix routes to the GPU over the tailnet) and is a config value (default surya/surya-ocr-2, swappable to e.g. lightonocr) — never hard-coded. Ladder: fast -> structured -> ocr-incluster -> ocr-upstream (queues ingest-ocr-incluster / ingest-ocr-upstream). - escalation.py: 4-tier ladder; in-cluster flag folded into the dead-letter signature. - ocr.py: OcrProcessor(name, tier, model_setting, gateway_only); build_ocr_backend( ..., model=, gateway_only=) — gateway_only forces the gateway backend (never the direct Mistral fallback), disabling the tier with a warning if no gateway URL. - registry.py: per-rung enable map; scanned docs target minimum="ocr-incluster"; inline path runs the cheapest available OCR rung. - procrastinate.py: two OCR queues; legacy ingest-ocr kept as a drain target. - config.py: DOCUMENT_OCR_INCLUSTER_ENABLED (off) + DOCUMENT_OCR_INCLUSTER_MODEL. - __init__.py: register the two OCR instances; vector/processor.py: pages_ocr metered for the upstream (paid) rung only; cli.py: new --tier choices + legacy drain. - metrics.py: zero the legacy ingest-ocr queue gauge during rollout. - tests: migrated to the split ladder + new tests (gateway-only forcing, per-tier model incl. lightonocr override, no-hard-coded-surya guard). 1792 pass; ruff + ty green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
49 lines
1.5 KiB
Python
49 lines
1.5 KiB
Python
"""Unit tests for the escalation-tier signature used by dead-letter keying.
|
|
|
|
``escalation_tiers_signature`` fingerprints the runtime escalation config so a
|
|
dead-lettered document becomes retryable when a new tier appears (e.g. an
|
|
operator enables OCR). It must be settings-derived (role-independent) and must
|
|
change when OCR is toggled.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from types import SimpleNamespace
|
|
|
|
import pytest
|
|
|
|
from nextcloud_mcp_server.document_processors.escalation import (
|
|
escalation_tiers_signature,
|
|
)
|
|
|
|
pytestmark = pytest.mark.unit
|
|
|
|
|
|
def _settings(
|
|
*, ocr: bool, ocr_incluster: bool = False, engine: str = "pypdfium2"
|
|
) -> SimpleNamespace:
|
|
return SimpleNamespace(
|
|
document_ocr_enabled=ocr,
|
|
document_ocr_incluster_enabled=ocr_incluster,
|
|
document_tier1_engine=engine,
|
|
)
|
|
|
|
|
|
def test_signature_is_stable_for_same_config() -> None:
|
|
assert escalation_tiers_signature(
|
|
_settings(ocr=False)
|
|
) == escalation_tiers_signature(_settings(ocr=False))
|
|
|
|
|
|
def test_enabling_ocr_changes_signature() -> None:
|
|
# Enabling OCR adds an escalation tier -> previously dead-lettered docs retry.
|
|
assert escalation_tiers_signature(
|
|
_settings(ocr=False)
|
|
) != escalation_tiers_signature(_settings(ocr=True))
|
|
|
|
|
|
def test_tier1_engine_change_changes_signature() -> None:
|
|
assert escalation_tiers_signature(
|
|
_settings(ocr=False, engine="pypdfium2")
|
|
) != escalation_tiers_signature(_settings(ocr=False, engine="pymupdf"))
|