Chris CoutinhoandClaude Opus 4.8
3bd1b46d9c
Adds the OCR escalation target the tiered registry already routes to. Scanned /
no-text-layer PDFs (the tier-0 "ocr" verdict) escalate here when
document_ocr_enabled (default off).
Two interchangeable backends, selected by document_ocr_provider
(auto | gateway | mistral | none):
- gateway: POST to the Astrolabe Cloud model gateway's /v1/ocr -- the same
M2M-authenticated gateway as embeddings, so NO provider keys live in the pod
(the platform default; reuses EMBEDDING_GATEWAY_URL + the M2M creds).
- mistral: call the Mistral OCR API directly from the pod (MISTRAL_API_KEY), for
self-hosters / deployments without the gateway.
"auto" prefers the gateway, then direct Mistral.
Both return per-page markdown joined into text + exact page_boundaries (the
pdf_highlighter contract; bbox re-derived from the PDF bytes as for other tiers).
Validated end-to-end via direct Mistral on the scanned Student 147.pdf:
success, 15 pages, 22k chars, offsets exact, ~4s.
Settings: document_ocr_provider (enum-validated), document_ocr_model
("mistral/mistral-ocr-latest" -- gateway routes on the prefix, the direct mistral
backend strips it). OcrProcessor registered at lowest priority so it is never the
non-tiered default.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
29 lines
1.1 KiB
Python
29 lines
1.1 KiB
Python
"""Document processing plugins for extracting text from various file formats."""
|
|
|
|
from .base import DocumentProcessor, ProcessingResult, ProcessorError
|
|
from .ocr import OcrProcessor
|
|
from .pymupdf import PyMuPDFProcessor
|
|
from .pypdfium2_fast import Pypdfium2FastProcessor
|
|
from .registry import ProcessorRegistry, get_registry
|
|
|
|
# Register processors at module initialization. The tiered PDF pipeline selects
|
|
# by tier (not priority): Pypdfium2FastProcessor is the ``fast`` tier,
|
|
# PyMuPDFProcessor the ``structured`` rollback, and OcrProcessor the ``ocr``
|
|
# escalation target (reached only when document_ocr_enabled). OcrProcessor gets
|
|
# the lowest priority so it is never the non-tiered default for PDFs.
|
|
_registry = get_registry()
|
|
_registry.register(Pypdfium2FastProcessor(), priority=20)
|
|
_registry.register(PyMuPDFProcessor(), priority=10)
|
|
_registry.register(OcrProcessor(), priority=1)
|
|
|
|
__all__ = [
|
|
"DocumentProcessor",
|
|
"ProcessingResult",
|
|
"ProcessorError",
|
|
"ProcessorRegistry",
|
|
"get_registry",
|
|
"PyMuPDFProcessor",
|
|
"Pypdfium2FastProcessor",
|
|
"OcrProcessor",
|
|
]
|