admin/mcp-nextcloud
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: tier-0 document classifier in shadow mode

Browse Source 
First step of the tiered document-processor effort (Deck #203): a cheap, local
pre-pass that recommends which extraction tier a PDF should start in, emitting
metrics WITHOUT changing routing yet -- so we gather per-tenant doc-mix data
before turning escalation on.

document_processors/classifier.py: classify_pdf(content) -> DocClassification.
Page-sampled (bounded on large docs), <~1s. Cheap signals only -- text-layer
chars, a text-quality score (catches the "Student 147" failure where a text
layer exists but is mashed/space-less junk), and image coverage. A page that is
mostly a raster image routes to OCR: its content (handwriting, stamps) isn't in
any text layer. Deliberately no get_drawings/graphics-density signal -- it's
slow on the exact pages it'd flag, the hotfix's graphics_limit already makes the
parse safe, and the (future) tier-1 quality gate catches lost tables.

Validated on the sample corpus: born-digital 2-col arxiv and a digital student
record -> fast (tier 1); a scanned+handwritten form -> ocr (tier 3).

Wiring (vector/processor.py): _shadow_classify runs the classifier on PDFs in a
worker thread, best-effort (never blocks/fails indexing), gated by the new
DOCUMENT_CLASSIFY_ENABLED setting. Metrics: astrolabe_document_classified_total
{recommended_tier}, astrolabe_document_classifier_flag_total{flag},
astrolabe_document_text_quality histogram.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-05 00:12:25 +02:00
co-authored by Claude Opus 4.8
parent dd335275ac
commit 044c1da750
6 changed files with 390 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
nextcloud_mcp_server/observability/metrics.py
+38
View File
@@ -272,6 +272,30 @@ document_parse_failed_total = Counter(
["reason"], # reason: timeout | oom | error
)
# --- Tier-0 classifier (shadow mode) -----------------------------------------
#
# The classifier runs a cheap pre-pass per PDF and recommends a starting tier.
# In shadow mode it changes no routing -- these metrics gather the per-tenant
# doc-mix needed to tune the thresholds before routing is enabled.
document_classified_total = Counter(
"astrolabe_document_classified_total",
"Documents classified by tier-0, by recommended starting tier",
["recommended_tier"], # fast | ocr
)
document_classifier_flag_total = Counter(
"astrolabe_document_classifier_flag_total",
"Tier-0 classifier flags raised on documents",
["flag"], # image_heavy | scanned | bad_text_layer
)
document_text_quality = Histogram(
"astrolabe_document_text_quality",
"Tier-0 mean text-layer quality per document (0=junk, 1=clean prose)",
buckets=(0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0),
)
# --- Embedding stages ---------------------------------------------------------
embedding_duration_seconds = Histogram(
@@ -637,6 +661,20 @@ def record_document_parse_failed(reason: str) -> None:
document_parse_failed_total.labels(reason=reason).inc()
def record_document_classification(
recommended_tier: str, flags: set[str], mean_text_quality: float
) -> None:
"""Record a tier-0 classification result (shadow mode -- observability only).
Primitive args (not the DocClassification object) keep the observability
layer free of a dependency on document_processors.
"""
document_classified_total.labels(recommended_tier=recommended_tier).inc()
for flag in flags:
document_classifier_flag_total.labels(flag=flag).inc()
document_text_quality.observe(mean_text_quality)
def record_embedding(
kind: str,
provider: str,