feat(observability): astrolabe_* metrics + traces for the document pipeline

Make per-tier bottlenecks in the document-processing pipeline
(scan -> fetch -> parse -> chunk -> embed -> Qdrant upsert) visible via
metrics, traces, and structured logs. Today the document_processors layer
emits only a logger.info line: no metric, no span, and page counts live only
inside a log string. The single processing-duration histogram is unlabeled and
whole-document, so it cannot isolate parse vs embed vs upsert.

New astrolabe_* metric family (distinct from the mcp_* protocol metrics):
- astrolabe_document_parse_{duration_seconds,total} + pages/chars/bytes counters
  recorded at the ProcessorRegistry.process() boundary (covers all current and
  future processors uniformly)
- astrolabe_document_escalation_total (dormant; tiered-pipeline readiness)
- astrolabe_embedding_{duration_seconds,requests_total,chunks_total,chars_total}
- astrolabe_document_chunks_total, astrolabe_documents_indexed_total{source,status}

Tracing: new document_processor.parse child span + enriched embed/chunk span
attributes (provider/model/batch_size/chunk_count). Structured logs gain a
consistent field vocabulary (doc_id, doc_type, processor, tier, pages, chars,
byte_size, chunks, duration_ms, status) so Loki can aggregate without regex.

Tier-readiness: processor/tier are labels from day one and a tier property is
added to DocumentProcessor, so adding docling/OCR/LLM tiers later is additive
(new label values, never new metrics). Tenant comes from the kube namespace
label; mime_type/model are span attributes only (cardinality). Existing
mcp_vector_sync_*/mcp_qdrant_* are left untouched.

Refs Deck #175 (superset of #173 Phase 2). Dashboard/recording-rules follow-up
tracked on #175 for homelab-argocd.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-02 17:46:20 +02:00
co-authored by Claude Opus 4.8
parent 7e4b83dc94
commit 5d205fcaab
8 changed files with 799 additions and 14 deletions
@@ -56,6 +56,19 @@ class DocumentProcessor(ABC):
"""Unique identifier for this processor (e.g., 'unstructured', 'tesseract')."""
pass
@property
def tier(self) -> str:
"""Extraction tier this processor belongs to (escalation ladder).
Used as the ``tier`` label/attribute in observability so that adding new
extraction tiers later (docling, OCR, LLM) is purely additive. Vocabulary
(cheapest first): ``fast`` -> ``structured`` -> ``ocr`` -> ``llm``.
Defaults to ``"fast"``; override in processors that belong to a higher
tier.
"""
return "fast"
@property
@abstractmethod
def supported_mime_types(self) -> set[str]: