fix(observability): address third review round

Remaining items from the PR #831 Claude review:

- processor span symmetry: add "vector_sync.total_chars" to the sparse
  embedding span (already on the dense span) and drop the redundant
  "embedding.batch_size" attribute from both spans — it always equalled
  vector_sync.chunk_count and would mislead once batching is split.
- metrics: document the deliberate "throughput counts only on full success"
  contract in record_document_parse (partial extractions flagged
  success=False are counted as a parse-error but never inflate
  pages/chars/bytes throughput).
- config: extract _detect_base_provider() -> (family, model) as the single
  source of truth for the provider-detection priority chain, shared by
  get_embedding_model_name() and get_embedding_provider_family(). Preserves
  the intentional gateway asymmetry (only the family method short-circuits).
- base.py: Optional[...] -> PEP 604 `... | None`; drop now-unused import.

Behavior unchanged (get_embedding_* outputs covered by test_config.py).

Refs Deck #175, PR #831.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-03 02:01:28 +02:00
co-authored by Claude Opus 4.8
parent 68c9e20636
commit b779627fa6
4 changed files with 49 additions and 55 deletions
+1 -2
View File
@@ -631,7 +631,6 @@ async def _index_document(
"embedding.kind": "dense",
"embedding.provider": provider,
"embedding.model": settings.get_embedding_model_name(),
"embedding.batch_size": len(chunk_texts),
},
):
embedding_service = get_embedding_service()
@@ -659,9 +658,9 @@ async def _index_document(
"vector_sync.embed_sparse",
attributes={
_ATTR_CHUNK_COUNT: len(chunk_texts),
"vector_sync.total_chars": total_chars,
"embedding.kind": "sparse",
"embedding.provider": "bm25",
"embedding.batch_size": len(chunk_texts),
},
):
bm25_service = await get_bm25_service()