fix(vector): don't inflate qdrant-error metric on embed drops (#893 r3)

Round-3 review on PR #893:
- record_qdrant_operation("upsert","error") now fires only when the exhausted
  retry was actually a Qdrant failure (reason=="qdrant"); an embed/connection
  failure exhausts retries before Qdrant is called, so attributing it to
  mcp_qdrant_operations_total{error} inflated that signal. The cause is still
  captured by record_ingest_dropped.
- Add test_mistral_embed_retries_on_5xx: exercises the full Mistral retry path
  (5xx SDKError then success), not just the predicate.
- Add test_generate_does_not_retry_on_bad_request: generate() fast-fails on a
  permanent 4xx.
- Move astrolabe_vector_ingest_dropped_total's definition into the astrolabe_
  pipeline-metrics block (was in the mcp_ section).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-11 06:17:59 +02:00
co-authored by Claude Opus 4.8
parent 8f7a8432f5
commit c4b6d4a017
4 changed files with 66 additions and 20 deletions
+11 -8
View File
@@ -528,14 +528,17 @@ async def process_document(
"drop_reason": reason,
},
)
# Record the failed Qdrant upsert. The processing-error
# metric is recorded once by the outer handler below, so
# exhausted-retry failures aren't double-counted. The
# drop counter is labelled by cause so a transient
# rollover (connection/timeout) is alertable distinctly.
# The document is NOT marked failed, so the next scan
# re-picks it (re-queue via the scan loop, card 309).
record_qdrant_operation("upsert", "error")
# Count a failed Qdrant upsert ONLY when Qdrant was the
# failing component; an embed/connection failure exhausts
# retries before Qdrant is ever called, so attributing it
# to mcp_qdrant_operations_total{error} would inflate that
# signal. The cause is captured by record_ingest_dropped
# instead, and the processing-error metric is recorded
# once by the outer handler below (no double-count). The
# document is NOT marked failed, so the next scan re-picks
# it (re-queue via the scan loop, card 309).
if reason == "qdrant":
record_qdrant_operation("upsert", "error")
record_ingest_dropped(reason)
raise