docs(usage): note Ollama batch-token semantics + Prometheus/billing query divergence
Round-8 claude-review (no blockers; comment-only): - 🟡 Documented that Ollama's /api/embed prompt_eval_count is assumed batch-level total and is unverified against a live instance (Ollama isn't the Cloud billing provider); if it proves last-item-only, switch to per-item summing. The char estimate already covers versions that omit the field. - 🟡 Noted on the astrolabe_embedding_tokens_total counter that operation="query" is recorded pre-Qdrant, so it can legitimately exceed the billing-store tokens_embedded aggregate when a search fails post-embed — dashboards shouldn't alert on that healthy gap. Deferred (reviewer: "minor nit, acceptable"): record_indexing_usage awaited in the task group — the group awaits all child tasks regardless, the write is best-effort + fast, and start_soon would need the tg threaded into the closure for marginal gain. Deck #284. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
9369832977
commit
28de737c26
@@ -342,6 +342,12 @@ embedding_chars_total = Counter(
|
||||
# measure, Deck #67). On a dedicated counter (not folded into the chunk/request
|
||||
# metrics above) so query embeds don't inflate indexing dashboards; labelled by
|
||||
# operation = index | query. Always emitted, independent of USAGE_METERING_ENABLED.
|
||||
#
|
||||
# Dashboard note: operation="query" is recorded at embed time (before Qdrant /
|
||||
# verify-on-read), whereas the billing-store tokens_embedded row is written only
|
||||
# after the search fully succeeds. So this counter can legitimately exceed the
|
||||
# billing aggregate when a search fails post-embed — don't alert on that gap as
|
||||
# a divergence bug.
|
||||
embedding_tokens_total = Counter(
|
||||
"astrolabe_embedding_tokens_total",
|
||||
"Total embedding tokens consumed (provider-reported or estimated)",
|
||||
|
||||
@@ -164,6 +164,13 @@ class OllamaProvider(Provider):
|
||||
if self._dimension is None and data["embeddings"]:
|
||||
self._dimension = len(data["embeddings"][0])
|
||||
|
||||
# ``prompt_eval_count`` is assumed to be the batch-level total for a
|
||||
# multi-input /api/embed call. Ollama's API docs aren't explicit
|
||||
# about batch aggregation; if a version reports only the last
|
||||
# input's tokens this understates the batch. Unverified against a
|
||||
# live instance — Ollama isn't the Cloud billing provider (Mistral
|
||||
# is). If it proves last-item-only, switch to per-item requests and
|
||||
# sum. The char-based estimate covers versions that omit the field.
|
||||
prompt_eval = data.get("prompt_eval_count")
|
||||
total_tokens += (
|
||||
round(prompt_eval)
|
||||
|
||||
Reference in New Issue
Block a user