feat(usage): meter embedding tokens as embeddings_queries on both paths

embeddings_queries now records the embedding request's token count (the unit
upstream providers bill on) instead of an operation count, and fires on the
indexing path too. Previously only semantic search recorded it (value=1), so a
re-indexing run produced no embeddings_queries events at all — only pages_chunks.

- Provider layer: additive embed_with_usage / embed_batch_with_usage surface the
  per-request token count (Mistral/OpenAI usage.total_tokens, Bedrock Titan
  inputTextTokenCount, Ollama prompt_eval_count); a char-based estimate is the
  fallback (Simple, and any provider/response without a token field). Gateway and
  EmbeddingService forward through. The count travels as a return value / a
  per-request SearchAlgorithm attribute — never on the singleton — so concurrent
  indexing + search can't mis-attribute bills.
- Indexing (vector/processor.py): records embeddings_queries (value=batch tokens)
  alongside the existing pages_chunks event.
- Search (server/semantic.py): value is now the query embedding's token count,
  relayed from BM25HybridSearchAlgorithm via query_token_count.

The astrolabe_embeddings_queries Stripe meter (sum aggregation) now sums tokens
with no CP/Terraform change. The meter "queries"->tokens naming/unit
clarification (homelab-terraform #254) + CP rollup/portal copy is a follow-up.

Deck #67.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-08 00:53:58 +02:00
co-authored by Claude Opus 4.8
parent fe17994c4d
commit 64318f0b25
17 changed files with 647 additions and 35 deletions
+13 -5
View File
@@ -528,10 +528,18 @@ def configure_semantic_tools(mcp: FastMCP):
logger.info("Returning %d results from BM25 hybrid search", len(results))
# Usage metering (Deck #67): one billable 'embeddings_queries'
# event per successful search (the query embedding is the metered
# cost). Best-effort and gated on the flag so the off-path touches
# no storage. nc_semantic_search_answer reuses this tool, so it
# records here too — do not add a second hook there.
# event per successful search. The value is the query embedding's
# token count (provider-reported, or estimated) — the unit upstream
# providers bill on, and the same metric the indexing path records
# for chunk embeddings. Best-effort and gated on the flag so the
# off-path touches no storage. nc_semantic_search_answer reuses this
# tool, so it records here too — do not add a second hook there.
#
# query_token_count is set by BM25HybridSearchAlgorithm during the
# search() above. The doc_types loop reuses one search_algo instance
# for the same query string, so the final value is the single query
# embedding's cost (matches the prior one-query semantics). Falls
# back to 0 only if the embedding never ran (e.g. a pre-embed error).
#
# Privacy note: user_id stays tenant-local. The CP rollup
# aggregates GROUP BY (day, metric) into usage_daily, which has no
@@ -543,7 +551,7 @@ def configure_semantic_tools(mcp: FastMCP):
store = await UsageEventStore.shared()
await store.record_usage_event(
metric="embeddings_queries",
value=1,
value=search_algo.query_token_count or 0,
metadata={
"user_id": username,
"fusion": fusion,