Chris CoutinhoandClaude Opus 4.8
973f80e7b9
Billing product model finalized (Deck #281): bill pages externally, record tokens internally. Rename the data-plane metric literals to match the now- canonical contract (Deck #284) — the control plane's METRIC_EVENT_NAMES is already renamed, so the old names would be unmapped and never sync to Stripe. Rename (values unchanged): - embeddings_queries → tokens_embedded (value = real token count, already emitted by this PR; the unit upstream providers bill on). - pages_chunks → pages_embedded (value kept as len(chunk_texts) interim; TODO(#282): real normalized "pages indexed" count — real pages for paginated types, chars/tokens-per-page constant otherwise — is deferred to the instrumentation card, this only lands the name/contract). - All literals, log strings, docstrings, comments, the migration comment, and tests renamed; grep confirms zero old strings remain. Observability (new): export embedding token cost to Prometheus as astrolabe_embedding_tokens_total{provider,operation} (operation = index|query) so the billed cost unit is visible in Grafana, not just the per-tenant billing DB. Dedicated counter (doesn't inflate the existing chunk/request metrics) and always-on (independent of USAGE_METERING_ENABLED, so OSS/self-host gets it). Wired on both the indexing batch embed and the search query embed (query inside the per-request cache-miss branch, so reused embeddings aren't double-counted). Note: the rename orphans any pre-existing embeddings_queries/pages_chunks rows in tenant app DBs (CP no longer maps them) — acceptable; pipeline is inert with throwaway dev/sandbox data. Deck #284 (folded into PR #875). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
53 lines
1.9 KiB
Python
53 lines
1.9 KiB
Python
"""Token-usage surfacing: the Provider ABC estimate default + SimpleProvider.
|
|
|
|
The usage-metering hooks (Deck #67) bill ``tokens_embedded`` by tokens. Real
|
|
providers report exact counts from their API response; providers without a token
|
|
field (Simple, and the ABC default) fall back to a char-based estimate so the
|
|
billable value stays non-zero and monotone with input size.
|
|
"""
|
|
|
|
import pytest
|
|
|
|
from nextcloud_mcp_server.providers.base import Provider
|
|
from nextcloud_mcp_server.providers.simple import SimpleProvider
|
|
|
|
|
|
@pytest.mark.unit
|
|
def test_estimate_tokens_is_char_based():
|
|
"""~4-chars-per-token, ceil-rounded, summed across inputs."""
|
|
assert Provider._estimate_tokens(["abcd"]) == 1 # 4 chars
|
|
assert Provider._estimate_tokens(["abcde"]) == 2 # 5 chars → ceil(5/4)
|
|
assert Provider._estimate_tokens(["ab", "cd"]) == 1 # 4 chars total
|
|
assert Provider._estimate_tokens([]) == 0
|
|
assert Provider._estimate_tokens([""]) == 0
|
|
|
|
|
|
@pytest.mark.unit
|
|
async def test_simple_provider_embed_with_usage_estimates():
|
|
"""SimpleProvider has no real usage → estimate path via the ABC default."""
|
|
provider = SimpleProvider(dimension=8)
|
|
embedding, tokens = await provider.embed_with_usage("abcdefgh") # 8 chars → 2
|
|
|
|
assert len(embedding) == 8
|
|
assert tokens == 2
|
|
|
|
|
|
@pytest.mark.unit
|
|
async def test_simple_provider_embed_batch_with_usage_estimates():
|
|
"""Batch estimate sums character counts across all inputs."""
|
|
provider = SimpleProvider(dimension=8)
|
|
embeddings, tokens = await provider.embed_batch_with_usage(["abcd", "efgh"])
|
|
|
|
assert len(embeddings) == 2
|
|
assert tokens == 2 # 8 chars total → 2 tokens
|
|
|
|
|
|
@pytest.mark.unit
|
|
async def test_simple_provider_empty_batch_with_usage():
|
|
"""Empty batch returns no embeddings and zero tokens."""
|
|
provider = SimpleProvider(dimension=8)
|
|
embeddings, tokens = await provider.embed_batch_with_usage([])
|
|
|
|
assert embeddings == []
|
|
assert tokens == 0
|