Billing product model finalized (Deck #281): bill pages externally, record
tokens internally. Rename the data-plane metric literals to match the now-
canonical contract (Deck #284) — the control plane's METRIC_EVENT_NAMES is
already renamed, so the old names would be unmapped and never sync to Stripe.
Rename (values unchanged):
- embeddings_queries → tokens_embedded (value = real token count, already
emitted by this PR; the unit upstream providers bill on).
- pages_chunks → pages_embedded (value kept as len(chunk_texts) interim;
TODO(#282): real normalized "pages indexed" count — real pages for paginated
types, chars/tokens-per-page constant otherwise — is deferred to the
instrumentation card, this only lands the name/contract).
- All literals, log strings, docstrings, comments, the migration comment, and
tests renamed; grep confirms zero old strings remain.
Observability (new): export embedding token cost to Prometheus as
astrolabe_embedding_tokens_total{provider,operation} (operation = index|query)
so the billed cost unit is visible in Grafana, not just the per-tenant billing
DB. Dedicated counter (doesn't inflate the existing chunk/request metrics) and
always-on (independent of USAGE_METERING_ENABLED, so OSS/self-host gets it).
Wired on both the indexing batch embed and the search query embed (query inside
the per-request cache-miss branch, so reused embeddings aren't double-counted).
Note: the rename orphans any pre-existing embeddings_queries/pages_chunks rows
in tenant app DBs (CP no longer maps them) — acceptable; pipeline is inert with
throwaway dev/sandbox data.
Deck #284 (folded into PR #875).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- store: guard UsageEventStore.shared() with a class-level anyio.Lock so
two concurrent cold-start callers don't both build (and one silently
overwrite) the cached instance — mirrors get_shared_storage(). Document
that tests should construct the store directly to avoid singleton leak.
- migration: rename 20260610 -> 20260607 and fix Create Date to today so
`alembic history` isn't future-dated (revision id 007 / down_revision
006 unchanged; single head verified).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- remove accidentally-committed .claude/scheduled_tasks.lock (Claude Code
runtime artifact swept in by `git add -A`) and gitignore it; the rest
of .claude/ stays tracked.
- store: cache UsageEventStore.shared() as a process-wide instance so the
hot search path doesn't allocate a fresh wrapper per metered query (the
wrapper is stateless beyond its storage handle).
- hooks: pass enabled=True directly (the outer guard already confirmed
the flag) instead of re-reading settings.usage_metering_enabled.
- migration: document the no-TTL retention design (control-plane rollup
owns the lifecycle; the data plane only appends).
- tests: assert the best-effort error path logs at WARNING (observability
contract).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- store: add optional `enabled` param to record_usage_event so hot-path
callers (nc_semantic_search) pass the already-resolved flag instead of
forcing a second uncached Settings build (ADR-024); falls back to
get_settings() when None so the store stays self-gating for standalone
use.
- hooks: thread enabled= through both call sites; bump the outer
shared()/construction failure log from debug → warning so "metering
enabled but no billing data" is visible at the default INFO level.
- migration: instantiate postgresql.JSONB() to match the sibling
TIMESTAMP(timezone=True) column.
- tests: fix the misleading "asyncpg returns JSONB as a JSON string"
comment; add occurred_at dialect round-trip test and an enabled-param
short-circuit test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Deck #67 data-plane slice: tenant Pods record billable operations
(embedding queries, pages/chunks embedded) into an app-DB usage_events
table that the control plane later pulls read-only into the billing
ledger and syncs to Stripe Meter Events.
- migration 007: usage_events table (Postgres TIMESTAMPTZ/JSONB/UUID
with portable SQLite fallbacks), indexed (occurred_at, metric) for the
CP rollup's per-day range scan + GROUP BY metric.
- UsageEventStore: best-effort, flag-gated writer reusing the shared
RefreshTokenStorage engine; ON CONFLICT (event_id) DO NOTHING for
idempotent retries; dialect-branched occurred_at bind. All work
(incl. metadata JSON encode) is swallowed so a metering failure never
surfaces to the user op.
- USAGE_METERING_ENABLED flag (default off) wired through Settings +
env map; off-path touches no storage, so OSS self-hosters get an empty
table and zero write overhead.
- two recording hooks: embeddings_queries (per nc_semantic_search, which
nc_semantic_search_answer reuses) and pages_chunks (after dense
embedding succeeds, covering both in-process and procrastinate paths).
- storage.acquire()/.dialect public seams so the sibling store doesn't
reach into the underscored internal.
- tests parametrized over SQLite + Postgres: flag-off no-op, roundtrip,
ON CONFLICT dedup, JSON/NULL metadata, and the best-effort swallow of
both DB errors and unserializable metadata.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
