feat(usage): record per-tenant usage events into the app DB
Deck #67 data-plane slice: tenant Pods record billable operations (embedding queries, pages/chunks embedded) into an app-DB usage_events table that the control plane later pulls read-only into the billing ledger and syncs to Stripe Meter Events. - migration 007: usage_events table (Postgres TIMESTAMPTZ/JSONB/UUID with portable SQLite fallbacks), indexed (occurred_at, metric) for the CP rollup's per-day range scan + GROUP BY metric. - UsageEventStore: best-effort, flag-gated writer reusing the shared RefreshTokenStorage engine; ON CONFLICT (event_id) DO NOTHING for idempotent retries; dialect-branched occurred_at bind. All work (incl. metadata JSON encode) is swallowed so a metering failure never surfaces to the user op. - USAGE_METERING_ENABLED flag (default off) wired through Settings + env map; off-path touches no storage, so OSS self-hosters get an empty table and zero write overhead. - two recording hooks: embeddings_queries (per nc_semantic_search, which nc_semantic_search_answer reuses) and pages_chunks (after dense embedding succeeds, covering both in-process and procrastinate paths). - storage.acquire()/.dialect public seams so the sibling store doesn't reach into the underscored internal. - tests parametrized over SQLite + Postgres: flag-off no-op, roundtrip, ON CONFLICT dedup, JSON/NULL metadata, and the best-effort swallow of both DB errors and unserializable metadata. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
42505f8f87
commit
1c6b1a84ea
@@ -29,6 +29,7 @@ from nextcloud_mcp_server.observability.metrics import (
|
||||
)
|
||||
from nextcloud_mcp_server.observability.tracing import trace_operation
|
||||
from nextcloud_mcp_server.search.pdf_highlighter import PDFHighlighter
|
||||
from nextcloud_mcp_server.usage import UsageEventStore
|
||||
from nextcloud_mcp_server.vector import payload_keys
|
||||
from nextcloud_mcp_server.vector.document_chunker import (
|
||||
DocumentChunker,
|
||||
@@ -821,6 +822,28 @@ async def _index_document(
|
||||
chunks=len(chunk_texts),
|
||||
chars=total_chars,
|
||||
)
|
||||
# Usage metering (Deck #67): record chunks embedded as a billable
|
||||
# 'pages_chunks' event. Best-effort and gated on the flag so the
|
||||
# off-path (OSS default) touches no storage; placed after the
|
||||
# embedding succeeds so it can never affect the indexing path.
|
||||
if settings.usage_metering_enabled:
|
||||
try:
|
||||
store = await UsageEventStore.shared()
|
||||
await store.record_usage_event(
|
||||
metric="pages_chunks",
|
||||
value=len(chunk_texts),
|
||||
metadata={
|
||||
"provider": provider,
|
||||
"model": settings.get_embedding_model_name(),
|
||||
"doc_type": doc_task.doc_type,
|
||||
"user_id": doc_task.user_id,
|
||||
"total_chars": total_chars,
|
||||
},
|
||||
)
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"usage metering hook (pages_chunks) skipped", exc_info=True
|
||||
)
|
||||
|
||||
async def generate_sparse_embeddings():
|
||||
"""Generate sparse embeddings (BM25 for keyword matching)."""
|
||||
|
||||
Reference in New Issue
Block a user