feat: add opt-in MCP decomposition hook points (design §10)
Adds the seven §10.2 hook-point modules + five env vars so Astrolabe Cloud can offload document processing to the external document-processor / embedding gateway. Purely additive: with every setting unset the server behaves exactly as today, so self-hosters are unaffected (Deck #92). Hook points (all default to current monolith behavior): - config: EMBEDDING_PROVIDER, INGEST_MODE, STATUS_BACKEND, COLLECTION_METADATA_SOURCE, FACT_EVENT_EMITTER (+ supporting settings), validated in Settings.__post_init__ (fail-fast STATUS_BACKEND=local with INGEST_MODE=external); shared canonical.py. - vector/payload_keys.py + acl_hash.py: cross-impl NAMESPACE/point_id (§2.2) and BLAKE2b-128 ACL hash (§11), pinned by fixtures shared with the document-processor repo. - embedding/gateway_client.py: OpenAI-compatible GatewayProvider authenticating via M2M OIDC client-credentials (separate realm); manual-only registry entry. - vector/collection_metadata.py: sentinel-point / API metadata source with env fallback. - vector/queue/: hexagonal ingest producer ports + memory/NATS adapters (Postgres seam); INGEST_MODE=external publishes mcp.ingest.requested.{tenant} instead of the in-memory stream and skips the in-process processor pool. The lifespan becomes a composition root across both deployment branches. - vector/queue/status.py: STATUS_BACKEND=bus subscriber feeding a StatusStore the vector-sync status endpoint reads. - admin/payload_backfill.py: POST /api/v1/admin/payload-backfill (admin scope); processor writes the new payload keys; query-side ACL pre-filter gated behind ACL_PREFILTER_ENABLED (default off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
c7da612f20
commit
d883052fb8
@@ -0,0 +1,45 @@
|
||||
"""Qdrant payload-key constants + the shared point-ID namespace (design §2.2).
|
||||
|
||||
These names and the ``NAMESPACE`` UUID are a cross-implementation contract: the
|
||||
external document-processor (astrolabe-cloud-website) computes identical chunk
|
||||
point IDs and writes the same payload keys. Divergence in ``NAMESPACE`` would
|
||||
break Qdrant upsert idempotency and duplicate chunks at the Phase 2 cutover, so
|
||||
the literal is pinned by a fixture checked into both repos
|
||||
(``tests/fixtures/namespace_uuid.txt``) and asserted equal in each repo's suite.
|
||||
|
||||
Even in the default local mode the MCP server writes these keys on upsert, so a
|
||||
later migration to the external processor is friction-free (design §10.2).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
from nextcloud_mcp_server.canonical import canonical_json
|
||||
|
||||
# Payload keys introduced by the decomposition (design §10.2).
|
||||
EMBEDDING_IDENTITY = "embedding_identity"
|
||||
ACL_HASH = "acl_hash"
|
||||
PROCESSOR_VERSION = "processor_version"
|
||||
PARSED_AT = "parsed_at"
|
||||
PIPELINE_TIER = "pipeline_tier"
|
||||
|
||||
# Fixed platform namespace for deterministic chunk point IDs (design §2.2).
|
||||
# Derived once from ``uuid5(NAMESPACE_DNS, "astrolabe.cloud/mcp/point-id/v1")``
|
||||
# and pinned here as a literal so neither repo recomputes it. DO NOT CHANGE —
|
||||
# see the module docstring for the cross-implementation contract.
|
||||
NAMESPACE = uuid.UUID("b050b8ac-c6aa-5566-9584-506b39c1c096")
|
||||
|
||||
|
||||
def point_id(tenant_id: str, doc_id: str, chunk_index: int) -> str:
|
||||
"""Deterministic chunk point ID, identical across MCP server + processor.
|
||||
|
||||
``uuid5(NAMESPACE, canonical_json({...}))`` per design §2.2. The canonical
|
||||
JSON name (sorted keys, no whitespace) must match the processor
|
||||
byte-for-byte so re-indexing the same chunk upserts in place rather than
|
||||
duplicating.
|
||||
"""
|
||||
name = canonical_json(
|
||||
{"tenant_id": tenant_id, "doc_id": doc_id, "chunk_index": chunk_index}
|
||||
).decode("utf-8")
|
||||
return str(uuid.uuid5(NAMESPACE, name))
|
||||
Reference in New Issue
Block a user