Chris CoutinhoandClaude Opus 4.8
d883052fb8
Adds the seven §10.2 hook-point modules + five env vars so Astrolabe Cloud can offload document processing to the external document-processor / embedding gateway. Purely additive: with every setting unset the server behaves exactly as today, so self-hosters are unaffected (Deck #92). Hook points (all default to current monolith behavior): - config: EMBEDDING_PROVIDER, INGEST_MODE, STATUS_BACKEND, COLLECTION_METADATA_SOURCE, FACT_EVENT_EMITTER (+ supporting settings), validated in Settings.__post_init__ (fail-fast STATUS_BACKEND=local with INGEST_MODE=external); shared canonical.py. - vector/payload_keys.py + acl_hash.py: cross-impl NAMESPACE/point_id (§2.2) and BLAKE2b-128 ACL hash (§11), pinned by fixtures shared with the document-processor repo. - embedding/gateway_client.py: OpenAI-compatible GatewayProvider authenticating via M2M OIDC client-credentials (separate realm); manual-only registry entry. - vector/collection_metadata.py: sentinel-point / API metadata source with env fallback. - vector/queue/: hexagonal ingest producer ports + memory/NATS adapters (Postgres seam); INGEST_MODE=external publishes mcp.ingest.requested.{tenant} instead of the in-memory stream and skips the in-process processor pool. The lifespan becomes a composition root across both deployment branches. - vector/queue/status.py: STATUS_BACKEND=bus subscriber feeding a StatusStore the vector-sync status endpoint reads. - admin/payload_backfill.py: POST /api/v1/admin/payload-backfill (admin scope); processor writes the new payload keys; query-side ACL pre-filter gated behind ACL_PREFILTER_ENABLED (default off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
27 lines
988 B
Python
27 lines
988 B
Python
"""Canonical JSON encoding shared across cross-implementation hashes.
|
|
|
|
The Astrolabe Cloud decomposition (design §2.3) fixes a single canonical JSON
|
|
encoding so hashes computed here match those computed independently by the
|
|
external document-processor and embedding-gateway services. Any drift in
|
|
separators, key ordering, or unicode handling would break NATS dedup keys,
|
|
Qdrant point-ID idempotency, and ACL-hash compatibility.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import json
|
|
from typing import Any
|
|
|
|
|
|
def canonical_json(obj: Any) -> bytes:
|
|
"""Encode ``obj`` to canonical JSON bytes.
|
|
|
|
Deterministic across implementations: sorted keys, no inter-token
|
|
whitespace, non-ASCII preserved (UTF-8). Consumers: the NATS
|
|
``Nats-Msg-Id`` dedup header (vector/queue/nats.py), Qdrant point IDs
|
|
(vector/payload_keys.py), and ACL hashes (acl_hash.py).
|
|
"""
|
|
return json.dumps(
|
|
obj, sort_keys=True, separators=(",", ":"), ensure_ascii=False
|
|
).encode("utf-8")
|