feat: add opt-in MCP decomposition hook points (design §10)
Adds the seven §10.2 hook-point modules + five env vars so Astrolabe Cloud can offload document processing to the external document-processor / embedding gateway. Purely additive: with every setting unset the server behaves exactly as today, so self-hosters are unaffected (Deck #92). Hook points (all default to current monolith behavior): - config: EMBEDDING_PROVIDER, INGEST_MODE, STATUS_BACKEND, COLLECTION_METADATA_SOURCE, FACT_EVENT_EMITTER (+ supporting settings), validated in Settings.__post_init__ (fail-fast STATUS_BACKEND=local with INGEST_MODE=external); shared canonical.py. - vector/payload_keys.py + acl_hash.py: cross-impl NAMESPACE/point_id (§2.2) and BLAKE2b-128 ACL hash (§11), pinned by fixtures shared with the document-processor repo. - embedding/gateway_client.py: OpenAI-compatible GatewayProvider authenticating via M2M OIDC client-credentials (separate realm); manual-only registry entry. - vector/collection_metadata.py: sentinel-point / API metadata source with env fallback. - vector/queue/: hexagonal ingest producer ports + memory/NATS adapters (Postgres seam); INGEST_MODE=external publishes mcp.ingest.requested.{tenant} instead of the in-memory stream and skips the in-process processor pool. The lifespan becomes a composition root across both deployment branches. - vector/queue/status.py: STATUS_BACKEND=bus subscriber feeding a StatusStore the vector-sync status endpoint reads. - admin/payload_backfill.py: POST /api/v1/admin/payload-backfill (admin scope); processor writes the new payload keys; query-side ACL pre-filter gated behind ACL_PREFILTER_ENABLED (default off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
c7da612f20
commit
d883052fb8
@@ -0,0 +1,49 @@
|
||||
"""Postgres-queue ``TaskProducer`` — documented seam, not implemented.
|
||||
|
||||
The external document-processor may later drain a Postgres-backed queue instead
|
||||
of NATS to limit NATS operational overhead. Processing stays *external*; only
|
||||
the transport changes — so on this server it would be a drop-in producer swap.
|
||||
The consume side + the queue-table migration belong to that processor-side
|
||||
refactor (cross-repo), NOT here. This stub exists so the transport value and the
|
||||
``TaskProducer`` Protocol conformance are testable today.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from types import TracebackType
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ..scanner import DocumentTask
|
||||
|
||||
_NOT_IMPLEMENTED = (
|
||||
"Postgres ingest transport is a documented seam. The external "
|
||||
"document-processor owns the Postgres-drain refactor (transport swap only; "
|
||||
"processing stays external). Use INGEST_BUS_URL=nats://… for now."
|
||||
)
|
||||
|
||||
|
||||
class PostgresTaskProducer:
|
||||
@classmethod
|
||||
async def connect(cls, settings: Any) -> PostgresTaskProducer:
|
||||
raise NotImplementedError(_NOT_IMPLEMENTED)
|
||||
|
||||
async def send(self, task: DocumentTask) -> None: # pragma: no cover
|
||||
raise NotImplementedError(_NOT_IMPLEMENTED)
|
||||
|
||||
def clone(self) -> PostgresTaskProducer: # pragma: no cover
|
||||
return self
|
||||
|
||||
async def __aenter__(self) -> PostgresTaskProducer: # pragma: no cover
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc: BaseException | None,
|
||||
tb: TracebackType | None,
|
||||
) -> None: # pragma: no cover
|
||||
return None
|
||||
|
||||
async def aclose(self) -> None: # pragma: no cover
|
||||
return None
|
||||
Reference in New Issue
Block a user