feat: add opt-in MCP decomposition hook points (design §10)
Adds the seven §10.2 hook-point modules + five env vars so Astrolabe Cloud can offload document processing to the external document-processor / embedding gateway. Purely additive: with every setting unset the server behaves exactly as today, so self-hosters are unaffected (Deck #92). Hook points (all default to current monolith behavior): - config: EMBEDDING_PROVIDER, INGEST_MODE, STATUS_BACKEND, COLLECTION_METADATA_SOURCE, FACT_EVENT_EMITTER (+ supporting settings), validated in Settings.__post_init__ (fail-fast STATUS_BACKEND=local with INGEST_MODE=external); shared canonical.py. - vector/payload_keys.py + acl_hash.py: cross-impl NAMESPACE/point_id (§2.2) and BLAKE2b-128 ACL hash (§11), pinned by fixtures shared with the document-processor repo. - embedding/gateway_client.py: OpenAI-compatible GatewayProvider authenticating via M2M OIDC client-credentials (separate realm); manual-only registry entry. - vector/collection_metadata.py: sentinel-point / API metadata source with env fallback. - vector/queue/: hexagonal ingest producer ports + memory/NATS adapters (Postgres seam); INGEST_MODE=external publishes mcp.ingest.requested.{tenant} instead of the in-memory stream and skips the in-process processor pool. The lifespan becomes a composition root across both deployment branches. - vector/queue/status.py: STATUS_BACKEND=bus subscriber feeding a StatusStore the vector-sync status endpoint reads. - admin/payload_backfill.py: POST /api/v1/admin/payload-backfill (admin scope); processor writes the new payload keys; query-side ACL pre-filter gated behind ACL_PREFILTER_ENABLED (default off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
c7da612f20
commit
d883052fb8
@@ -50,6 +50,44 @@ class ProviderRegistry:
|
||||
"""
|
||||
settings = get_settings()
|
||||
|
||||
# 0. Gateway (manual-only; never autodetected). When
|
||||
# EMBEDDING_PROVIDER=gateway, bypass the autodetect chain entirely
|
||||
# and route embeddings through the Astrolabe Cloud embedding gateway
|
||||
# (design §10.1/§10.2). Lazy import to avoid an import cycle
|
||||
# (embedding/gateway_client → providers/openai → ... → registry).
|
||||
if settings.embedding_provider == "gateway":
|
||||
from ..embedding.gateway_client import ( # noqa: PLC0415
|
||||
GatewayProvider,
|
||||
GatewayTokenProvider,
|
||||
)
|
||||
|
||||
# Settings.__post_init__ guarantees the URL is set when the
|
||||
# provider is gateway; assert narrows the type for the checker.
|
||||
assert settings.embedding_gateway_url is not None
|
||||
# __post_init__ enforces all-or-nothing on the M2M creds, so
|
||||
# checking one is enough to know the full triple is present.
|
||||
token_provider = None
|
||||
if settings.embedding_gateway_client_id:
|
||||
assert settings.embedding_gateway_token_url is not None
|
||||
assert settings.embedding_gateway_client_secret is not None
|
||||
token_provider = GatewayTokenProvider(
|
||||
token_url=settings.embedding_gateway_token_url,
|
||||
client_id=settings.embedding_gateway_client_id,
|
||||
client_secret=settings.embedding_gateway_client_secret,
|
||||
scope=settings.embedding_gateway_scope,
|
||||
)
|
||||
logger.info(
|
||||
"Using embedding gateway provider: url=%s, model=%s, auth=%s",
|
||||
settings.embedding_gateway_url,
|
||||
settings.embedding_gateway_model,
|
||||
"oidc-m2m" if token_provider else "none",
|
||||
)
|
||||
return GatewayProvider(
|
||||
base_url=settings.embedding_gateway_url,
|
||||
embedding_model=settings.embedding_gateway_model,
|
||||
token_provider=token_provider,
|
||||
)
|
||||
|
||||
# 1. Bedrock
|
||||
if (
|
||||
settings.aws_region
|
||||
|
||||
Reference in New Issue
Block a user