feat: add opt-in MCP decomposition hook points (design §10)
Adds the seven §10.2 hook-point modules + five env vars so Astrolabe Cloud can offload document processing to the external document-processor / embedding gateway. Purely additive: with every setting unset the server behaves exactly as today, so self-hosters are unaffected (Deck #92). Hook points (all default to current monolith behavior): - config: EMBEDDING_PROVIDER, INGEST_MODE, STATUS_BACKEND, COLLECTION_METADATA_SOURCE, FACT_EVENT_EMITTER (+ supporting settings), validated in Settings.__post_init__ (fail-fast STATUS_BACKEND=local with INGEST_MODE=external); shared canonical.py. - vector/payload_keys.py + acl_hash.py: cross-impl NAMESPACE/point_id (§2.2) and BLAKE2b-128 ACL hash (§11), pinned by fixtures shared with the document-processor repo. - embedding/gateway_client.py: OpenAI-compatible GatewayProvider authenticating via M2M OIDC client-credentials (separate realm); manual-only registry entry. - vector/collection_metadata.py: sentinel-point / API metadata source with env fallback. - vector/queue/: hexagonal ingest producer ports + memory/NATS adapters (Postgres seam); INGEST_MODE=external publishes mcp.ingest.requested.{tenant} instead of the in-memory stream and skips the in-process processor pool. The lifespan becomes a composition root across both deployment branches. - vector/queue/status.py: STATUS_BACKEND=bus subscriber feeding a StatusStore the vector-sync status endpoint reads. - admin/payload_backfill.py: POST /api/v1/admin/payload-backfill (admin scope); processor writes the new payload keys; query-side ACL pre-filter gated behind ACL_PREFILTER_ENABLED (default off). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
c7da612f20
commit
d883052fb8
@@ -41,8 +41,9 @@ def _warn_missing_secret_once() -> None:
|
||||
async def handle_nextcloud_webhook(request: Request) -> JSONResponse:
|
||||
"""Receive a Nextcloud webhook and queue a DocumentTask for vector sync.
|
||||
|
||||
Returns quickly so NC's webhook worker is not blocked. The send-stream is
|
||||
read from ``request.app.state.document_send_stream``; when vector sync
|
||||
Returns quickly so NC's webhook worker is not blocked. The task producer is
|
||||
read from ``request.app.state.task_producer`` (the in-memory send stream in
|
||||
local mode, or the NATS bus producer in external mode); when vector sync
|
||||
isn't running we return 503 so NC retries delivery.
|
||||
|
||||
When ``WEBHOOK_SECRET`` is set, the request must carry
|
||||
@@ -93,8 +94,8 @@ async def handle_nextcloud_webhook(request: Request) -> JSONResponse:
|
||||
status_code=200,
|
||||
)
|
||||
|
||||
send_stream = getattr(request.app.state, "document_send_stream", None)
|
||||
if send_stream is None:
|
||||
producer = getattr(request.app.state, "task_producer", None)
|
||||
if producer is None:
|
||||
logger.warning(
|
||||
"Webhook received but vector sync is not running; rejecting so NC retries"
|
||||
)
|
||||
@@ -105,7 +106,7 @@ async def handle_nextcloud_webhook(request: Request) -> JSONResponse:
|
||||
|
||||
try:
|
||||
with anyio.fail_after(1.0):
|
||||
await send_stream.send(task)
|
||||
await producer.send(task)
|
||||
except TimeoutError:
|
||||
# Queue is saturated (default 10 000 tasks). Returning 503 lets NC
|
||||
# retry rather than pinning this handler until its outbound timeout
|
||||
|
||||
Reference in New Issue
Block a user