feat: replace NATS ingest with procrastinate Postgres queue (#183)

Re-architect document ingest from the shared NATS-glued document-processor to a
per-tenant, in-process model owned by nextcloud-mcp-server (Deck #183). The MCP
server now owns both sides of ingest:

- Producer (api role): the scanner defers one job per changed document into the
  app's Postgres via procrastinate (queueing_lock dedup; no execution lock, so a
  crashed worker can't deadlock a doc — Qdrant upserts are idempotent).
- Consumer (worker role): `nextcloud-mcp-server worker` drains the queue and runs
  the existing process_document pipeline; a periodic task reclaims jobs orphaned
  in `doing` by a crash.

INGEST_QUEUE selects the transport (auto: postgres when DATABASE_URL is Postgres,
else the in-process anyio queue for SQLite/dev). procrastinate manages its own
tables (applied on a fresh DB at startup and by `db upgrade`). The vector-sync
status surface reads job counts from Postgres in postgres mode. procrastinate +
psycopg3 ship in the [postgres] extra; the app's own engine still uses asyncpg
(driver unification is a follow-up handled in the rendered Helm chart).

NATS JetStream, the Postgres-queue stub, the bus status subscriber, and nats-py
are removed.

BREAKING CHANGE: the external-NATS-ingest env vars are removed
(INGEST_MODE, STATUS_BACKEND, INGEST_BUS_URL, INGEST_BUS_NUM_REPLICAS,
FACT_EVENT_EMITTER). Use INGEST_QUEUE (memory|postgres) and the `worker`
command instead. TENANT_ID is retained (no longer NATS-subject-charset-validated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/vector/queue/factory.py

@@ -1,16 +1,17 @@
-"""Composition root for the ingest producer (design §10).
+"""Composition root for the ingest producer (Deck #183).
 
-``build_external_producer`` is called by the lifespan only when
-``INGEST_MODE=external``; local mode uses the in-memory stream directly (it
-already satisfies :class:`TaskProducer`). The transport under ``external`` is
-selected from the ``INGEST_BUS_URL`` scheme (``nats://`` now, ``postgres://``
-later) so moving the external processor to Postgres needs no new INGEST_MODE.
+The transport is selected from ``INGEST_QUEUE``:
+
+- ``postgres`` → :class:`ProcrastinateTaskProducer`, which defers jobs into the
+  per-tenant Postgres for the out-of-process ``worker`` role to drain.
+- ``memory`` (SQLite/dev default) → the in-process anyio stream, built inline by
+  the server lifespan (it owns both the send and receive ends), so it is not
+  produced here.
 """
 
 from __future__ import annotations
 
 import logging
-from urllib.parse import urlsplit
 
 from ...config import Settings
 from .ports import TaskProducer
@@ -18,43 +19,19 @@ from .ports import TaskProducer
 logger = logging.getLogger(__name__)
 
 
-def _transport_for(url: str) -> str:
-    scheme = urlsplit(url).scheme.lower()
-    if scheme.startswith("postgres"):
-        return "postgres"
-    if not scheme.startswith("nats"):
-        logger.warning(
-            "INGEST_BUS_URL scheme %r is neither nats:// nor postgres://; "
-            "defaulting to the NATS transport",
-            scheme,
-        )
-    return "nats"
+async def build_producer(settings: Settings) -> TaskProducer:
+    """Build the Postgres (procrastinate) ingest producer.
 
-
-async def build_external_producer(settings: Settings) -> TaskProducer:
-    """Build the external-ingest producer for the configured transport.
-
-    Precondition: ``settings.ingest_mode == "external"`` (so __post_init__ has
-    guaranteed ``ingest_bus_url`` and ``tenant_id`` are set).
+    Precondition: ``settings.ingest_queue == "postgres"`` (the memory transport
+    is constructed inline by the lifespan because it needs the paired receive
+    stream for the in-process processor pool).
     """
-    # Defence-in-depth (robust under ``python -O``, which strips asserts):
-    # __post_init__ already guarantees these when ingest_mode == external.
-    if settings.ingest_bus_url is None or settings.tenant_id is None:
+    if settings.ingest_queue != "postgres":
         raise ValueError(
-            "build_external_producer requires INGEST_BUS_URL and TENANT_ID "
-            "(guaranteed by Settings validation when INGEST_MODE=external)"
+            "build_producer is only for INGEST_QUEUE=postgres; the memory "
+            f"transport is built inline by the lifespan (got {settings.ingest_queue!r})"
         )
 
-    transport = _transport_for(settings.ingest_bus_url)
-    if transport == "postgres":
-        from .postgres import PostgresTaskProducer  # noqa: PLC0415
+    from .procrastinate import ProcrastinateTaskProducer  # noqa: PLC0415
 
-        return await PostgresTaskProducer.connect(settings)
-
-    from .nats import NatsTaskProducer  # noqa: PLC0415
-
-    return await NatsTaskProducer.connect(
-        url=settings.ingest_bus_url,
-        tenant_id=settings.tenant_id,
-        num_replicas=settings.ingest_bus_num_replicas,
-    )
+    return await ProcrastinateTaskProducer.connect()