feat: replace NATS ingest with procrastinate Postgres queue (#183)

Re-architect document ingest from the shared NATS-glued document-processor to a
per-tenant, in-process model owned by nextcloud-mcp-server (Deck #183). The MCP
server now owns both sides of ingest:

- Producer (api role): the scanner defers one job per changed document into the
  app's Postgres via procrastinate (queueing_lock dedup; no execution lock, so a
  crashed worker can't deadlock a doc — Qdrant upserts are idempotent).
- Consumer (worker role): `nextcloud-mcp-server worker` drains the queue and runs
  the existing process_document pipeline; a periodic task reclaims jobs orphaned
  in `doing` by a crash.

INGEST_QUEUE selects the transport (auto: postgres when DATABASE_URL is Postgres,
else the in-process anyio queue for SQLite/dev). procrastinate manages its own
tables (applied on a fresh DB at startup and by `db upgrade`). The vector-sync
status surface reads job counts from Postgres in postgres mode. procrastinate +
psycopg3 ship in the [postgres] extra; the app's own engine still uses asyncpg
(driver unification is a follow-up handled in the rendered Helm chart).

NATS JetStream, the Postgres-queue stub, the bus status subscriber, and nats-py
are removed.

BREAKING CHANGE: the external-NATS-ingest env vars are removed
(INGEST_MODE, STATUS_BACKEND, INGEST_BUS_URL, INGEST_BUS_NUM_REPLICAS,
FACT_EVENT_EMITTER). Use INGEST_QUEUE (memory|postgres) and the `worker`
command instead. TENANT_ID is retained (no longer NATS-subject-charset-validated).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

nextcloud_mcp_server/cli.py

@@ -281,6 +281,69 @@ def run(
     )
 
 
+@click.command()
+@click.option(
+    "--concurrency",
+    "-c",
+    type=int,
+    default=None,
+    help="Max concurrent jobs. Defaults to VECTOR_SYNC_PROCESSOR_WORKERS.",
+)
+def worker(concurrency: int | None):
+    """Run the ingest worker (Deck #183).
+
+    \b
+    Drains the per-tenant Postgres ingest queue (procrastinate): for each
+    deferred document it fetches the content as the owning user, parses, chunks,
+    embeds, and upserts into Qdrant. This is the scale-to-zero ``worker`` role of
+    the api/worker split; run it as a separate Deployment from the API pod.
+
+    \b
+    Requires INGEST_QUEUE=postgres (a PostgreSQL DATABASE_URL); procrastinate is
+    Postgres-only.
+
+    \b
+    Example:
+      $ export DATABASE_URL=postgresql+asyncpg://mcp:mcp@db/mcp
+      $ nextcloud-mcp-server worker -c 4
+    """
+    import anyio  # noqa: PLC0415
+
+    settings = get_settings()
+    if settings.ingest_queue != "postgres":
+        raise click.ClickException(
+            "worker requires INGEST_QUEUE=postgres (a PostgreSQL DATABASE_URL); "
+            f"resolved INGEST_QUEUE={settings.ingest_queue!r}"
+        )
+
+    from nextcloud_mcp_server.vector.queue.procrastinate import (  # noqa: PLC0415
+        INGEST_QUEUE_NAME,
+        apply_ingest_queue_schema,
+        get_procrastinate_app,
+    )
+
+    workers = concurrency or settings.vector_sync_processor_workers
+    app = get_procrastinate_app()
+
+    async def _run() -> None:
+        # Defensive apply (the always-on API pod is the authoritative applier).
+        await apply_ingest_queue_schema(app)
+        async with app.open_async():
+            click.echo(
+                f"Ingest worker started: queue={INGEST_QUEUE_NAME} concurrency={workers}"
+            )
+            await app.run_worker_async(
+                queues=[INGEST_QUEUE_NAME],
+                concurrency=workers,
+                install_signal_handlers=True,
+                # Drop succeeded jobs so the queue table stays lean and the KEDA
+                # queue-depth metric reflects only outstanding work.
+                delete_jobs="successful",
+            )
+
+    anyio.run(_run)
+
+
 @click.group()
 def db():
     """Database migration management commands."""
@@ -374,6 +437,21 @@ def upgrade(database_url: str | None, database_path: str | None, revision: str):
     try:
         click.echo(f"Upgrading database to revision: {revision}")
         upgrade_database(url, revision)
+        # Apply procrastinate's ingest-queue schema on Postgres so a one-shot
+        # migration/init job provisions everything the api + worker roles need
+        # (Deck #183). Idempotent + lazy import (Postgres-only extra).
+        from nextcloud_mcp_server.config import is_sqlite_url  # noqa: PLC0415
+
+        if not is_sqlite_url(url):
+            import anyio  # noqa: PLC0415
+
+            from nextcloud_mcp_server.vector.queue.procrastinate import (  # noqa: PLC0415
+                apply_ingest_queue_schema,
+                build_app_for_url,
+            )
+
+            anyio.run(apply_ingest_queue_schema, build_app_for_url(url))
+            click.echo(click.style("✓ Ingest queue schema applied", fg="green"))
         click.echo(click.style("✓ Database upgraded successfully", fg="green"))
     except Exception as e:
         click.echo(click.style(f"✗ Upgrade failed: {e}", fg="red"), err=True)
@@ -483,6 +561,7 @@ def cli():
 
 
 cli.add_command(run)
+cli.add_command(worker)
 cli.add_command(db)