fix: address PR #836 round-3 review (lock-key invariant, single open)

🟡 Document the _doc_queueing_lock ":" delimiter invariant (user_id and the
   controlled doc_type enum are colon-free, so the key is collision-safe; a
   future doc_type with ":" must not be added).
🟡 API pod no longer opens the procrastinate connector twice on startup: add
   ProcrastinateTaskProducer.ensure_schema() (applies the schema on the
   already-open pool) and have both lifespan branches build the producer then
   ensure_schema — one open/close cycle, matching the worker. build_producer now
   returns the concrete producer type.
🟢 Document in ports.py that a long-lived-connection producer may optionally
   provide drain() (lifespan probes via getattr).
🟢 Add a unit test that a non-credential pipeline error propagates (for
   procrastinate's RetryStrategy) and still closes the client via finally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Chris Coutinho
2026-06-03 15:32:16 +02:00
co-authored by Claude Opus 4.8
parent 820b98dac1
commit b10ce15032
5 changed files with 75 additions and 25 deletions
@@ -290,7 +290,15 @@ async def get_ingest_job_counts(app: App | None = None) -> dict[str, int]:
def _doc_queueing_lock(task: DocumentTask) -> str:
"""Per-document enqueue-dedup key (partial-unique on ``status='todo'``)."""
"""Per-document enqueue-dedup key (partial-unique on ``status='todo'``).
Collision-safe with a raw ``:`` delimiter because the first two segments can
never themselves contain ``:``: ``user_id`` is a Nextcloud username/UID (no
colons) and ``doc_type`` is a controlled enum (``note``/``file``/
``deck_card``/``news_item``). The trailing ``doc_id`` may contain anything —
it's the final unambiguous segment. A future ``doc_type`` containing ``:``
would break this invariant, so keep doc_type colon-free.
"""
return f"{task.user_id}:{task.doc_type}:{task.doc_id}"
@@ -328,6 +336,15 @@ class ProcrastinateTaskProducer:
# advances after a successful index), so this is not a lost update.
logger.debug("ingest.already_enqueued key=%s", key)
async def ensure_schema(self) -> None:
"""Apply the ingest-queue schema on the producer's already-open pool.
Lets the API lifespan provision the schema without a second open/close
cycle (it already opened the connector to build this producer) — the
``worker`` command shares the same single-open pattern.
"""
await _apply_ingest_queue_schema_open(self._app)
async def job_counts(self) -> dict[str, int]:
"""Ingest job counts by status (for the vector-sync status surface)."""
return await get_ingest_job_counts(self._app)